I don't mean to jump on anyone, and I'm pretty much the most non-confrontational person you'll ever meet. But I'm also not someone who will stay awkwardly silent when faced with a 'bad joke' like that, and I also have the right to express why I feel it was inappropriate.

If people brushed off every single racist, sexist, or bigoted remark with the excuse that it's 'just a joke', and that made it okay, then this would be a very sad world to live in.