It's kind of ironic that the people complaining about this phrase are coming from a somewhat conservative perspective, because when the meaning of the phrase is explained, they often respond with something like "well, I took it as an insult, so therefore it is an insult."
That's the exact same line of thinking that they accuse "PC liberals" of using when they're offended by something. Basically what both groups are saying is "it doesn't matter what you were actually trying to say. All that matters is how it made ME feel. Therefore, I have no responsibility to understand what was being said before I respond, but you're obligated to confirm your language to my feelings."
I'm not calling out anyone in particular, it's just something I see people do all the time.