Strange question! Is this an issue?
I suppose I sometimes can be seen in public wearing them. I have sometimes been seen bikeing in them in Spring / Autumn. Perhaps I have even been to the supermarket getting ice-cream in them.
If someone is offended, I couldn't care less...
I saw some Patton Oswalt standup the other night and this was one of the jokes. To be fair, he was mocking himself about it. I think it's like Khem said, people insinuate that it's white-trash-y. I was just interested if it was universal, regional or American, etc. etc.
Plus, you know, I couldn't think of anything better for the daily poll that day.