Well, for the record you ARE weird. We all are, in the sense that we like things that that are well outside the range of what most people like. As for why some people think this particular sort of weirdness is bad, as opposed to just unusual, it's mostly because our weirdness has to do with sex.
People will look at someone with unusual tastes in clothes, food, or movies, and just say "well, it takes all kinds." But unusual sexual interests make people uncomfortable. They get this reaction on a deep level, below the ability of most people to analyze, so they don't even really know why it freaks them out. It's just "weird."
My own theory is that it's because sex is very personal, and necessarily interactive. You can think of someone wearing loud clothes, or listening to music you don't like, and you don't necessarily think of them doing that WITH anyone. In particular, you don't think of them doing that with you. But if you run across someone whose sexuality seems strange then you do sort of have to think of them doing it with someone else, and that someone could be you or someone you care about. That makes their "weirdness" more personal, and harder to shrug off.
Also because sex is so deeply personal, it seems to speak to some sense of what kind of person you are. It's a more intimate and evocative part of you. If you liked peanut butter and dill pickle sandwiches then most people would pass that off as a sort of "surface weirdness." It's a strange habit, like collecting doorknobs. But it doesn't label you - it doesn't say anything deep about the sort of person you are. People are inclined to think that any otherwise ordinary person might possibly like pickles with their peanut butter.
Sex isn't like that. Sex is part of your core. If you're "weird" sexually then you're alien. You might be weird in all sorts of other ways. It makes it hard for some people to trust you, because they just can't understand how your mind works.
I think this is why unusual sexual interests of all kinds tend to be received poorly by people who can't separate out what they like to do from what other people like to do.