Freudian Theory
I actually kind of studied this in school this semester!
There's a theorist from a while back named Sigmund Freud, and he's came up with the best analysis of fetishism so far, at least I think so and so do most other people. But it actually derives psychologically. According to Freud, fetishism happens when a young boy realizes that his mother (or other female figure) doesn't have a penis (we're all adults here, lol), so he directs his attention somewhere else, such as a foot or something like tickling that he knows that both of them have in common.
I know it's not the most reliable source, but it matches what I learned in school, so here's what Wikipedia says, just search for the word Freud:
Sexual Fetishism
I gave this some thought, and then I remembered that when I was young, I was in constant denial that women didn't have penises... I simply wouldn't believe it! For some reason or another, it was shocking to me that they didn't. So, it makes a lot of sense to me and I think Freuds concept is at least somewhat true.
It doesn't comment on female fetishism, though...
So, in terms of the way we think, I think it is possible that there are genes that cause family members to think similarly, so, it's still quite possible that in some distant way, fetishism has a gene.