Moot Point
Cut off the head and the body will die; the reverse is also true.
In the same manner, you can't have a surviving sensual relationship with a person if you aren't physically attracted to them and have at least a minimal amount of respect for them.
You can have a dying or short term relationship with those people, though; and those can be kind of fun.
I mean, myself, I'm a rarity; I won't hook up with a girl to which I'm not physically attracted. I believe that physical attraction is one of the separators between FRIENDS and LOVERS. (I do have a bad habit of hooking up with my hot friends from time to time ... to time, though; but that's another story.)
But lots of guys hook up with girls that they aren't physically attracted to for one night stands or even indefinite sex buddies, and in our generation, whatever a man is doing, you can be sure women are doing twice as much of it.
And who wouldn't love to rail, tickle or otherwise torture a beautiful shrew or jerk. I know you can sign me up!!!!
So, if you want it to last, you're gonna need to have the whole package. Else, just do what feels right.
T