Seems kinda natural to me. You thought there was an attraction and it turned out there wasn't. These are emotions, and you can only control them up to a certain point. As well, any relationship is going to have at least one person who dislikes the notion, for a variety of reasons. Anyone watching on the sidelines can say anything they want, but at the end of the day, you are the one having the experience and you are the one who has the emotions. Be honest with yourself, your emotions, and the other person in question. Yes, all of this may hurt currently, but imagine how much more guilt and pain will build up if you stall this point for a long time. Hope all goes well!!