Here is a story.I'm afraid it's true.
I've lived in a small town for almost 5 years, I still have no friends, sometimes someone in my family takes pity on me and visits. We usually go out for a drink, I never go out socially if no-one drags me out, I try to keep up with them drinkwise, but I'm not used to drinking outside of my room and my pc, all the social stuff just destroys me. Three weeks ago, my very good friend and ex-BIL(he was married to my big sister) visited me in my small town and I tried to match his drinking, as you do, I ended up asking a barmaid out for a coffee. I'm completely mortified at my actions, it's embarrassing firstly, but I also feel bad for asking her out while she was working, just seems wrong. I may never visit that pub again.
I understand how you feel, I still think you should go for it.
Good reason - you felt those first little sparks at church, very dull and boring place, that counts for something.
If you do ask him out and he runs screaming, then you can tell us and we will understand. If he says 'yes, I'd love to grab a coffee with you' then you can tell us and we can cheer you on.
It's really a win-win when you think about it.