Catster - I too think your instincts were correct and you handled it well.
With advances like that (conversational) you can just be firm with what you think is appropriate (including his wife, meeting in public). If he did just want to be friends, he would have made that clear when you invited his wife. If he wants an affair, he shuffles and acts silly. I have a few friends now that I met in odd places and their initial, "Do you want to get together for coffee?" could have been an advance, but wasn't. The only way to figure it out was to ask more questions. If they really get obtuse, be blunt - "Are you asking me out on a date?"
_________________
Apologies if I sound judgmental, preachy, dictatorial, offensive or overly rigid. Constructive criticism via PM is welcome.