To be honest, this guy sounds like a busy person.
I don't think he dislikes you; I just think he has lots on his plate. You are not his priority--but you are not last on his list, either.
Hopefully, you could take the opportunity to meet other people at the events this guy invites you to.