I rarely address people, too. I also have extreme difficulty remembering people's names. Actually, in a given classroom, I remember the names, but not the faces they're attached to!
I wonder why Aspies often have this difficulty with people's names, what we know of the brain's functioning and Asperger's that would account for this, as I see it very often, for instance, on forums like this one.
_________________
"There are things you need not know of, though you live and die in vain,
There are souls more sick of pleasure than you are sick of pain"
--G. K. Chesterton, The Aristocrat