I think the thing you've left out of the equation in your thinking is that it's a public place where people don't really choose one table for certain and sit like in a restaurant.
Usually, the NT thing to do would be to ask "Mind if I join you, I'd enjoy talking with you some more" or something. THat's one of those things I'd sometimes know to do because of my copying kids like Charlie Brown, but I must admit I'd stay seated like you did more often than not. I would do like you did in the cafeteria at school and just sort of cat with people. Then, I think partly because of my low vision as well, I would just not remember who they were. Which leads to embarrassing situations at times, but most people will be very understanding and chalk it up to short term memory problems
My best friend in college, though, the first day of first year orientation I turned around and said he'd asked a *really* dumb question, and while he thought it rude at first, we struck up a conversation later and found a lot in common. So, don't lose hope, the good friends will understand anything.