Many sayings and phrases people used aren't meant literally, and quite often bear little relation to the reality of the situation. A common saying in the UK when relating a past conversation to someone is "I turned round to him/her, and I said...". At the time, both people were probably facing each other, and nobody turned around, but still the saying is used, quite often repeatedly in the same story.
"I don't know what planet you're from" is usually used to back up a perceived "right" way of doing something. For example, "I don't know what planet you come from, but where I'm from, everybody holds their fork in their left hand while eating". It's used to suggest that any other way of doing a thing is alien and wrong.