It can seem a little odd.
We can get a lot of information from body language -what mood the person is in (relaxed/tense/happy/angry/etc), how they feel about us (i.e. if that person likes or dislikes us), if they are our social equal or not (e.g. two friends meeting up for coffee will have relaxed and mirrored body language to each other, while if you are a formal meeting with your boss the body language will be different and show the boss to be in charge.)
We rely on body language a lot on a subconscious level, so if someone does not display any body language we losing out on a lot of information that comes instinctively to us and can feel uncomfortable. It's the same if the wrong body language is displayed e.g. if I dropped my purse in the street and a stranger picked it up and returned it to me, that would be a nice thing to do. But if he returned it by standing too close to me with his shoulders tensed I would feel threatened even though his intentions were good.
I hope this answer is okay. I've probably missed out on some things.