I also do poorly on those tests, and I have hard time with proper use of maps and direction. I get left, right, East, West etc, I just can't easily find my way around, and if we go one way and take some streets back and forth, I have no sense of direction or where we'll end up, and unless I was familiar with the place, I would have to slavishly back track to find my way back. I easily get lost.
I'm also hopeless with seeing things mentally and rotating them in my mind. That, and lack of drawing skills, made technical drawing next to impossible for me.