Yup.
I'm very much an INTP and have a great deal of difficulty with math, algebra, trig and calc. I went through 4 years of this stuff in high school and can't remember a darned bit of it. The only thing I had any success with was geometry and I think that is because I could visualize it.
I also don't like word games or chess, but then I'm not a big fan of rules. I could be wrong, but my sense is you have to understand rules of a game or of grammar to get the most out of these sorts of games.
My preference, much like you describe of yourself, is to take things apart, figure out how they work, gain some level of skill in that area and then move on. My skills are of a more visual and tactile nature. My father (an ISTJ), who is very good with math, seems to be able to look at a formula and understand how it applies to the real world. To him, the mathematics of physics makes a lot of sense. I enjoy physics from the conceptual angle, but cannot make heads or tails of it if I look at the math.
If I had to guess what math was, it is the purest abstract description of reality. I understand and can relate to the world around me by looking at things, visualizing how they interconnect, taking them apart and feeling how one mechanism interlocks with another to create a machine. I can hold these visualizations in my head and I can think visually about these things. But even something as simple as diameter x pi to find the circumference of a circle only makes logical sense, but doesn't really help me have a more accurate concept of the world.
Mysteries don't really work for me either. Maybe I'm unlike a lot of Aspies in that what appeals to me most in a story is the subtle complexities of human interaction and emotion. But i could never get into Sherlock Homles.
I like analogies more than algorithms. I guess I understand the world better by comparing one tangible thing to another tangible thing. Abstractions like math or algorithms are too far removed from the thing itself for me to be able to hold it in my mind and compare it to the actual thing.
Lars
_________________
Never let the weeds get higher than the garden,
Always keep a sapphire in your mind.
(Tom Waits "Get Behind the Mule")