I believe you are correct. It's the complex time signatures which gives 'Math Rock' it's monicker. You are also correct that you wouldn't call bebop jazz 'math jazz'. I think part of the Math Rock labeling comes from the techie or geek cultures. Like some forms of jazz, much of this music would've been simply labeled avant garde.
I wonder if it helps, or hurts, to market music within communities with specific labels? Does it cause music to fall off the scene quicker, because a label quickly dates an aesthetic..?
_________________
If you can, help others; if you cannot do that, at least do not harm them.
~ Dalai Lama