I put it as a slash because metal sub-genres can get so granular that its insane and I dont always know one from the other. They would be slightly different from my understanding. Math metal being more rooted in metal, and mathcore more in hardcore. Its characterized by odd time signatures, unconventional composition, chuggy and or experimental/noise guitar (effects/scrapes/slides), many changes, often with few or in some cases no repeating parts. It's like ADD music. Polyrythmic drumming, and in the case of mathcore is almost exclusively screamed rythmically and any clean singing is sometimes intentionally flat or poor or overprocessed. Most people, even metal fans hate alot of this stuff because "thats not how you're supposed to play your instrument/write songs" or "its just noise". Which it sorta is, but in it's best it well constructed evolving noise unconfined by conventional song structure or expectations. And very energetic, sometimes real deep grooves, and always pissed off.
Some more well knows examples would be:
Meshuggah (Djent and math metal sorta cross over in some areas Periphery qualifies Id guess tho not my thing)
The Dillenger Escape Plan
The Tony Danza Tapdance Extravaganza
Norma Jean
The Chariot
The Red Chord (some call deathcore again subgenres dicey)
Iwrestledabearonce
Candiria
(Old) Mudvayne
Car Bomb
Torrential Downpour
Into the Moat
Ion Dissonance
Barrier
Arsonists Get all the Girls
Veil of Maya
It's relatively recent. The forefathers of the genre would have to be the likes of Candiria and Meshuggah wo both cam out in the early 90's but didnt get noticed until later in the decade. Dillenger Escape Plan would more or less be the template if you were to tell someone one band that defined the sound.
Tool is sometimes called math metal, but O dont see how they aremetal at all really.