Skeletons. The blasted things. If there's any mob in the game that probably just ENJOYS annoying the hell outta the player, it's those jerks. Them and their stupid bows. And their accurate aim. And their ability to knock you off of cliffs with arrows that you didnt expect because the stupid thing snuck up on you.
As far as which mob is the most DANGEROUS, ahh, I'm gonna go with the Guardians. You have to be underwater to deal with them, you pretty much have to use a sword, as a bow just wont work down there, they have the "spikes" ability where you take damage whenever you hit them, they can move fast and will do their best to keep a distance from you, and they have an undodgeable deathray that can only be stopped by breaking it's contact once it's latched onto you. Not that that matters much, because the horrid thing will just lock another beam onto you ASAP. Oh, and they appear in hordes! And there's boss versions! 3 of them in each temple. And you cant break blocks so long as even one of those 3 still exists nearby!
They're definitely an "endgame" sort of mob, and fortunately only appear in ocean monuments.
Other than that.... maybe the Witch as well. If they manage to get too close, you're in trouble. All the armor in the world wont protect you from poison and debuffs. And while you're poisoned and debuffed, skeletons then come and shoot you, because that's exactly what those jerks WOULD do, isnt it...