Unless you are allergic to either you should not really worry, they do not cause much pain and do not readily sting. Bees in general cause less pain and more itching because they have more histamines and less acetylcholine in their venom, vespids(wasps,hornets, and yellow jackets) have the opposite. This may not be one hundred percent accurate, various sources contradict themselves.
It is pointless to compare bee and wasp stings to spiders or anything else, the pain varies between species. Bee stings cause very little pain at all, paper wasps are kinda meh, yellow jackets suck,white faced and asian hornets suck a little more. If you live out west then pepsid wasps suck a whole lot. As long as you do not go stomping around any hives or colonies you shouldn't have any problem with any of them. If one is near you just go about what you were doing, if one lands on you either don't touch it or brush it off of you, do NOT squish it, if you do you will certainly be stung.
I was stung by bees and vespids so often when i was little that they had no effect on me at all for awhile. Now they do but they generally do not bother me that much. The only two exceptions were when a yellow jacket went into my pepsi can and when i took a drink it stung my tongue and lips like four times, that was unpleasant. The second was something that i have yet to be able to identify, it was similar to a large yellow jacket with a reversed pattern, maybe a little more black. It stung my thumb when i was trying to pick a blackberry.
My first thought was actually " wow if a wasp sting is making me hop around in circles then perhaps i should reconsider getting a copperhead and another subspinipes centipede" rofl. It felt like someone stabbed my thumb with an extremely hot nail and was twisting it, my thumb swelled up to about twice its normal size and my arm hurt up to my elbow for around a day. My thumb hurt and stayed swollen for around a week. Other than those experiences i have never had any worth complaining about, i think most people let things like this take too much pleasure away from them.
Just steer clear of soda cans that have been outside unattended,pepsid wasps,trees turned over in storms(yellow jackets seem to love making nests in the soil that comes up with the roots), and strange yellow jacket like vespids that are slightly too large with extra black.