A few years ago some neighbors moved out and their resident mice spread-out looking for new sources of food.
I'm married and was strongly encouraged to try live-catch traps.
We had reasonably good luck with Tomcat Tomcat® Live Catch Mouse Traps. The concept is basically a teeter-totter tunnel with bait at the "up" end and a door at the "down" end. The mouse enters the "down" end and goes to the "up" end to get the bait, but when they get to the "up" end their weight causes it to go down allowing the door at the other end to close. Other brands have comparable live traps.
It was important to ration the bait to make the weight balance as delicate as possible, and to put the trap on a surface where the door could easily close when that end went up. It was also a good idea to check the traps frequently--the little critters don't have bathroom facilities in there and after awhile...
I read somewhere that if you don't want the mouse to find its way back into your house you should release them at least 2 miles (3+ km) away from your house--as a courtesy to other people you should release it far away from any homes (and, I guess, businesses).
Especially small mice, unfortunately, were able to go into those live-catch traps, eat or steal the bait, and leave without tipping the trap so I was eventually authorized (I'm married!) to switch to Tomcat Tomcat® Press 'N Set® Mouse Traps.
Also, we did catch a couple little ones with one of those copper hemisphere cage live traps (similar to this). That trap wasn't as successful but it was more picturesque, did catch a couple of mice, and the little buggers looked so silly in it. 
The best bait turned out to be peanut-butter or unsalted, not oiled nuts. I placed the traps in areas where we knew the mice visited, along likely travel paths (like along the base of a wall or on a shelf where there was anything they'd eat).
You have to be persistent. Plan on keeping traps out for a few months after you think you got all the mice. And if you are in a building then plan on keeping traps out forever, or at least until everyone has been successful in catching their mice.
Cats are good, too! 
_________________
When diagnosed I bought champagne!
I finally knew why people were strange.