I have a large collection of baseball books, and play baseball text-simulations. But it's the history, and strategy that I find interesting. Much of the time watching the game itself bores me. If I'm around a bunch of (non-annoying) people then OK, or perhaps if turning inti a tie-game in the seventh, but otherwise I rarely watch a game.
The game I most enjoy watching as a spectator sport is hockey. (American) football I might also watch depending if the game is important. I don't play any sports myself unless you can't count occassionally shooting a few hoops by myself.