I have two PSPs. One's got 1.5 (for homebrew), and the other had 2.6. A couple months ago, I managed to take the 2.6 PSP and put Dark_Alex's 3.10 OE-A firmware on it. Now I can do 1.5 homebrew, play the latest games, AND play them from my memstick via the integrated ISO loader. There's also a few other caveats to OE-A.
Initially, I only got the PSP for Liberty City Stories and homebrew. Now that I take the train to work, I've got a half hour to burn each way. The PSP then became indispensable.
I've got:
GTA:Liberty City Stories (completed 100%)
Metal Gear Ac!d (finished Extreme mode)
Valkyrie Profile (got all 3 endings, completed Seraphic Gate)
GTA:Vice City Stories (haven't started yet)
Valkyrie Profile is my favorite, and easily one of the best RPGs ever. I really regret missing this when it was out for the PS1.
I've borrowed Exit and LocoRoco from a friend. Exit's pretty fun in terms of puzzle-solving. LocoRoco is a lot of fun, if a bit hard.