to be honest i've caught myself doing the same thing, mainly because I was bored... however, i've found that it can turn into a good conversation that throws the boredom straight in the bin.
Back in college I rode the train to and from school. This trip was usually about 25-30 minutes, so i'd get out my laptop and do some coding, simple maintenance or playing solitaire. Once there was this older woman sitting beside me while I was writing some java code for a hobby project... after a while she asked me what I was writing, and she seemed very interested in what I was doing. I stated it was a simple calculator with some added stuff for some things we were going through in maths and explained a bit about how it worked. She told me her son was a programer at some company and that he occasionally would code some small games for her as gifts.
we met on the afternoon train a few times a week for about half a year or so before she had to move out of town, talking while i coded some simple, funny programs for her...
While I think it's rude while someone is writing private stuff (like IMs, private messages and emails), I don't think it's rude while reading a book, a newspaper or something that isn't private...