Joi Ito talks about photographing strangers.
I added a comment (trackback breaks on most web sites i visit) about this. I'm not entirely comfortable with it when it's not requested and i may well chase you for more than a few minutes if i caught your lense pointing at me - or worse my family.
I like photographs that can portray a culture in almost one shot, but there seems to be a fine line when it is a close up of one or two people (why, i dunno). I would always ask - you can argue you are too shy to ask, but to take close up photos of you can't be too shy.
Imagine a paperazzi based on your best (or worst) photo ever. Does a still image really catch who you are - does it matter? If your image becomes a hit for good (or bad) reasons and it was a stranger that took it what is the repercussion? If you get your 15 minutes and it is a cool/uncool/nasty/funny etc all based on a single image would you take that chance?