I've only had one time that I can recall being annoyed by a cell phone user. I was watching a movie at the theater with my friend. Someone had left their cell phone on during the movie and worse, he actually answered it. The guy did finally stand up and walk out of the room, though he was talking the entire time.
Also, when I'm out with a friend and she gets a call on her cell phone, I don't really like it when she talks to the caller for a long time. I'm stuck sitting across from her with nothing else to do but listen to a one-sided conversation. Since this doesn't happen often, I'm able to shrug it off. So, I wouldn't say this is exactly a pet peeve of mine, just unappreciated.