Good question! If I'm sat listening to someone in person spinning a yarn then I have no issues, although story-telling seems oddly different to a written story somehow. I don't really listen to audiobooks because I find it quite hard to concentrate on them, but I am curious as to whether it would work better there. Maybe it hits some sort of weird mid-point?
Could be worth a shot. I used to strongly dislike first person narrative, but it is less of an annoyance now, and some of my favorite books are from that perspective. I think it was partly reading and discussing some of my friends' favorite books (e.g., The Perks of Being a Wallflower), where their enthusiasm helped me appreciate it, and partly listening to good readers (e.g., Martin Jarvis reading Wodehouse or The Dresden Files read by James Marsters).
It might also have helped to read first person narratives where the narrator was more observer than actor (e.g., The Great Gatsby).
