> I am actually surprised I have never heard of any fatalities.

There are loads of toxic flowers, plants, and trees.

As for your question. I guess because people generally don't live like Into The Wild [1].

[1] http://www.imdb.com/title/tt0758758/