Well, I wouldn't assume flour is harmless in an industrial setting. For example, there's "popcorn lung", (https://en.wikipedia.org/wiki/Obliterative_bronchiolitis) you don't think of artificial butter flavored popcorn as a toxic terror, but in a factory, it can have severe effects over a period of time.
In positions we talk about the LD50. When it comes to environmental exposure of things like dust to incidents of lung damage, I'm not sure how we measure said effects. Yea, breathing any significant amount of dusts can be harmful, be we should be very concerned for products where insignificant amounts of material can be harmful.
I've wondered about drywall/gypsum dust. The first time I was on a construction site, I was pretty young and nobody was insisting I wear a mask so I didn't at first.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5799097/