Not vegetarians in general. But there are certain largely vegetarian castes and communities, a majority of whose members have consistently shown bigoted and casteist behaviors towards the lower caste members, so much that those entire communities and castes are considered that way by the lower castes. Of course, exceptions are always there - it's likely that even a large majority of those castes aren't hostile to non-vegetarianism, but a small vocal minority is extremely hostile to compensate for that.

This reflects in other ways. For example, it's well known that in many cities, only certain castes are allowed to rent or buy apartments within certain housing societies and colonies, the excuse used being vegetarianism (and that non-veg cooking stinks, etc.). Many of my friends who are non-vegetarians have to stick to vegetarian food at home, and only eat non-vegetarian on the occasional outing.