When you mine asteroids in orbit on a large scale around your star, the released dust/debris would form an IR halo around the star that would be very easy to detect and reveal your presence but we dont see any of it.
Your point on missing IR halos is valid, but don't overlook anomalies like Tabby's Star (KIC 8462852) [0]. Its odd dimming led to theories about alien megastructures like Dyson Spheres, though dust or comets are possible explanations. Still, Tabby's Star highlights the difficulty in excluding advanced alien activities with our current tech. [1]
This person [1] ran a data search for stars with a similar light profile (“slow dippers”) to Tabby/Boyajian’s Star, and claims to have found a cluster of similar stars in the region. But the results are not particularly high confidence and are probably just data artifacts.
I don’t think we’re going to build rock crushers in space. With all that available energy it’d make more sense to just throw the whole rock into a smelter and fractionate the elements as they boil off. Why waste the slag either? You need all the material you can get so hang on to it and use it as ballast or extract the carbon and silicon from it. It’s more likely that we don’t see waste because there isn’t any, a dollar saved is a dollar earned.
So, early humans it propably is.