I see issues like contracts--they have a scope and you are agreeing to work on that scope. If you go beyond the scope, you're breaking the contract. Someone might already be fixing the tech debt via a different issue, you may need to submit a new RFC, etc. Management and PMs will also start to notice and not appreciate the risk that your fixes might cause to the release.

I think the point is that these tech debt changes are often not really making the code better. Also, when the PR has other things going on, then it's harder to properly give a CR on the actual rason the code was changed. Sure, sometimes a person is able to remove a bunch of unnecessarily complex code or what not as part of the change, which I think are generally good.