And... what if you don't have a smart phone? Did you forget that there are other kinds of cellphones too?
The issue is that the telephone system basically has no security/privacy/abuse defense mechanisms, and phone companies have done nearly nothing to fix that in the last 100 years. Well, unless your scam is stealing from the phone company (ala phreakers). Then maybe they'd try to fix it.
And if they spoof a number that’s on your contacts list? A false sense of security is not the same thing as security. The solution is action by the network providers.
This idea doesn't work for landlines, of course.
For legitimate new callers, they'd be shunted just once.