Nuclear weapons haven't even been around for a century yet and we've already had multiple extremely close calls that were adverted by sheer luck, in one case a Russian officer openly disobeying standing orders. The Russian invasion of Ukraine still has a non-trival risk it will eventually turn nuclear.
Looking at nukes and trying to use them as an example of our ability to control existentially dangerous technology is mind boggling myopic.
Myopic or not, it is what happened. You can't stop sovereign state's willingness to develop a technology if they concluded that benefits coming from the technology is so huge compared to the cost. North Korea was sanctioned to the oblivion but they're not stopping. India is doing fine.
There is a treaty completely banning existing nuclear weapons as well as developing new one (TPNW); but none of permenant member of UNSC ratified it. Would the U.S. itself ratify the hypothetical AI-prohibition treaty? I doubt it.
P.S.: The AGI being "existential threat" practically doesn't mean anything. I can already imagine arguments against the ban from diverse perspective; Libertarian, Communist ("Marx predicted this centuries ago!"), Socialist, Developmentalist, Decolonization theoriest, etc. Taiwan and South Korea will argue that it will remove their silicon shield; China will regurgitate that they have the right to development; Americans will claim it's not what founding fathers thought in late 18th century. Their concerns are all existential from their own viewpoint.
Looking at nukes and trying to use them as an example of our ability to control existentially dangerous technology is mind boggling myopic.