There is an entirely theoretical possibility for this to happen in some universe. It doesn't particularly concern me. Its arguing about how many angels can dance on a pin's head.
The messianic prophecy of the imminence of the in-silico birth of the AGI is based on extreme, simplistic mental models: the brain is a computer, everything is input and output, there is some neural net black box "learning" inside etc.
The stance reminds me of the arrogant ignorance displayed by Laplace's demon [1]. Laplace postulated that if someone (the demon) knows the precise location and momentum of every atom in the universe, their past and future values for any given time are entailed - they can be calculated from the laws of classical mechanics.
Alas, he completely missed quantum mechanics, the very basis of how the universe works. It was discovered a hundred years after he died.
In fact Laplace's overconfidence sin is probably orders of magnitude smaller than what the breathless AGI enthusiasts currently commit: Classical mechanics and quantum mechanics share some formal mathematical similarities [2].
It feels very unlikely that anything resembling AGI will share much mathematical form with current machine learning models. The evolutionary, in-principle agnostic and general purpose "wiring" of the human brain, assuming it can abstracted somehow, lives on a different mathematical plane.
[2] In fact the process of conjuring up the formal apparatus required for quantum mechanics is one of the most dramatic examples of the versatile abstraction powers of the human brain.
The answer actually might be, no, why do you assume silicon can actually do what is required?
Considering we've never seen a brain only made from silicon do much, then it's likely not the right stuff.
Maybe we can brute force things that act like a brain in silicon, but considering that a bee and an ant are intelligent, communicate and solve problems in groups and run on almost zero energy compared to an entire datacenter, you have to admit, something isn't quite there.
I know this is a sensitive topic for many, so I'm not being hostile, I genuinely mean it though.
In my honest opinion, if ants or bees had thumbs and hands, they'd evolve to do some amazing things.
