For many years my "big question" has been "why do software engineering projects fail at a disproportionally higher rate than other engineering fields?"
Though I have to say the other engineering fields are not doing so hot lately either, so I might have the wrong question.
After 20+ years in the field: main reason is unclear requirements. Software malleability is taken for granted so much that the under specification and fluidity of requirements became a norm and canonicalized in “agile”. That’s not necessarily a bad thing, it does allow for quick advancement of what is possible but high failure rate is the price we pay. Second: unlike traditional engineering, there are not too many university or college programs that take rigorous approach to building software at scale. Till today, software is a free-for-all industry. That allowed us to grow very fast, but - again - high failure rate is the price we pay. Third - lack of formal standards and by that I mean not just APIs or protocol specs but formal requirements on things like performance, fault-tolerance, lifespan etc.
Also, bugs in software are seen as a fact of life. You can't get warranty on software (at least this holds in practice for most consumers). If MacOS deletes all your files, good luck proving that it wasn't your fault, and good luck suing Apple.
So this gives programmers a carte blanche to create crap, effectively.
That's a very broad big question. Like, do they really fail more? What's the numerator and denominator? There are are tons of high-profile failures, but there are innumerable successes as well. Do you count pushing the 100 or so versions of Chrome that have been pushed out, across millions of devices as one success, or a billion? How do you count projects that don't work right at launch, but do end up working okay after a couple versions?
I once saw someone complaining because they moved into a freshly-built apartment building and things were constantly wrong. A/C not working, shower not working, every time it came down to the contractor doing something wrong.
but the apartment building was finished, the project for building it didn't fail.
In the same way, software projects don't typically fail either, they just have a million things wrong with them.
I don't think I've ever actually seen a software project outright fail, it's more that it never lived up to expectations.
Though I have to say the other engineering fields are not doing so hot lately either, so I might have the wrong question.