Good on Apple for fixing this quickly. Even the best team can let a bug like this slip through, and the best solution is a fast response. However, I'm confused by the blog post implying that the solution to unbricking the apps was somehow novel or praiseworthy. This is pretty much textbook for an auto-update system. You just bump the revision to force an application update; there shouldn't be any need to reinstall or muck with the data files. I'd honestly have been shocked if they hadn't been able to handle it just the way they did (which would have been noteworthy, but not in a good way).
However, I'm confused by the blog post implying that the solution to unbricking the apps was somehow novel or praiseworthy.
In the 4 years of the App Store we've never seen a distribution problem of this magnitude (it would be an absolute nightmare for any devs affected) and we really have no idea how Apple would respond to such a problem.
We've also never seen Apple unilaterally update a specific group of apps like this. Many App Store devs likely didn't think Apple would or could take such action, and those of us familiar with what it takes for a device to accept and install new bits are now wondering just how this is being done.
Are they manually bumping each affected app's version string? Is there there some hidden field that forces a device to reinstall the same version of an app? It's curious.
There's a difference between being curious about the exact mechanics and considering the activity itself novel. Apple manages the updates and signs app bundles that developers upload. They need to be able to redeploy packages and trigger updates just to deal with normal operational issues (eg. malformed bundles, versioning mistakes, or reversions). Those are just the operating expectations of any system like this. There's nothing unusual about Apple being able to competently manage a relatively straightforward system that's such a big part of their business.
My theory: there's a version number that devs set on an app, and there's another version (say an integer starting from zero) that Apple puts on apps. They bumped their version, leaving app's version intact.
I'd say that's a good theory, and if they baked it in from the beginning, it's a great solution to this sort of problem.
The fact that Apple is so strict about monotonically increasing dotted numbers for version strings led many of us to believe that they were being parsed for important things inside the App Store publishing platform.
But maybe they are just part of the UI. Apple is known to have strong opinions and strict adherence requirements about that, too.
Do they need to update any part of the app on the devices for this? I think it could be something as simple as "if a device checks for updates, check whether it (might have) gotten a faulty binary. If so, lie to him that there is a new version, and send them a version he already has, but use a correctly signed binary this time".
Marco used the word "interesting", not "novel". Nevertheless, it is "novel" in the context of the App Store because, as far as we know, it has never been used before and none of us know how it works yet, nor that it could be done at all.
And it is "interesting". Apple did not bump the user-visible version strings (which are set by the developer), which would have been easy and expected, but cause some minor confusion for devs.
They didn't even bump the semi-invisible fourth field in the version string, which almost no devs use, and would have solved the problem quickly.
They also did not just mark transfers as having failed and requeue the downloads, which would have been fairly unsurprising.
They were able to determine which apps were corrupted, of all the updates in the affected period. This shouldn't have been too hard, but it shows that they are optimizing their solution pretty well.
They were apparently unable to determine who got corrupt copies of the affected apps and who did not, or maybe they are just erring on the side of caution here. Since their reupdate mechanism seems to work so well, the extra caution costs nothing.
I don't know how many kinds of catastrophic failures you've recovered from, but at App Store scale, recovery is often hard. They were either prepared for this sort of problem, or figured something out quickly that resolves the issue quite cleanly, taking care of all the details.
I guess this might seem impressive if you're unfamiliar with automated client software updates. But this really is the norm when you're packaging, signing, distributing, and updating software bundles. You're going to treat the bundle you get as relatively opaque and not rely on its data. First, it's not reliable to trust third-party data. Second, it's just easier because you own the metadata wrapper (for the signatures, etc.) and the update channel.
It's a special case of automated client software updates, though.
Most of us are probably more familiar with systems like Firefox, Chrome, or Sparkle. The App Store mechanism is more complicated, and interesting.
The third party data (version string) that you don't want to trust is, in this case, very carefully screened by Apple as part of the submission process. It is trustworthy by the time it lands at the App Store -- but since Apple owns the whole workflow, they have much better options.
