Currently building most libraries usually involves executing a series of highly complex byzantine scripts, requiring turing-complete environment. This gives attacker an endless attack surface, and when the build process is hijacked - well, the opportunities are there.
Moving to a declarative build process with only a limited state machine as an executor would help. Requiring all source blobs being reproducible might also be something to think about.
This is how we got autoconf in the first place - an under specified and under powered build configuration language (make) which led to people that did need arbitrary logic to generate the build files at configuration time.
Don't limit the build system language. It makes builds more complex, not less.
Having a "simple" declarative build definition won't help if the thing that interprets and executes that definition is shipped with the package, as said interpreter is likely orders of magnitude more complex and harder to personally review. As is what happened with the xz example - the "attack" was hidden in code that is autogenerated by autotools, not the build or configuration definitions.
People put trust in distros and packagers to having something of a review chain - there's 0% chance you personally be an expert in everything executing on your workstation right now (outside of maybe toy systems). I'm not an expert in m4 or bash, but I hope enough experts are in the chain to get to my distro's package library are that such things are less likely. But that is all bypassed here.
I think this particular approach is a one-off, as I know of no other build environment where it's expected to have the generated executable of the build system "helpfully" packaged in the tarball as a packaging step.
If it is in some I'm not aware of, I hope that decision is being re-examined now.
No, the attack was not hidden in code that is autogenerated by autotools. If that was the case, rerunning autoconf (which most distros do) would have disabled the attack.
Instead it was hidden in plain sight in the source that is processed by autoconf.
> it's expected to have the generated executable of the build system "helpfully" packaged in the tarball as a packaging step.
While this is true, most distros as I mentioned above rerun autoconf to ensure consistency in the tests and possibly to include autoconf bug fixes.
The jumping off point was modified autotools output. re-running autogen did effectively disable the attack. The payload was stashed in some test files, but the build system needed to jump through quite some hoops to actually get that into the compiled library. Apparently the target distributions didn't re-run autogen for this package.
This is what the early reporting said but the article has additional info.
The code to include the backdoor in the build was in an m4 script.
The initial reporting said that this code was not present in the github source, but the post-autogen code (including the attack) was included in the github releases.
The article says that this modified script was present in the source on tukaani.org, which was controlled by the attacker and used by the distros as their upstream.
If you downloaded from github and reran autogen you were OK. If you downloaded from tukaani and reran autogen, like the distros did, you lost.
the "build-to-host.m4" file seems to originally be from gnulib, and if that is installed on the system is not required by the build. So I see that as "part of the build system" myself.
I mean the github repository with exactly the same .ac and .am files works fine with local automake/autoconf generation without that file existing. And thus no backdoor (the test files are still there, but "harmless" without the initial kicking off point to actually de-obfuscate and include their contents)
Gnulib is not installable, it is meant to be copied (aka vendored) in the sources of the program that use it.
> if that is installed on the system is not required by the build
This specific file defines a macro that is used by autoconf, not by the build. If it is installed on the system it is not required by autoconf, but then gnulib is practically never installed.
Your original message blamed the backdoor on "the generated executable". This m4 file is not a generated file and not an executable. It is simply vendoring like you often see in other languages.
I think it was more "hiding" as vendored code rather than really being in that category. The git repo never contained that "vendoring", as the m4/gettext.m4 file doesn't exist autoreconf just copies one from a system store, (which on my machine never calls the tainted BUILD_TO_HOST macros in the first place, which also doesn't exist in the upstream xz git repo).
"Vendoring" by copying untracked files into the tarball seems discourteous to the definition. It seems to rely on the "possibly odd" behavior of autoreconf to allow files that happen to have the same name to override system-installed versions? I guess on the belief that local definitions can override them is useful? But that certainly bit them in the ass here. As to get a "completely" clean autoconf rebuild it looks like you have to delete matching files manually.
For example if you've ever taken a look at the bluetooth specs you would not trust a single person in the world to implement it correctly and you probably wouldn't even trust an arbitrarily large team to implement it correctly.
Unless they had a long demonstrated and credible track record of shipping perfectly functional products and an effectively unlimited budget, i. e. Apple and maybe 1 or 2 other groups, at most.
> For example if you've ever taken a look at the bluetooth specs you would not trust a single person in the world to implement it correctly and you probably wouldn't even trust an arbitrarily large team to implement it correctly.
I messed around a tiny bit with Bluetooth on Linux recently. Going to rankly speculate that Bluetooth is such a special case of hell such that it makes a distracting example here.
I mean, as a joke suppose we wanted to design a 3.5 mm patch cord that pitch shifts down a 1/4 step for randomly chosen stretches. It turns out to be easy-- just remove the wire from the casing and replace it with cheap bluetooth chips at either end. You'll get that behavior for free! :)
Compare to, say, USB, where your point above would apply just as well. I wouldn't be distracted by that example because even cheapo, decades-old USB drives to this day let me read/write without interleaving zeros in my data.
Shit, now I'm distracted again. Does Bluetooth audio even have the notion of a buffer size that I can set from the sender? And I guess technically the receiver isn't interleaving the signal with zeros-- it's adjusting the rate at which it sends blocks of the received data to the audio subsystem.
Was Bluetooth audio basically designed just for the human voice under the assumption we're constantly pitch shifting?
Oops, I almost forgot-- the sample-rate shift is preceded by a dropout, so I do get interleaved zeros in the audio data! I actually get a smoother ramp in my Prius changing from battery to ICE than I do in my wireless audio system!!!
Yes. Other than the vulnerability of developers/maintainers the other
big takeaway I get from this incident is that build systems have become
dangerously unwieldy. There's just too many moving parts and too many
places to hide bad stuff.
Build systems have always been like this. It is in fact more recent build systems that are limiting the amount of crazyness you can do compared to the older ones.
Currently building most libraries usually involves executing a series of highly complex byzantine scripts, requiring turing-complete environment. This gives attacker an endless attack surface, and when the build process is hijacked - well, the opportunities are there.
Moving to a declarative build process with only a limited state machine as an executor would help. Requiring all source blobs being reproducible might also be something to think about.