Man, this disappoints me. I was a tech lead on the packages project about 3 years ago, specifically on the redesign for OCI container support and making anonymous downloads of public packages as reliable as possible was a top priority.
If that flow was broken there’s only a handful of things it could be; specifically azure blob store or azure MySQL, but both of those should have layers of redundancy. Public anonymous download bypasses most everything else; no auth services, rails monolith, or metered billing. It does emit some events to the message bus for metrics, but that’d effect much more than packages if there was an issue with it.
As far as I’m aware this is the first time anonymous public package download broke since I left a few years back.
Fun fact: Linux distributions (and some older open source programming language package managers) use hundreds of mirrors distributed around the world to distribute their assets. If any mirror goes down, you just pick a different one. Even when they could just use SourceForge as a mirror (formerly the largest repository for open source software), they still used hundreds more mirrors. Distribution was made easy with rsync, and mirrors could choose what files they mirrored (just the latest release, or all releases, or just binaries and not source code)
GitHub uses pools of mirrors, too, they're just transparent.
twice this year I've had to spend an hour or more with a user because a mirror was down. that's one more time than I've had to deal with a GitHub packages outage this year.
the latest was yesterday. they chose another pool of mirrors and the mirror continually chosen from that pool was down as well. finally I manually checked a mirror, made sure it was up and that signatures matched, then gave them that specific hostname.
the Linux package distribution system is not better. it's just different.
you don't have to, but hb will ask you to if you have a saved credential because your quota on GitHub for downloading packages is much higher if you are logged in.
anonymous stuff on GitHub is usually limited to 60 requests per hour per ip address. if you're authenticated, it's several hundred if not several thousand.
no, this is GitHub packages, not GitHub repositories. Cargo doesn't use this. It doesn't use GitHub repositories either, they store the crates internally.
EDIT: I was wrong, the crates index does use a GitHub repo
The crates.io index actually is a GitHub repository, so I think a GitHub outage that affected repositories (not just packages) could break Cargo. Only metadata is stored there, though, not the actual crates. I'm not 100% sure why it works like this; there seems to be a plan to change things soon so that Cargo running on users' machines doesn't talk directly to GitHub by default (https://blog.rust-lang.org/inside-rust/2023/01/30/cargo-spar...), though the GitHub repo would still be the source of truth.
crates.io also uses GitHub as an OAuth provider (and it's currently the only one offered), so if that broke then people wouldn't be able to publish crates, though downloading existing ones would presumably still work since you don't have to log in to do that.
That's being addressed with a new index protocol, specified a while ago[1], available for testing since the middle of the last year[2], and slated for release in a week's time[3].
In Python, we don't say "we don't host packages on a proprietary platform", we say "we have absolutely no clue where they are hosted and nobody audits them anyways, and we don't enforce package signing, and we'll just build from source with no build isolation what so ever, unless you remember to specify an obscure command-line option when installing... and have a nice day!"
it's amazing how we never learn. Things that Perl and CPAN and Linux distros figured out decades ago are constant issues today. It wasn't that long ago that NPM didn't even have checksums. CPAN runs unit tests on install. I can't imagine how slow that would be with npm.
Package signing is, well... I suppose that's another lesson from the '90s people will learn about soon enough. With a web of trust as broad as python or npm you'll just have everyone running around with signing keys and "trusting" any key they come across because none of it is built on personal relationships. When Archlinux asks me to confirm adding package keys, what am I going to do? Say no? I don't know these people, but I want my shit to work.
When it comes to my personal laptop, I also, typically, blindly trust the keys coming from developers because I don't have time for that. Not so much if I have to deploy a system into environment that several orders of magnitude more expensive than my laptop...
With systems like Python, I'd imagine that a solution to web of trust would be that some group of developers would organize a curated set of packages. So, for the cases where you need better security assurances, you'd use that. I mean, of course there's no guaranteed solution for the web of trust, but, in practical terms, something like that would be good enough for regulators.
There's already stuff like NumFOCUS. They don't particularly focus on the technical side of things, or endorsing more secure practices, but, in principle, they could. Maybe there will also be others once we have been bitten more times by some security breaches.
Well, we know GitHub is real. But we should also remember that they can change their API at anytime and basing a package manager on their priorities is not the best situation for the long term success of that package manager, unless they are owned by Microsoft
GitHub redirects when this happens. if you move/rename within GitHub, anyway. I don't know how long those redirects last but they probably last until the name is used again.
so it's not quite as bad as you're imagining but still not great.
fortunately GitHub is starting to require 2FA for very popular projects (starting with NPM) because of supply chain attacks like what you describe.
You presume incorrectly. GitHub Packages, the package registry and binary hosting service, does not even support Nix. As for GitHub in general—
The binary hosting at https://cache.nixos.org/ is independent of GitHub, and so are the old-style channels at https://channels.nixos.org/. The new-style flake registry used to be fetched from GitHub but has now been moved to https://channels.nixos.org/flake-registry.json. Admittedly in a new-style situation you’re likely to be using unlocked flake references that refer to GitHub (e.g. Nixpkgs), but it’s on you to lock them and pull them into your Nix store in that case.
Of course, you also get GitHub references for upstreams that host their code there, but that applies to almost any distro(’s build system) except the oldest of the old-timers which host the source for the whole distro on their own infrastructure, like Debian. (I happen think the old-timers are right here, but that’s beside the point.)
This is actually the second (maybe third; I didn't even know about GitHub's outage a week and a half ago so idk how Nix was or was not affected) time in three months or less that a partial GitHub outage or GitHub change has taken down Homebrew while leaving Nix unaffected.
