> brew update will now be run automatically less often (every 24 hours rather than every 5 minutes) and these auto-updates will be much faster as they no longer need to perform the slow git fetch of the huge homebrew/core and homebrew/cask taps’ Git repositories.
Excellent! Thanks to the brew team for this — it was always frustrating to install something and have to wait for a full, slow update first.
I was just wondering, have they considered storing all that in an SQLite file? And then you could refresh the file every 6 hours, say, and post it on a CDN?
But the JSON API has been around for a while so brew 4.0 just makes use of it by default.
Brew 3.3 added an off-by-default HOMEBREW_INSTALL_FROM_API bool to let you install from brew's JSON-based API instead of checking out the large (and slow) homebrew/core and homebrew/taps repos: https://formulae.brew.sh/docs/api/
Brew 4 just makes that the default and deprecates the env var (you can use HOMEBREW_NO_INSTALL_FROM_API=1 if you still want to check out the git repos for some reason).
> instead of checking out the large (and slow) homebrew/core and homebrew/taps repos
I actually wanted to ask why they don't just prune the history of those repos to cut down on their clone size? OT1H, I could actually imagine some software historian being curious what flags were required to build sqlite 3.6.20 from 2009 (872f50ac61d7) but OTOH building a whole new distribution system instead of effectively git-squash feels weird
As someone who regularly patches those local repos to work around silliness, I was bitten by that newfound API business and I'm thankful that env-var exists.
As someone who's seen The New Homebrew Way, I dread the day my HOMEBREW_NO_INSTALL_FROM_API gets taken away :-(
I believe they used to do `--depth` but GitHub complained because their systems weren't optimized for that and it ended being cheaper for the servers to send you the whole thing (much easier to figure out what refs you needed, or something along those lines).
I don't know anything about how git servers are implemented, but this is super weird to me. Why couldn't you basically just copy the files (excluding .git), then do `git init`, set the upstream remote, and fetch only the most recent commit? I'm guessing it wouldn't work _exactly_ like that because the git cli is convoluted and nobody remembers the exact semantics of every flag, but it seems surprising that there aren't any possible sequences of commands that do that. Even if they implemented it on server side by just making a "fake" client locally doing this, and then sending the tarball of the repo with just that commit, it would be way more efficient than making clients download the entire history.
I wasn't super sure how subsequent `git pull` works with a shallow clone, but if it works the same, then such an obvious fix makes the invention of that API solution even more painful
IIRC, the only thing that will potentially need to be changed is manually fetching the tags, since those won't be brought in. You can just do `git fetch --tags` to get all of the tag metadata though (without actually needing to pull the actual commits they reference; it just gives you the ability to look up a commit hash with a tag so that you can fetch it later).
maybe even just create an aliased sh function that just prepends that instead of typing it out each time.
I've been doing that forever. It really is incredibly annoying how it just keeps endlessly running without doing some gymnastics (yeah sure I could add that env var set to 1 in my zshrc but I mean its just a silly default behavior)
I use homebrew like a black box. I’m glad it exists, but I have no idea how it works and why it’s so complicated. When something breaks and Google is no help I just nuke it and start from scratch.
It’s only happened twice in more than a decade, but I still have the feeling that I was better off compiling stuff manually and cultivating a tidy /usr/local/bin like I did in early 2000s. But it’s probably rose-colored glasses and I’d definitively never take the time these days.
I think you’re right about the rose-colored nostalgia: it’s easy to forget how far behind people got on updates. I remember so many support emails where someone had hit a bug which had been patched a year earlier but never thought to install it. A couple of times where people reinvented something which had been added to PHP in the meantime because they didn’t know it was there.
That’s why they keep it private by default, but the answer is often yes. There have been multiple times where a package like Python 3.10 required a newer version (1.1.1) than the OS release included (I suspect that’s also why the AWS Lambda runtime is stuck on 3.9 since it’s based on Amazon Linux 2 and that uses 1 .0).
This has settled down a bit but the big HTTPS-everywhere push last decade flushed out a ton of things which had been quietly aging in place, unnoticed until someone started depending on SNI, newer crypto suites, actually disabling deprecated versions of SSL/TLS, etc.
Maybe it’s quieted down recently after the HTTPS effort, but I feel like I need to download a whole Unix environment, down to the readline, every now and then when I just wanted to install a trivial thing.
> It’s only happened twice in more than a decade, but I still have the feeling that I was better off compiling stuff manually and cultivating a tidy /usr/local/bin like I did in early 2000s. But it’s probably rose-colored glasses and I’d definitively never take the time these days.
A lot of Homebrew packages are pre-compiled binaries ("bottles" produced by Homebrew) that are pretty much just downloaded and untarred onto your system.
The Homebrew formulas' are what's used to produce those binaries, or - if there's no pre-compiled one available - that's when it's compiled live on your system.
Seems like a decent tradeoff, as otherwise installing (say) Qt could take an hour+, since that's a bit of a monster to compile. :)
The problem is that every time you install something it will update a gazillion dependencies because there is a minor version difference in a library I don't even use. And of course I did not ask for those binaries to be upgraded.
When I do a yum install, 99% of the cases it just add the package I need. And won't mess with my dev tools.
Don’t do this! not only is it wrong for Apple Silicon Macs, you may be screwing up software not installed by Homebrew. Several GUI apps with CLI components install to /usr/local/bin.
But other users might not, and you wrote that as a reply to someone asking how to do it. My comment was meant to warn others they should not go that route, especially when there’s an official uninstallation script (linked in my other comment).
> Why do you say it’s wrong on Apple Silicon?
Because in that case Homebrew installs to /opt/homebrew, not /usr/local.
While I use Homebrew on my Mac — it is a major risk. I am afraid that the project is not following strict security policies and will be compromised at some point. Evil package or evil version of brew can steal or modify my data.
I think this will also hurt the Mac platform. I wish Apple would invest some effort into building an App Store for cli programs. While it may be complicated for the community(require signing, sandboxing, developer effort), it is likely to have better security guarantees than Homebrew.
> While I use Homebrew on my Mac — it is a major risk. I am afraid that the project is not following strict security policies and will be compromised at some point. Evil package or evil version of brew can steal or modify my data.
This is a genuine question: what is your threat model, and what security policies would you like to see? It's hard to imagine a threat model that assumes an "evil version of brew" that doesn't also prevent you from installing any third-party software on your machine. Malicious packages are always possible, but Homebrew's integrity checks do an adequate (but improvable!) job of making such attacks auditable; I'm not aware of any successful supply-chain attacks mounted against Homebrew users.
I have a lot of experience with various package managers, and Homebrew is not particularly out of band with its peers (which should be read, partially, as a general indictment of packaging practices).
It is not feasible to avoid installing any third-party software. Moreover I admitted that I use Homebrew.
However I don’t understand what makes Homebrew reliable. Homebrew is a non-profit project run entirely by unpaid volunteers.
I trust repositories controlled by corporations (RHEL, Ubuntu) or properly(?) governed non-profit organizations (Debian) more. Also I trust App Store more, because of sandboxing, static and dynamic analysis.
I don't understand the distinction you're making between Homebrew and other non-profit OSS orgs: Homebrew has a documented governance structure[1] and is hosted under a non-profit foundation.
The App Store exists primarily to distribute proprietary applications. Homebrew exists primarily to distribute open source packages. Comparing the two (and Apple's justified focus on sandboxing, etc.) feels like a mismatch.
I'm not a very technical person, but I think MacPorts adheres better to macOS standards and security practices. (At least it asks my admin password before any major interaction.) Been using for a few years, coming from Homebrew, and it's running smoothly.
By default, Homebrew requires world-writeable location in the PATH (doesn't matter if it's /opt/homebrew or /usr/local). In my opinion, this isn't a sensible choice.
Homebrew, as the name implies, was built for hobbists with little regards for security. Not a problem for many, but it does make it incompatible with a corporate environment.
It does matter though? /opt/homebrew is specifically used by Homebrew, whereas /usr/local is kind of a more shared location that old Homebrew hijacked.
Either way, I think bad default folders is better than bad default security practices. Most package manager forces you to use sudo to run them as root, and I think that's much much worse in terms of security practices and encourages the wrong behaviors and potentially allowing build/install scripts to wreck havoc on your system. (Note that /opt/homebrew is not world writable as you suggested. It's only writable by the user)
Instead, Homebrew prefers to run in a user account, default to yours. If you have a multi-user computer, you can install Homebrew to a local user folder to have isolation and not using a global location like /opt/homebrew, or you can make a special Homebrew user account (rather than using root), which is what the docs suggest (https://docs.brew.sh/FAQ#why-does-homebrew-say-sudo-is-bad).
I don't think the current solution is perfect, but I do think the motivation to prevent usage of sudo is correct in terms of security. In fact, I would argue that the other package managers that use sudo are taking the easy way out (albeit insecure).
MacPorts drops elevation on package installation, so no, sudo is a better way than globally messing with the permissions and writing a confused FAQ about it
I just whish macports had buikdslaves for m1/m2 architecture. It's just too much to have to rebuild clang and gcc (as build deps) on every other upgrade...
I've even considered donating a Mac mini or something - but not to the point that I've yet emailed the macports list to ask how I could do that...
Or just hire a few FTEs to properly support and maintain the CLI. Pull in new FreeBSD base at releases if they really want to axe their overhead. It's always just read as absolutely lazy how slowly the CLI progresses. Hire a few more and they could absolutely offer a package repo. This company dwarfs Red Hat prior acquisition.
A company like Apple could kick whatever money to an OSS project and write it off as charity for taxes, I think for that matter.
Same here.. I just want a default sandbox (no network + only file access to it's subdirectories)
The alternative is to run VMs/containers/etc for each project. But at that point, it will the beginning of the end for Apple's software, cause you're esentially running linux and using macOS as a terminal/thin client.. (Unit) devtools is how they got big with OS/X, and I think they sometimes forget this.
They have, it's called Macports. It started as a port to Darwin of the existing *BSD ports and (I believe) is subsidized by Apple.
It installs itself in /opt/local, it respects the Apple SIP.
Homebrew basically sucks, because it doesn't follow Unix conventions and it doesn't follow MacOS conventions.
Macports has both pre-compiled and build-from-source, supports current and past MacOS releases, uses xcode/clang for compilation, supports port variants and multiple versions installed in parallel (with "port select" to choose a default) etc etc.
No thank you. I don't want a walled garden where Apple has absolute authority to enforce anti-competitive policies like they do with the existing App Store (especially on iOS), and where they require you to pay an annual fee to publish anything, and where they tie everything to an AppleID.
I use a Macbook for work, and every day I go without having to create an AppleID is a good day.
What I want is a dang CLI package manager, and Homebrew is pretty alright at that. No, it's not perfect, but it's far better than anything Apple has done related to app stores.
By this logic, the ability to install arbitrary software is a major risk.
I strongly disagree with this stance, and it's a very authoritarian one at that. The common people can't be trusted with freedom so we should give up that freedom and trust in <insert authority here>. I find this stance morally repugnant and entirely at odds with where the future of technology should be going.
Tl;Dr: Hard disagree. Some of us prefer to use our computers as general computing devices, as opposed to just being desk-driven iPhones.
Apple should close this gap and I am puzzled why they have not done that yet. Because even for iOS development people need things beyond the xcode toolcahin and they resort to either of the available options. Homebrew is just one of them.
Not only cli app store but the mac app store also badly needs some love.
In 2023 I want Apple to stay as far the hell away as possible from App and package management as possible on my Macintosh. I don’t have the same hang ups on my iPhone, but I don’t want a merger of policies and incentives between the two platforms either.
I think the idea is that there should be an option to have Apple-App-Store-audited capability-manifest-sandboxed CLI apps with auto-updates, as well as untrusted, non-sandboxed CLI apps.
Specifically, I would expect two things:
1. In general, macOS .app bundles should be able to “provide” a bin/ directory that gets added (appended, not prepended!) to your PATH while you have the package available on disk + non-quarantined. Just like .app bundles can “provide” file extension bindings that cause things to open in them. (And probably, just like with file extension bindings, you’d want to give the user the same “this is a program downloaded from the Internet…” Gatekeeper warning the first time a command-line invocation causes the user to end up running an executable from a given .app package, if the .app hasn’t yet been Gatekeeper-blessed.) Kind of surprised this hasn’t always been a thing since NeXTSTEP, actually.
(And, ideally, rather than needing to care about PATH ordering re: these applications’ exposed bin/ executables, instead make a generic system for interactively choosing preference-ordering, like Debian’s update-alternatives(1) but as a per-user runtime-bound preference.)
2. The App Store should allow at least apps that contain both GUI and CLI versions (e.g. gVim packaging its own command-line `vi`; or Sublime Text with its `subl` command) — where all the same capability-manifest sandboxing applies to the command-line executables that applies to the GUI app; and where the CLI version of the executable linker-loader is made slightly smarter re: auto-granting capabilities, such that if you e.g. pass a file or directory path as a command-line argument to a CLI program, then that CLI program should be auto-granted a capability on that file/directory during that run.
Most packages that Homebrew install just wouldn't fit nearly into the kind of sandbox model that you suggested. Just think about command-line applications that can take in any file as input and process them, e.g. Git, ffmpeg, etc. Or like a compiler like clang. Even for GUI apps most Mac applications do not use sandbox / App Store because it's actually quite difficult to work within that framework for a lot of types of applications.
macOS also doesn't have a way to tag a command line application to have sandboxing, as sandboxing only works for an "app" bundle, not random executables.
This isn't really a Homebrew problem anyway. If there is this magical sandboxing capability that works, nothing stops a Homebrew package from using it.
I have a lot of love for Homebrew from many years of using it, but I don't think you'll ever find me using it in macOS or Linux again. I use Nix on both now and I'm convinced that it is the saner way of doing things.
Does Nix still have that extremely steep and high learning curve and huge time invest before you actually grok it? I'd love a Nix that is about as hard to use as homebrew, but no luck so far.
To be fair you only really need to know the commands to invoke which are similar to brew as far as what they do, and not the underlying nix expression language which is where things get difficult. Unless you intend to go full power user and add your own overrides, in which case might not be worth it.
Almost certainly; I still have CC on a ticket from a few years ago that one of the literal steps in the reference manual (teaching --switch-profile) was extremely misleading, didn't work, and actively broke your ability to continue working on nix.
The ticket was closed because the man-page has a technical description of what the command actually does, and people should just know how the command is supposed to work, even if the reference manual tells them otherwise.
Beginner-hostility appears to be baked into the ecosystem.
Imo, no. I think the documentation is very good, unlike what I had heard. The thing that I really had to invest in was my ability to patiently read documentation closely instead of aimlessly trying to find a shortcut by jumping around the doc pages.
Haha - I really began to learn nix when I started to read the docs before bed like a novel. I’ve started to do this for more libs / frameworks and it’s going quite well - It seems to be the fast track to power user status!
If you really want to keep the old behaviour add `export HOMEBREW_NO_INSTALL_FROM_API=1` to your shell’s startup files. You won’t get the speed boost from the new behaviour but it seems like a tradeoff you’d be willing to make.
I can't swear to it, because I have a locally patched copy that reports the way I want, but I believe `export HOMEBREW_UPDATE_REPORT_ALL_FORMULAE=1` may put that behavior back: https://docs.brew.sh/Manpage#environment
I moved from Homebrew to MacPorts on a local, personal-use Intel Mac.
I chose Homebrew some years ago when I was not as experienced a programmer, because it was more popular than MacPorts. When I revisited the choice last year, I chose MacPorts.
On Homebrew:
* It has a Frankenstein permissions model; for example "brew install" writes files into /usr/local/* with your regular user account as the owner of the files.
* The permissions model means that Homebrew-managed parts of the system become a single user system, as far as I know. Multiple user accounts on the same Mac can't easily use Homebrew, as far as I know, and I dislike software with such design choices.
* Noisy messages and undesired colors in the command line output.
These issues are absent in MacPorts. Overall, MacPorts appears more mature and more Unix-like than Homebrew.
The cons of the move are that not all the packages I want are available in MacPorts. However I have sufficient experience now to package crucial missing ones into MacPorts.
> permissions model means that Homebrew-managed parts of the system become a single user system, as far as I know. Multiple user accounts on the same Mac can't easily use Homebrew
The single user thing is problematic in a corp env where you may have a mgmt admin user and a non-admin dev user.
Long story short, temporarily have the dev user be an admin, install brew, drop the admin priv. That works for cli tools (which is actually great), but may not work for /Applications unless you change its ownership. Used to be OSX effectively (not literally) merged ~/Applications and /Applications, but now in MacOS the security sandboxing and entitlements don't like that.
I switched from MacPorts to Homebrew very early on because it was better contained. I had to support developers who would build with MacPorts and end up with something which would crash with a linker error when they shared it with someone off the team because it was using a MacPorts binary in a non-standard location, or whose shell scripts broke expecting a GNU CLI utility version which was in their path but not their users. MacPorts also took forever to build since it dragged in so many replacements for system libraries.
Homebrew has done a much better job of staying out of the way.
Moved from Homebrew and Nix to MacPorts for a while around the M1 transition. Came away quite liking MacPorts; it was simple, and it worked.
Moved back to Nix when it was ready for Apple Silicon, mainly for better isolation / reproducability, ephemeral environments, and parity with my other systems. But if I need something that's not in nixpkgs, I'll look at MacPorts first.
1. I have used Fink. And I have used MacPorts. Now I use Homebrew.
2. When I get super-annoyed with a system, perhaps because it has entered a state where it won't update, I do a web search to see if folks have jumped from that ship to another. If there is a consensus, I jump with crowd. If not, I try to clear the decks, e.g. reinstalling from scratch, to see if I can improve my situation.
3. I don't have a list for you on the pros and cons. There may be some merit in choosing a system that is widely used, because then others might be able to help you if you encounter problems. By that measure, I think Homebrew is the best choice at the moment. But I've no reason to think there won't be something else just around the corner. I wish Apple would get in this game, but after so many decades of Apple standing by without acting, I am not especially sanguine.
Macports had broken multiple times in a year, under totally ordinary operation (nothing weird, just installing and uninstalling packages without doing anything that ought to be risky). A couple times I'd decided it was easier to just delete its entire directory and start over. Big rpm-hell vibes from the bad old days.
Switched to Homebrew. It's broken a couple times in a decade or so, only on OS upgrades. Easy fixes. I like the sudo-free package management. I like that it lets me manage a lot more of my software than Macports did (bigger selection, and "casks" for commercial software, which are now integrated into the main UI so that's not even some extra thing to learn anymore—it feels like being back on Gentoo with a good binary package cache)
I have zero experience with pkgsrc. From a first glance (from pkgsrc.org), I can say that nix claims to have 80k packages whereas pkgsrc 26k. Also, nix'es main selling point is reproducible builds, which is pretty cool. Other than that, nix is satisfactorily fast (much faster than homebrew, not sure in comparison to pkgsrc).
Switched to MacPorts on an older machine which Homebrew dropped support for (and deleted all prebuilt binaries)
While neither is great as a package manager, especiall with their dependency resolution, here are a few pros&cons:
Apps: ++ Brew has a much more up-to-date collection, also more non-source binaries.
App customization:
+ Port, Brew dropped support for custom install flags a while ago
+ Brew you can setup autoupdate for your personal customizations with github actions since the main repo is using them, Ports does it manually, so you can't copy&paste their action for your use
(+ Brew allows you to install Mac .app bundles in a custom folder)
Security: + Ports has a slightly better security model (folder persmissions are better though I think it only matters for multi-user machines, so mostly no relevant; it also does some sandboxing on install via a specially created user), the downside is annoying sudo, but you can remove the need for sudo for some less sensitive ops like update/uninstall to cut down on the annoyance
Space: + Brew. Both waste it with their poor package repository architecture, but with Brew you can at least delete the repo after the first install and use their API to download updates. With Ports you can't do that, moreover it duplicates its registry (one for sync, uncompressed, another for local use)
Docs: + Brew, also random Google/SO answers are less likely to be outdated
UI: + Brew. Has more info (like size), better formatted
> Space: + Brew. Both waste it with their poor package repository architecture, but with Brew you can at least delete the repo after the first install and use their API to download updates.
BTW, with this release the Homebrew uses the API by default and doesn't clone the repos unless necessary.
I did recently, when I switched from an old computer to a new Apple Silicon Mac. It was pretty easy! I was already familiar with it because I used homebrew-cask for application installs, and used to help maintain that project. The most difficult part was really just redoing location of library paths for Perl and the such.
I like brew because it’s easier to audit, faster to fix issues, and tends to update first. Years ago, macports just had so much more, but that’s become less of an issue over the years, for me anyway. Getting away from sudo is always a plus. MacPorts probably(?) is still better for very old machines, though, both in terms of speed and availability.
I switched from Homebrew to Macports, although I may switch back.
Basically, Homebrew will completely crap out on you if you are ever more than two macOS versions out of date. The brew people simply don't keep the older bottles around for those versions of macOS. I think there's a grace period of like one extra version, but eventually they will delete it. Once that happens, your system is hosed if you ever need to install any new packages from Homebrew ever again. You will only be able to install from source, and it's a tossup if that will work.
This is a fairly controversial Homebrew policy and lots of people have complained about it, and there's lots of rhetoric on both sides. My stance: while it's a good idea in general to stay up to date, if you are the kind that likes to keep old legacy machines around to do a few things here and there, you should know your Homebrew installation will eventually break if you don't keep updating the OS. You will eventually not want to do when the system gets old enough, and at that point Homebrew will be a mess.
I'm not interested in the debate as I get both sides, particularly involving security concerns. My experience: simply put, Homebrew's policy will cause you to have to sideline your system, or at least work around this really tedious roadblock, while it still has a good amount of life in it. I wish they kept old bottles around a few years longer, because 2-3 years is not long enough.
So I said that on my current system, which I do keep current, I'm going to not use Homebrew at all. I figured I'd use Macports, since one of the big selling points is that it works on older versions, so I won't eventually run into this problem five years from now. So I did.
The main problem is that there just aren't anywhere near as many packages as there are on Homebrew. Homebrew has become a standard for all kinds of stuff, and if there's ever a package you want on Homebrew, and it's not on Macports, what do you do? You will run into this when using Macports all the time. I've also had issues using Macports with M1; I don't remember the details but there was some weirdness I ran into one point having to configure it to set the architecture correctly on some package I was trying to install. I'm not quite sure how that compares to brew, though, since I haven't tried brew on M1.
So the net result of all of this is, I have a bunch of installation setup scripts that I moved from Homebrew to Macports, and then half of the packages just weren't on Macports, so I had to write scripts to just download those things from elsewhere - GitHub, website installers, whatever.
So now I'm debating moving back to brew. On the one hand, it just has everything, and Macports doesn't. On the other hand, it will eventually break, and Macports isn't that bad - has basic terminal commands I need and so on. I'm on the fence. If there were some user-maintained repository of legacy bottles for brew I'd move back to that.
Portfiles are definitely a little more outré than brew's Ruby, but I've found it's not too hard to make one for a tool by starting from another similar existing port. You get the contents of the example one with `port cat $existing_tool`, tweak it, and then add it to a local port repo: https://guide.macports.org/chunked/development.local-reposit... And then you can just `sudo port install $new_tool`
One thing about Homebrew that's really great and somewhat obscure is that you can use Ruby code inside brewfiles. I use that to have just one brewfile in yadm and make it do slightly different things on work vs personal laptop.
I despise, despise, Homebrew's "we know better than you" approach to always installing the latest versions of every dependency. How tf is this fine? They don't even offer alternative. They just say "we can't test all of the packages installed and all version combinations, so this is the best we can do". Well, no one asked you to test my exact setup. All I'm asking for is: if I have perfectly working Node 18 and I need to install fcking Shopify CLI, don't fcking install Node 19 and break my whole setup! Tf?!
> I despise, despise, Homebrew's "we know better than you" approach to always installing the latest versions of every dependency
I love it. I'm using a desktop, not a server. I specifically want my dependencies to be always up to date
> All I'm asking for is: if I have perfectly working Node 18 and I need to install fcking Shopify CLI, don't fcking install Node 19 and break my whole setup! Tf?!
Then install nvm. Nvm will always be latest, the Homebrew packages will always depend on the latest node version, and locally you'll depend on your own node version that you installed with nvm.
If you have project dependencies, Homebrew's not a good fit for that—but, crucially, that's far from just being a problem with Homebrew, it's true of most (not all, but most) system package managers.
Using a tool-specific system (e.g. nvm), or Docker, or a VM, is far better for reproducibility. Homebrew's better for installing things you're going to use directly, yourself. Like, don't use it to install Postgres system-wide if you need it for some rails "app"—you'll just run into pain the first time you have to git-bisect or go back and fix some older deployed branch and that version was running on some earlier version of Postgres, or the first time you want to work on a second project that relies on a different version.
That's not just a Homebrew thing, the same problem applies, to, say, Debian and dpkg—don't use those to install project deps, unless you're very sure those deps will always be "whatever's available from Debian today".
Homebrew's great for installing your tools. It's not good for project deps. Again, that's far from being just a Homebrew problem.
I've been using Nix (and, occasionally, MacPorts) instead of Homebrew for a few years now. Works great, and doesn't have that particular issue. If Nix seems like a bridge too far, the Devenv project offers some nice, lightweight porcelain atop the raw Nix experience.
That's a Node problem. They make semver-major breaks on a regular schedule, which is a flabbergasting policy. Besides, you're still in trouble even if you don't upgrade, because then your JS deps will eventually start breaking from a too-old Node. And if you don't upgrade deps, you risk vulnerabilities, and you won't know which one of the 20 "critical" npm reports are ReDoS spam, and which one are actually important.
Homebrew’s dependency resolution was a complete joke the last time I checked it out, but they were completely happy with the laughable results so I won’t be surprised if it hasn’t changed.
Essentially it did all dependency resolution at build time (because brew used to have no prebuilt binaries and everyone’s workstation would spend 90 minutes compiling the same things over and over again). But when they added binary distribution they didn’t change the dependency resolution to work like eg deb’s or rpm’s do.
It used to be that if you told it to install from source it would thus be able to “detect” your local existing package that provides the dependency. It may just take a while depending on what you’re installing.
You can specify specific package versions in brew if you want specific versions pinned.
The default behaviour of using the latest in repo is correct and is how every package manager works. It’s the behaviour people expect and it’s the behaviour that is arguably more secure (the fact that homebrew is bleeding edge does create issues here but generally speaking my point stands).
I believe you pin using the @ symbol followed by a version number. Eg brew install awesome-project@13.0
This is not at all how every package manager works. On debian if you install a package it will just install that package and it will warn you if dependencies will require upgrading first. It won't upgrade totally unrelated packages and break everything for you. In addition, you have different branches where the debian stable branch is extremely well tested.
Homebrew is way, way worse than any other package manager because it just breaks stuff all the time.
Brew follows a rolling-release style, it behaves the same way the Arch Linux package manager does. In Arch users are discouraged from upgrading specific packages instead of everything as a whole.
That said, the docs say that running "brew upgrade <package>" will only upgrade that specific package. Of course, if that package relies on new versions of dependencies, they will get updated too, and in turn this will update any packages that also depend on it, creating a chain of updates. But there's an option to see what will happen: "brew upgrade --dry-run <package>".
> This is not at all how every package manager works. On debian if you install a package it will just install that package and it will warn you if dependencies will require upgrading first. It won't upgrade totally unrelated packages and break everything for you.
What you're arguing about now is just that Homebrew doesn't give you a prompt. Which is a fair criticism but it doesn't really change much because if you need that other package that depends on the latest version of node, then all the prompting in the world isn't going to change the fact that you are still going to need to install that updated version.
But, as I'd mentioned before, `brew` does let you install specific versions of software too. So it's a bit of a moot point about whether you get upgraded or not since, in this circumstance, you can have your proverbial cake and eat it (ie both versions installed).
> In addition, you have different branches where the debian stable branch is extremely well tested.
Debian isn't bleeding edge and nor is it rolling release. Arch is a better comparison and people frequently complain about updates breaking things on that too (even though that does have a testing branch). To be clear, I love Arch. It is by far and away my favourite Linux and a large part of that is because of it's package management. But no system is perfect.
> Homebrew is way, way worse than any other package manager because it just breaks stuff all the time.
I agree Homebrew has it's warts. I just disagree with this particular complaint. I think Homebrew is doing the right thing here.
If you want to complain about how it has to operate outside of the core OS because it's a 3rd party tool, or its overuse of beer jargon as cute substitutes for the more recognisable package management terminologies then I'm 100% behind you. But to complain that it's keeping your system up-to-date when you haven't asked for a package to be version pinned feels more than a little unfair.
But the shopify cli, the given example, does depend on nodejs. If you tell Debian's apt to install a package that depends on a newer version of an installed package, it'll update that package for you as well.
If Debian stable updates their nodejs package to 19 and you install any package that depends on nodejs, expect to get the nodejs 19 update.
The difference here is that Debian pins versions of software per release so you won't run into these situations. That's fine if you're willing to deal with running older software maintained by Debian rather than the original developers, but that's just a choice.
From what I can tell, Homebrew acts more like pacman and friends; it expects you to stay up to date, and if you don't it's up to you to fix the problem.
> On debian if you install a package it will just install that package and it will warn you if dependencies will require upgrading first. It won't upgrade totally unrelated packages and break everything for you
This is exactly how Homebrew works, too. The difference is that Debian won’t ship a new version outside of a major release so if you started with Node 18 you won’t get 19 until next year.
Now, of course, developers usually don’t like that so there are non-core repositories which update more frequently … and you get exactly the behavior which Homebrew has. Being oriented at developers just means Homebrew does that by default.
> Homebrew is way, way worse than any other package manager because it just breaks stuff all the time.
I hear this a lot, but as a 12+ year user of Homebrew it simply does not match my experience, other than for Python which is in and of itself fundamentally broken regardless of where you get it from. Indeed, I’ve spent far longer screwing about trying to get up-to-date dependencies on recent Ubuntu releases than thinking about it.
I wanted to love Nix, and still use it sometimes, but it is unfortunately not very friendly to arbitrary package version selection. Asdf for the worse-is-better win!
You could always manage your packages and their dependencies yourself.
Or you could release a new package manager which does provide support for that functionality, build out a team of maintainers, and stay on top of all the releases
It's really not. It's borderline impossible (pun intended) to start your own country, mainly because you need to find a piece of land first. Starting a package manager is, like, a few months of work?
I wish we could get the industry to understand that when you build-in analytics and make it hard for people to opt out, you're using their resources.
Sure, people don't pay for things like Homebrew (can they even?) - but this is no reason to extract resources from end users without their consent or agency.
I have to work very hard to get something like Homebrew installed and in-use in my organization - the IT guys detest it, since they don't have the wherewithal to understand it (or else they'd be higher-paid developers) so when I finally do get it installed and in use (because it is a very productive tool), having it phone home just makes the situation worse. IT guys have asked for my blood on more than one occasion, just because they see something 'uncontrollable' phoning home .. this does a lot more harm than good.
That said, homebrew has been absolutely responsible for making MacOS usable for development purposes in this ol' hackers lab, so I'll keep using it - but I sure do wish they'd care a little more about the position they put us in, having to justify the use of it to IT guys ..
EDIT: Thinking about it, I'm actually not averse to phoning-home, just the automatic/hidden nature of it. If brew asked me, every 5 or so times that I ran it, if I would like to participate in sending anonymized analytics to the brew devs, and showed me what was being sent (instead of obfuscating it), I'd be a lot more inclined to click "[Yes] - this one time" almost every single time ..
The installer gives a warning message before sending the first analytics, with a pointer to the above URL for how to disable analytics. From install.sh:
ring_bell
# Use an extra newline and bold to avoid this being missed.
ohai "Homebrew has enabled anonymous aggregate formulae and cask analytics."
echo "$(
cat <<EOS
${tty_bold}Read the analytics documentation (and how to opt-out) here:
${tty_underline}https://docs.brew.sh/Analytics${tty_reset}
No analytics data has been sent yet (nor will any be during this ${tty_bold}install${tty_reset} run).
EOS
)
"
> and showed me what was being sent (instead of obfuscating it),
You can also view all the information that is sent by Homebrew’s
analytics by setting HOMEBREW_ANALYTICS_DEBUG=1 in your environment.
Please note this will also stop any analytics from being sent.
I verified that using it dumps JSON analytics to the terminal, and it didn't seem that obfuscated, given the documentation in that Analytics page.
None of this is user-friendly - all of it is obfuscated. I have to add resources to my system (env vars) in order to opt-out - and that is the issue!
There are no hints/tips about this produced to educate a new user - if it weren't for your help, I wouldn't have known.
The non-resource-using method I propose is that homebrew merely ask the user if its okay, after 5 or 10 runs, to send some info - and then shows me the info. No, I don't want to add resources to my environment to get this info - I want homebrew developers to be honest about it, up front, and overt.
They are being sly, using dark patterns to make it unfeasible to disable analytics, because they know this is an unpopular feature of their software.
> None of this is user-friendly - all of it is obfuscated
This is a message which is printed with very clear instructions the first time you run any command. It’s a simple command and there’s no noticeable difference in resource usage compared to the actual package management system, which also needs environmental variables to be set.
The big thing you’re missing is that the Homebrew maintainers are freely giving you the product of many thousands of hours of skilled work, the kind of service developers used to pay money for. In exchange, they ask for some non-personal statistics to help them avoid mistakes which could cause problems for you. I don’t think it’s fair to act like they’re tricking you when you’re getting so much out of the deal and it’s all disclosed publicly. They could be tracking stats on the package distribution infrastructure without your knowledge at all but instead chose to be fully above board about what the system does and why, and offer a trivial opt-out mechanism.
>the kind of service developers used to pay money for
I've donated to homebrew, I support them that way.
The moral issue is, I'm using their tools and trusting them, and they are violating that trust by phoning home and sending analytic data without my involvement or approval. While it may seem trivial to you to set an environment variable, having to do that with every new homebrew install adds up, and when its forgotten about, results in a negative impact on my networking and IT resources.
> a trivial opt-out mechanism
Its not trivial, and thats the point where we disagree. No, maintaining environment variables to prevent tooling from sending unknown analytical data: this is not trivial. It is an anti-pattern designed to gain the data from un-sophisticated users, rather than treating users with respect and letting them have agency over whether their resources are used by the homebrew developers.
Tracking should always be opt-in and if it isn't, that is simply a dark pattern being used to scam resources from the users.
> No, maintaining environment variables to prevent tooling from sending unknown analytical data: this is not trivial. It is an anti-pattern designed to gain the data from un-sophisticated users, rather than treating users with respect and letting them have agency over whether their resources are used by the homebrew developers
A dark pattern would conceal this, not prominently warn you, or it would be hard to do - not requiring only a skill you already used and giving you precise instructions. I think there’s a legitimate debate for opt in or out but I don’t think “dark” is warranted for something people are talking about because it’s in the open and prominently disclosed.
I updated to 4.0 and didn't get an alert about the new analytics - if I hadn't read about it here, I wouldn't even know there were different environment variables to stop analytics tracking now.
I'd wager there are thousands of users who would prefer this not be tracking them, but yet do not know it is. That's a dark pattern.
> I wouldn't even know there were different environment variables to stop analytics tracking now.
Where does it say there are different environment variables to stop analytics tracking now?
The linked-to post says there's a new setting if you are okay with the new analytics and don't want to use the Google analytics:
> If you had previously set HOMEBREW_NO_ANALYTICS because you didn’t like Google Analytics and/or data being sent to the USA: please consider unsetting this and setting HOMEBREW_NO_GOOGLE_ANALYTICS instead, allowing analytics data to be sent to our new InfluxDB host.
but that reads like the old settings still work with brew 4.0.
> I have to add resources to my system (env vars) in order to opt-out - and that is the issue!
FWIW, I use "brew analytics off", not an environment variable.
You can't install brew without adding resources to your system.
But I think your real complaint is not about resources, but that the default is "opt-in with a warning" rather than "opt-out then ask for permission".
Which I can understand and agree with.
What I disagree with is characterizing the process as "hidden or hard to opt-out."
> There are no hints/tips about this produced to educate a new user - if it weren't for your help, I wouldn't have known.
I don't understand how you can say this.
The hint/tip is produced in the installer output. I quoted it. It even uses a terminal bell and bold text.
I can understand that you might prefer, say entering "yes/no" to a "may we enable analytics?" question (though it's a bit more complicated as someone may be fine with the new EU-based InfluxDB analytics but not Google Analytics).
But that's not the same as saying there are "no hints/tips about this produced to educate a new user", when it's right there in the installation output.
> merely ask the user if its okay, after 5 or 10 runs
Tracking the number of runs since the previous report is also adding resources to your system, which makes it hard for me to understand how adding those resources to your system is fine but adding an on/off resource is not.
I agree it's a dark pattern which makes it less likely that people will opt-out.
But I also think "unfeasible" is not an good characterization of "read the installation output then run 'brew analytics off'".
> EDIT: Thinking about it, I'm actually not averse to phoning-home, just the automatic/hidden nature of it. If brew asked me, every 5 or so times that I ran it, if I would like to participate in sending anonymized analytics to the brew devs, and showed me what was being sent (instead of obfuscating it), I'd be a lot more inclined to click "[Yes] - this one time" almost every single time ..
I'd take a minute to rip the fckng line of code bugging me every 5 installs out of the code. Thing is, if you make telemetry opt-in, it's the same as just not adding it in the first place. Only a handful of users will opt in, rendering the data pretty much useless, especially as it's install and install-fail counts.
If they included any personal data, it'd be opt-in by law (at least here in EU). As they don't, they don't have to, and I'm fine with it.
If glib IT policy gets in the way of development, its a fair characterization imho. Both the Homebrew developers obfuscating their analytics, and IT guys wanting to lock everything down, suffer from the same malady: glibness.
Too many times IT has had authority over development to the detriment of the project. Developers must resist this.
… which is also why the cookie banner scourge shows how nefarious companies are these days. They don't need identifiable information for their analytics, they're just all super greedy.
I used Linuxbrew for a while before switching completely to nix-pkg due to some recurring issues I had downloading some packages. nix-pkg is quite nice and noticeably faster, although rough around the edges, and the commands are hard to remember (so I just set some aliases), but overall I am happy.
makes you wonder how much of the developer's mind is "works for me" instead of following more common sense conventions like at least put them under /usr/local or /opt instead of some special user folder in /home.
Thank you. I tend to rely for even open source GUI apps because the experience of installing/removing packages have been smooth (at least for me) so even on Ubuntu, I find myself using brew instead of apt/snap for personal use.
Side question - what can we do about improving security? Of the possibility that packages are not poisoned? Some sort of central CA/GPG or similar requiring to sign the packages or there would always be some holes and it is a hard problem to solve?
From the brew devs' point of view it doesn't make sense because they are different commands.
From the end user's point of view it makes perfect sense because brew is just a tool to install shit with and they do not wish to expend any brain cycles on it. That includes remembering whether this particular tool calls it update or upgrade.
You know the second point of view is right because if you type one without arguments, it asks you whether you meant to type the other one.
But they're too autistic to realize what that means and actually fix the interface.
Similarly, as someone who only rarely uses it, a huge frustration is that when I do, it invariably wants to update 100+ packages. I never asked for it to do that and I was happy with whatever was installed right before it decided to do that and block my system for 20 minutes.
Again, brew devs do not understand their audience.
Disagree. I've been a user for a lot of years and prefer the apt-like distinction between the commands. They do different things.
> Similarly, as someone who only rarely uses it, a huge frustration is that when I do, it invariably wants to update 100+ packages. I never asked for it to do that and I was happy with whatever was installed right before it decided to do that and block my system for 20 minutes.
There's a setting for that, no need for it to keep being a huge frustration.
Good interfaces have 1 to 1 mappings between the available levers and the user's model of the tasks they wish and need to perform.
Bad interfaces require you to care about implementation details that are arbitrary. Like whether the index is local or remote, whether it is stale or fresh, etc.
This is not an opinion, this is what decades of product design lessons tell us.
I don't use brew enough to ever go dig into settings, I wouldn't even know where to go find them, nor do I wish to expend cycles to go find them. They shouldn't design their software with such shitty defaults. All they're doing is ensuring my opinion of it gets worse every time I use it.
> Good interfaces have 1 to 1 mappings between the available levers and the user's model of the tasks they wish and need to perform.
It already does. Sometimes I want to update. Sometimes I want to upgrade. This model matches the models of the other package managers I've used, for the most part.
> I don't use brew enough to ever go dig into settings, I wouldn't even know where to go find them, nor do I wish to expend cycles to go find them. They shouldn't design their software with such shitty defaults.
That's fine—but I guess it's not a huge frustration, then.
It's not, it's you that persists in wanting update and upgrade to do the same.
They don't do the same, but if you want, you can completely forget about update and just use upgrade, it does an update behind the scenes. But don't try to force kill update just because YOU have no use for it.
Aside from what the sibling comment said about the "local" install, if you don't want them to run sudo for you, I'd bet $5 that so long as the directories end up with the permissions necessary, you could (effectively) `alias sudo='env'; brew install` and it'd be fine. I do not have `NOPASSWD` on sudo and I can attest that with all my years of Homebrew and Linuxbrew, it has never once tried to run sudo "under the covers"
It's not supported officially (ie. if you have issues with a package you're on your own), but it works great. I've used Homebrew like this for a few years.
I've heavily used probably four Linux package managers, plus both major ones on Mac, and have used two or three less-common Linux package managers at least a little bit, plus a couple of the BSDs. Oh, and Chocolatey on Windows.
I've never gotten this complaint about Brew at all. It has never seemed notably slow to me.
Nothing can feel slow to me anymore after dealing with emerge on gentoo, spending close to 20min solving dependency requirements. Surprisingly, compiling everything was the quick part, the frustration came from dependency resolution
> Homebrew’s analytics are now sent both to Google Analytics and our new, self-hosted InfluxDB instance hosted in the EU.
> If you had previously set HOMEBREW_NO_ANALYTICS because you didn’t like Google Analytics and/or data being sent to the USA: please consider unsetting this and setting HOMEBREW_NO_GOOGLE_ANALYTICS instead, allowing analytics data to be sent to our new InfluxDB host.
My package manager was reporting my actions to Google and I didn’t even know about that. Great…
Homebrew doesn't "report" anything to Google, much less anything of yours (implying unique identification). This is an unnecessary editorialization.
You can see exactly how Homebrew does analytics here[1], and you can see the sum total of the information collected here[2]. No identifiable information is collected or retained.
>You can see exactly how Homebrew does analytics here[1], and you can see the sum total of the information collected here[2]. No identifiable information is collected or retained.
If my computer is sending data to Google then Google has my IP address and can correlate that data however they want. You simply can not claim that identifiable information isn't collected or retained unless you work for Google.
I don't know if you work with Homebrew or not, but I would be much more comfortable if they used something like Plausible for analytics.
They're planning to completely drop and delete all Google data in 90 days, once they are sure the new plan works out.
FTA:
> We expect to migrate entirely from Google Analytics to our self-hosted InfluxDB instance in ~90 days at which point we will remove all Google Analytics and destroy all existing data.
> I would be much more comfortable if they used something like Plausible for analytics
Their intent is to migrate entirely away from Google to the San Francisco based InfluxData, which I guess is marginally better in some ways but seems to entirely miss the point of people's objections in spirit. Other than people's personal subjective trust of one US company over another, there's no inherent difference between Alphabet & InfluxData from a technical standpoint.
A Homebrew analytics user ID, e.g. 1BAB65CC-FE7F-4D8C-AB45-B7DB5A6BA9CB. This is generated by uuidgen and stored in the repository-specific Git configuration variable homebrew.analyticsuuid within $(brew --repository)/.git/config. This does not allow us to track individual users, but does enable us to accurately measure user counts versus event counts. The ID is specific to the Homebrew package manager, and does not permit Homebrew maintainers to e.g. track you across websites you visit.
IANAL, but an UUID is definitely PID under the GDPR:
‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;
Also see recital 30:
Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. This may leave traces which, in particular when combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them.
The GDPR doesn't only take into account whether an identifier can currently be used to identify a person, but also whether the data can be correlated in the future to do so (e.g. by correlating package installs with visiting project websites, thus deanonymizing the UUID).
The only safe way to abide by the GDPR is to avoid storing any non-essential data without consent.
I am pretty sure that Homebrew have been violating the GDPR for years by making analytics opt-out. Sadly, anyone who tries to warn them gets banned from their issue tracker.
Read your own cite: nothing about the UUID in question is associable with an identified or identifiable natural person, which is what the GDPR concerns.
We do not have the ability to correlate your package installs (again, we do not know what you install) with your browsing history, and we do not store any information that would allow us (or an adversary) to do so.
Read your own cite: nothing about the UUID in question is associable with an identified or identifiable natural person, which is what the GDPR concerns.
This is false and a misunderstanding of the GDPR. It is not about whether it is currently possible. But whether it would be possible if it was correlated with other data.
What differs pseudonymisation from anonymisation is that the latter consists of removing personal identifiers, aggregating data, or processing this data in a way that it can no longer be related to an identified or identifiable individual. Unlike anonymised data, pseudonymised data qualifies as personal data under the General Data Protection Regulation (GDPR). Therefore, the distinction between these two concepts should be preserved.
‘pseudonymisation’ means the processing of personal data in such a manner that the personal data can no longer be attributed to a specific data subject without the use of additional information, provided that such additional information is kept separately and is subject to technical and organisational measures to ensure that the personal data are not attributed to an identified or identifiable natural person;
So, basically if we have a data set with three columns:
Personal name, UUID, Action (e.g. brew install fzf)
Removing the first column is pseudonymization, and thus qualifies as personal data under the GDPR. Removing the first and the second column is anonymisation and is not personal data.
Again IANAL, but it is clear from the GDPR that the only thing you could do without consent is e.g. recording what packages get installed/uninstalled, but without a UUID.
Apply the counterfactual: what would have to be the case in order to correlate the UUID in question with user data?
We do not store anything else that could correlate with that UUID. We don't expose it to anybody else and it's unclear how, even if we did, it would result in personal correlation.
Apply the counterfactual: what would have to be the case in order to correlate the UUID in question with user data?
We do not store anything else that could correlate with that UUID. We don't expose it to anybody else and it's unclear how, even if we did, it would result in personal correlation.
You can argue against this, but it's simply how the GDPR defines personal data, and if you violate it, someone could report you to their data protection authority.
Secondly, the GDPR does not just do this to protect citizens against direct use of their personal data (I think most Homebrew users would be immediately convinced that you wouldn't misuse this data, including me), but also scenarios that are outside of your control. Such as: Google decides to violate the GDPR against your will and correlates the data. Or: Google Analytics gets hacked, the data set becomes available on the black market or wherever and people correlate the data with other leaked data.
They'd be well advised to make this opt-in only for legal reasons. This is not going to go down well in a lot of places and they might get exposed to law suits.
Opt-in analytics are useless, unless a large part of your userbase just clicks through the entire wizard without thinking; there’s little overlap with Homebrew’s userbase
Homebrew was fine before they started collecting analytics. There are plenty of great package managers outside the macOS ecosystem that don't use analytics.
Yes. More evidence for an observation I've been having that ultimately the software is always about what the creator of the software wants, and not the user. I'm working on moving to running only my own software for that reason.
As in the post: they're intending to drop the GA part entirely within 90 days, and it sounds like the new metrics are entirely anonymous, and so not covered by GDPR etc. IANAL but as far as I can tell that should avoid all legal concerns once GA is gone.
Why does a package manager need to track their users at all?
If you want usage statistics for packages just track how often individual packages are downloaded on the server side. A maintainer has no need to know who's installing what.
> If you want usage statistics for packages just track how often individual packages are downloaded on the server side. A maintainer has no need to know who's installing what.
To be clear: Homebrew has no idea which users are installing what. We only store counters for package install, failure, etc. events, and everything that's stored is visible on the Homebrew website[1].
Homebrew's architecture doesn't really have a "server side" in the way your suggestion requires: the formulae and bottle components rely heavily on public services like GitHub Packages and GitHub Pages, which don't offer those kinds of analytics.
What kind of decisions do you make based on the analytics? Do you drop unpopular packages? To me, one of the advantages of a package manager is having a huge database of long-tail packages that are just one command away. If you kept only the popular packages you might as well just have an installer that installs them all together in bulk.
Yes: Homebrew deprecates and/or disables packages if we see evidence that they're unmaintained and not actually supported on the platforms we support, or only used by a tiny fraction of users while also requiring disproportionate maintainer time (e.g. due to complex or flaky builds).
The goal is to balance conflicting user interests: 99% of users want maintainer effort focused on the top 100 (or 500, or 1000) packages, and many of those packages also require significant maintainer effort (e.g. making sure that they don't cause transitive breakages).
Generally speaking, yes: we don't remove things that aren't disproportionately burdensome. You can see that in the yearly analytics[1]: there are packages that only get a few dozen downloads a year, and we don't remove them because they don't cause any issues.
That's good to hear! I think small, well-maintained projects with loyal users can add tremendous value to a package ecosystem due to the long tail effect. I believe these long tail effects are a big part of the reason behind the staying power of tools like emacs, vim, and LaTeX.
> Why does a package manager need to track their users at all?
According to https://docs.brew.sh/Analytics they use it to measure how often formulas fail to install, to get overall metrics on which OS versions are used, and to correlate those (i.e. to tell on which OS versions specific packages fail to install correctly).
> A maintainer has no need to know who's installing what
Aside from the IP, they don't know who's installing what, and in the new model announced in this post they now don't store IPs or any other user token at all, so it should be purely anonymous aggregate metrics.
I agree that they don't need to know the "who", but it is perfectly understandable that they want to know "what" is being installed. And as part of the "what", they would want to know on which platform, and whether the install succeeded or failed, and probably a few other metrics about the install to ensure that things are working correctly and identify gaps that should be filled.
Based on what I read on the site, that looks like exactly what they are doing, and they are explicitly NOT storing information that would identify "who".
All stuff that should be in a trouble ticket from a whiney user. Which we know this type of user would be.
Edit-Also, this is for Mac OS. Chose a few standard OSes to support and test them. If a system update will fix the issue then it shouldn't be fixed at the package manager level.
Does anybody put a paywall in front of submitting tickets? Seems like homebrew could if anybody. Ie $10 to submit and might need more for a larger problem.
> Why does a package manager need to track their users at all?
Do any of you actually work in this industry shipping software products to end users? Without telemetry the problem there is literally one of trying to read the mind of your end users to figure out what they're doing, hoping that your internal CI manages to reflect the configuration in their environment.
I think HN has a very varied audience - some work in the industry, others want A/B testing to be made illegal on the grounds that it is non-consensual mind-control experimentation :P
I am in both groups. I work in the industry and I am so tired of colleagues wanting to grab or data they can get their grubby hands on and then barely use it at all for anything useful. So many companies collect data just in case.
Users report issues to GitHub? It's not like Brew users aren't sophisticated in that sense.
In addition to being INCREDIABLY slow, now I have to worry about what it might spy on. If I have a problem I'm more than happy to go to GitHub (or which ever site it's hosted on), and report it.
This industry has managed to ship software products without telemetry just fine - mass-collecting usage data from end users is only a relatively recent trend.
Homebrew is a package manager with thousands of packages, not a file server. We maintain those packages, and knowing when they break (or can be deprecated due to lack of use) is critical to the project's sustenance.
Homebrew can detect a lot of things during normal maintenance work: there are extensive tests and checks during compilation and bottle building, for example.
However, we can't catch everything: Homebrew has millions of users, and those users have all kinds of different setups. We can't predict every possible host and software interaction; basic failure analytics help bridge the gap there.
I understand the desire for privacy, and the seriousness that comes with it!
I've linked Homebrew's analytics data and the source code that collects it elsewhere in this thread.
And, to be absolutely clear: it is perfectly fine for you to disable Homebrew's analytics. There are an infinite number of legitimate reasons for doing so, including the most basic one of "I just don't want to." My sole goal is to dispel the small number of inaccurate beliefs about what Homebrew collects, why we collect it, etc.
Which the software that I used to be employed maintaining has actually broken homebrew compiles when they've been installed at the same time (which I think I made better but I never got the PM who actually owned the product to spend the resources to properly fix).
A good example of how the configuration in the end user environment can affect package installation.
How about calling it nothing more than a file server is also just a Reddit style insult.
If you think a package manager is nothing more than a file server, I'd suggest you learn more about what a package manger does or just go live in Reddit land with all of the other willfully ignorant types. It is not for us to convince you otherwise. It's such lame comparison that even searching for "file server vs package manager" doesn't even autofill because nobody searches for it because probably nobody would ever even sanely think to make the comparison.
Maybe they want to include the most common packages in their unit tests, or understand usage patterns so they can prioritize development?
It’s very hard to write and maintain good software without knowing how it’s used. No package manager needs to know how you specifically use it, but aggregate data and the ability to identify scenarios it does not handle well are both very important for SW lifecycle.
It's 2023. Hard drive space shouldn't be an issue. Test installing the full software suite, make it work, and you know the lesser installs will all work.
That US$100k seems to have arrived over the last year, and their overheads - from quickly looking over their OpenCollective expenses tab - seem to be mostly buying hardware for CI things, and a monthly GSuite bill.
It's also pretty likely they get a lot of their resources for free / sponsored too.
Assuming their claims of anonymity are true, they won't be tracking users at all.
I imagine they can get much richer metrics through this as opposed to only tracking downloads on the server side.
I'm not saying I like it. In fact, I plan to keep it disabled. I'm just saying it's a bit naïve to think client-side analytics are the same as server-side download tracking.
Richer how, exactly? I fundamentally don't 'get' what richness they actually need.
If anything they need money, of course, and to know their software works for their users. Prior to release have a test system install the full base, test those packages work, and you know anything less will work too.
I haven't looked at what they're actually collecting, but here's a few things that come to mind:
- Time to install packages
- Versions of things
- Has the compilation (when required) failed? What dependency versions are installed?
- CPU architecture
- OS version
...
There's a lot more that can be sent from the client that's not available on the server side.
> I fundamentally don't 'get' what richness they actually need.
That's fine. Perhaps you could ask them instead of ranting about what you don't know or don't 'get' in a public forum?
> to know their software works for their users
Sounds like you're not very far from understanding why they want better telemetry.
> Prior to release (...) and you know anything less will work too.
Things break in unexpected ways. OSs are complex systems and there's a lot of interactions between components. Homebrew's user base is enormous and very diverse. There's 2 different architectures, many OS versions, lots of environment variables that might be set differently in each user's systems, different versions of libraries, ... I could go on but I think you get the picture.
Im okay with sending data to the server if things break and be asked each time to okay it. Is it safe to assume that if no errors or exceptions are encountered nothing should be sent back home?
They really aren't. If a package is downloaded 100k times and fails 1k times it's a very different issue than if it's downloaded 1k times and fails 1k times.
The Homebrew folks think this is a non-problem. You may agree or disagree, but the pull request is certainly a non-solution to this maybe-problem. Not just because it gained zero traction and did not get merged anywhere, but also because it's just an obscure band-aid. Either opt-out anonymous telemetry is a good idea, or it's a problem. If to you it's a problem, advocate for its removal in its entirety.
So even if I have an issue with telemetry, good on the Homebrew maintainers for ignoring this MR.
The status quo is different variables and even different mechanisms for each program. Many badly documented. Only out of context is DO_NOT_TRACK obscure.
I think you know developers who reject informed consent will never adopt an informed consent model. The proposal was the best users could hope for realistically. Did you never compromise?
"Without telemetry, developers rely on bug reports and surveys to find out when their software isn’t working or how it is being used. Both of these techniques are too limited in their effectiveness."
Isn’t it GDPR compliant if you never store the source ip at all? So from a GDPR perspective there’s no user data to track and remove.
I’m not sure how organizations get audited to prove that they actually do that and that there’s no other way to reidentify users (eg, I download the prepend package every day and that’s unusual enough to link that it’s me, prepend, the author of that package, etc etc).
We only have their pinky promise that the new analytics are anonymous. For all we know this might be a PR operation because people increasingly dislike Google, and they'll sell the "anonymous" analytics to Google under the table.
I'll make it a goal to stop all their tracking on the level of my router.
> We only have their pinky promise that the new analytics are anonymous.
Isn’t Homebrew open source? One could audit the source themselves. If you’re talking about what happens on the server side yes, but I don’t see the difference with any other computer I connect with over the internet.
> Our self-hosted InfluxDB instance does not store either anonymised IP addresses or an anonymised user token so it has additional privacy benefits over Google Analytics.
It doesn't look likely though. I don't think it looks good that comments pointing out that Homebrew's existing behaviour (collecting analytics without obtaining informed consent from users) violates the law have been classified as abuse and hidden!
In retrospect (consoledonottrack operator here) I never should have pushed an opt out standard; it legitimizes opt-out which is indefensible and unethical.
Opt-in by advance consent is the only way. Homebrew devs are unethical jerks.
Errors are the property of the user on the system in which they occurred. Exfiltrating them without consent is unethical and oftentimes illegal, and leaks the user's IP to Google. Homebrew has no claim to them without the consent of the user. It's simple spyware.
Unless you report with Tor, it's not without PII. (Homebrew also includes a unique install UUID supercookie which persists, so every analytics data point includes PII in addition to IP address which allows Google to track that user's physical travel history.)
Consent doesn't work that way. Imagine posting a sign at the entrance to your house saying "all those who enter here consent to being groped".
Notice is not consent. Your statement that users are notified is a red herring. Users must also consent, not just be told of an entity's plan to violate/assume consent.
Surveillance without consent is unethical no matter who the data goes to, or whether or not the data is anonymized or otherwise stripped of PII. It's stealing of information unless the user agrees to it in advance.
Whilst Plausible does look great, one of their main goals, as I understand it, is to focus on simple core analytics for websites, not general-purpose do-it-all analytics for everything.
If you're not a website, and so your metrics are arbitrary events with metadata instead of page views, this tends to quickly run into awkward mess.
Self hosting an InfluxDB to capture core metrics seems like a good solution that avoids privacy concerns without being too complex imo.
It is fine for medium size applications, you do have to register all events (goals) on the website before you can use them [0]. I have written the Qt/QML plugin for plausible [1], I don't think predefining them is too bad.
Ah, neat, ok! I'll take another look. For now I've been testing out Posthog (https://posthog.com). They seem more focused on this use case - they let you do arbitrary queries and build graphs over all event data without having to predefine goals or anything, and they have an open-source & self-hostable version, in addition to a EU-hosted cloud option. Now that GA is so clearly dying (finally) it's an exciting space!
FWIW, it gives you a warning and the chance to disable analytics before sending any analytics. https://docs.brew.sh/Analytics
> Homebrew gathers anonymous aggregate user behaviour analytics using Google Analytics (until our in-progress migration to our own InfluxDB). You will be notified the first time you run brew update or install Homebrew. Analytics are not enabled until after this notice is shown, to ensure that you can opt out without ever sending analytics data.
That is a very simplifying view of the legal situation and that's not helpful at all.
First, it only applies if you collect PII - depending on what they collect, they might not be subject to the GDPR at all.
Second, informed consent is only one of the options that allows collection and storage of PII. There are various other reason that allow collection and storage of PII, among them "Legitimate interest". For example, it is considered legitimate to store webserver logs containing PII (IP Addresses) for purposes of fraud analysis, unauthorized system access etc. Whether a specific collection of data is legitimate under those clauses depends on the specifics of a case (who has access, what's the exact purpose, how long you store, ...) - ask a lawyer if you need an assessment.
Depending on what they log and how they log, they may be either in the clear or in a bad place, but it's definitely not as simple as "the law requires no logging".
The analytics page describes them tracking information across time with a unique user identifier. They claim that identifier doesn't identify you, but it's attached to an exact Brew install so it does track your personal account on your machine at the very least; I'd classify that as PII.
Had they not submitted unique user tokens I think you mag be right. However, that's not how the analytics seem to work.
The law does allow logging for a variety of things but in this case I'd say they're in the wrong. They assume that it's okay because they don't track you across websites and that's good to know, but that's not the point.
I don't think that's true - AFAICT there's no EU law banning analytics. EU law just restricts storing & processing _personal_ data (GDPR) and storing unnecessary data on machines without consent (ePrivacy/'cookie law').
If you want to log fully anonymized data, without persistent tracking ids and without leaking personal data to 3rd parties en route (so no "send it to Google and they promise to anonymize the IP afterwards") then you're all good (but IANAL!).
The only reason you see all those cookie notices and GDPR consent requests is because so few companies are willing to accept even the tiniest tradeoff in their metrics to protect their users' privacy.
> EU law just restricts storing & processing _personal_ data (GDPR)
To be clear to anyone reading: using Google Analytics without a non-Google-hosted anonymisation step breaks GDPR. This _has_ been litigated in court in several countries. There's no "ifs" or "buts" about it.
There's an implicit _for websites_ on there. Has it been litigated for non-website use like this where the program can control which fields are being submitted?
GDPR doesn't distinguish between websites and applications. It's about collection, storage, and transmission of personal information. No matter who, what, or where. One of the core pieces of information cracked down upon by the French DPA CNIL is that the IP of a user is considered protected under GDPR.
The German DPA has ruled that the usage of Google Fonts on a website broke GDPR because it forced the user's browser to reveal their IP to Google.
I think it’s not cool when orgs track telemetry with opt out. But it’s not cool like when you’re at a party and you go off and fart in the corner as no one’s there and then a few seconds later someone walks by and smells it.
Continuing the analogy, telemetry with no opt out is like farting silently amongst a group of people. And tracking identified user requests while selling data is like slapping each person at the party while farting in their face.
And I guess opt in telemetry is like holding in your fart and people notice and might feel some discomfort at your discomfort.
You forgot 3) Complain at different intensities, up to the shaming, about the unethical dark patterns employed by the software, no matter whether it is open source or not, to make authors of the software aware, that what they do is not welcome by their users.
I use Little Snitch to alert on any outbound connections and make a decision. The google stuff immediately got a permanent blackhole for Homebrew. Anything I'm uncertain of I'll give a short-term approval (30mins) to not break anything. After a couple of rounds of execution (and sometimes some trial & error) you can usually work out which requests are essential and which are some notifications/tracking thing.
I can't recall, though this approach isn't specific to homebrew. I block it permanently at a network/process level rather than having to remember to set a ENV var.
Thanks, a quick search on the home brew page brought no guidance on how to configure these settings. I can't remember if this is in the .bashrc or in some other obscure config file, would be great to mention this if you recommend changing settings to your user base.
> My package manager was reporting my actions to Google and I didn’t even know about that. Great…
This has been a known things for a long long time. That is on you for not knowing about it. And it isn't doing anything that you are trying to infer in your statement anyway.
I really do think that people who are concerned about telemetry should install Little Snitch and also look at each of these programs documentation. Telemetry is called out for most of them, as well as how to disable it. It’s also not inherently bad as it can be a valuable tool to improve their systems, though obviously could be abused too.
As an old hand who was around when homebrew started, I feel obliged to let people know there are alternatives. If you find brew's take-over-your-system-and-youll-thank-me approach off putting, and/or you value stability over the coolness factor, MacPorts is still alive and well (https://www.macports.org), and in-fact has basically all the same packages brew does.
In 18 years of using MacPorts, it's never messed up my system, broken something that used to work or decided that I don't get to run commands right now, because it wants to run an upgrade. (It's also never spied on me.)
The price you pay for that is that more things are compiled from scratch, but hey - you only have to do that once, and it only takes like 20 minutes on a modern Mac. Upgrades are generally pretty fast.
FWIW, Homebrew uses /opt/homebrew instead of /usr/local on Apple Silicon installations, as far as I know that was the main sticking point for the complaint that it took over your system. Still isn’t terribly useful for multi-user systems as far as I know, but for my personal laptop this will never be a problem that I have to deal with.
I used pkgsrc on macOS for a while but ended up going back to Homebrew because it was easier and there were more packages relevant to my needs; I also found packages in pkgsrc wouldn’t play nicely with zsh or bash completion sometimes, I believe this was related to patches specifically required for macOS for these packages. The concept of casks is also very nice, as it means I can manage most packages using a single package manager instead of doing dmg->drag-n-drop installations or mucking around with install/uninstall scripts. Building and maintaining a dumb cask formula is also pretty easy for apps where there isn’t already an available cask.
I haven’t used Macports so couldn’t compare - can Macports install .app applications?
I've been trying Nix devenv[0] and nix-darwin[1] on Mac recently and having a blast. Definitely better than the stuff I had to build around Homebrew to make development environments consistent across machines. Nix-darwin even has support to install Homebrew packages.
For anyone curious, I have a unified setup that works across my work M1 (currently bitrotting), work Ubuntu (hurrah Intune for Ubuntu!), and personal NixOS[1]. You'll probably find my mkChoose function particularly interesting[2], which makes parameterizing attrsets and lists a little easier for different hosts (I currently don't have/know a way to discriminate between Linux and NixOS, so you need to pass a function which does that[3]).
If you aren't using home-manager[4] yet, I strongly recommend taking a look at it.
nix-darwin significantly elevates the competency of MacOS.
Yes, it is really great! I haven't had to run homebrew in forever; Nix + home-manager does it as part of my overall config. I've configured home-manager to both install and update homebrew packages.
Did you know you can even install Mac App Store apps?
I was bummed out by that too initially.But it's kind of inevitable with root not writable on the latest macOS releases and needing the store to be under /nix.
I used to be a staunch macports evangelist but ended up switching to homebrew. I've found it to be much simpler.
And, as a counter to your experience of not having it mess up a machine, every time you upgrade macOS to the next major version you have to jump through some hoops to get it to play ball again. Painful.
At some point I gave up on the hoops and installed homebrew instead and haven't looked back.
Ah, yes I did mean `/opt/homebrew` - and yes I know on Intel Macs it works differently, but it can be installed at custom locations for people who still have them.
> MacPorts is still alive and well (https://www.macports.org), and in-fact has basically all the same packages brew does.
I used macOS for a few years to try it out, and I found that Macports lacked many things I needed. Taking a quick look again, let's see...
* Oclgrind: No.
* Futhark: No.
* GHC: Yes! And updated to boot. (But I'd expect such major programs to be in any package system.)
* MLkit: No.
* hledger: Yes, although not the newest version.
* git-annex: Yes, but a rather old version.
* ispc: Yes, but again a rather old version.
The impression I got (which may be inaccurate) is that Macports has a long tail of older packages, but it does not have a lot of newer stuff. It is very easy and convenient to add packages to Homebrew (just a PR with a package definition in a Ruby DSL; most things are obvious), which I think is the main reason it has become successful.
Nowadays I suggest everyone just use Nix, of course. It's not as slick and has a tough barrier to entry, but it's built on fundamentally better principles than legacy package managers.
MacPorts had more packages I wanted and fewer outdated packages the last time I compared them. Homebrew won the time before.
Is the Homebrew Ruby DSL much easier than the MacPorts Tcl DSL? I think the main reason Homebrew succeeded was it had binary packages before MacPorts. But a more familiar language helped probably.
I recently switched back to MacPorts after 8 years on Brew. However, a few weeks ago the outdated GDAL port broke, and I lack the expertise to quickly make a new port and haven’t had the time to figure out how to (I have contributed an updated version of Catch2, but that was very easy). So I’ve had to switch back to Brew. I hope this is only temporary.
The thing that pushed me over the Homebrew back when it was new was how with MacPorts, surprisingly often packages (at least those I was installing) were broken and required manual patching to make work.
While this would be a nuisance for me even today, I was only just starting to dip my toe into programming at that point which meant I lacked the technical knowledge to perform any changes more complicated than pasting in a single line somewhere, which meant it was often game over when a package was broken.
While Homebrew has its problems it’s easy to see how it became popular.
If you're looking for an alternative for creating dev environments, you should check out Devbox [https://jetpack.io/devbox]. Our goal is to combine the usability of tools like Brew or Yarn with the reproducibility and reliability of Nix.
MacPorts makes no effort to not look abandoned. It's very much 'the old way' of doing things from the look of the website to the terseness of the syntax. Before you take offense, I say this as a member of that generation.
These days if you have an ugly website, and opaque syntax, you're considered to be an abandoned project. Agree with it or not, your website sends a signal to the user and MacPort's signal says "it'll work till it doesn't and that's fine"
What are the odds that an Apple user finds that site attractive? I come from the linux world, so fine by me, but I'm no dummy, this site looks like it's going to tell me it runs better in a modern browser like IE6.
Completely agree, I'm a brew user myself but find the site of MacPorts rather clean and modern. It's responsive and doesn't feel abandoned at all (particularly https://ports.macports.org which support Dark mode). Plus, using brew, I never go to the website so I don't think this is really important.
Man there are a lot of complaints about this free software no one is forcing you to use. I would start with a thanks to the people who spend a lot of time on this, and probably not for much wealth. I know I greatly appreciate the existence of homebrew.
Not even just entitlement, but a lot of people thrown off by the existence of telemetry. Telemetry that it tells you about when you install it, and is mentioned on their site.
So not only are people complaining about free software, they’re complaining because they don’t pay attention. It’s the “Everything someone else does is not as good as what I do” mentality
Entitlement syndrome. Nothing is perfect, not even paid software is perfect. At least open source gives a chance for someone to look into how it's done, suggest/discuss something different and submit a modification if affordable.
Reminder that nothing in Homebrew has any signing or attempt at supply chain integrity. Now that they are moving away from clients using git, so even git commit signing is off the table now.
When you install brew you are giving hundreds of people (or anyone who has access to their Github accounts) full remote code execution on your machine.
Never ever use brew on a machine you use to access production.
> Reminder that nothing in Homebrew has any signing or attempt at supply chain integrity. Now that they are moving away from clients using git, so even git commit signing is off the table now.
This is incorrect: Homebrew pins bottle and source installs to digests, meaning that you do have a strong integrity check each time you install a package.
Homebrew does not currently do code signing, which is not particularly abnormal for non-Linux-distribution package managers (ask yourself when you last verified a PGP signature from PyPI). There are some plans in progress to remedy that but, in its absence, HTTPS and package digests are both reasonable and no worse than standard practice in packaging.
If they were using signed binaries wouldn’t the same people who have permission to merge code have permission to sign compiled binaries (or, realistically, start a CI process which does)?
With Homebrew, frankly. That's the Mac equivalent to apt. I've also had success with Anaconda (specifically, miniconda / miniforge) - it's not just for Python! It's great for creating disposable development environments (i.e., I need HDF5, but only for this project).
Other than that, your toolchain is what Apple gives you, and upgrades in ~parity with XCode.
Excluding 3rd party tools like Homebrew, one dosn't. The native 'package manager' is the Apple App Store. You can also manually download and install .dmg files of applications (much like .deb or .rpm).
macOS also has .pkg installer packages that are native. The only issue is that there is no uninstall functionality built in to them unless the package provides their own.
> If you're worried about it, compile your own builds.
You forgot an important part: If you're worried about it, compile your own builds, and read and fully audit all the code before compiling. Then upon every update, read and audit all the changes. I believe this is infeasible even for the tech giants.
Didn't the author sunset this project to become a web3 advocate and pivot to a new project where some dubious NFT would be exchanged for package installations? I guess 2022 came and went since then...
You’re thinking of Max Howell, the original creator of Homebrew. Rest assured, Max hasn’t been part of Homebrew for years—for longer than he was ever a part of it—so he has no hand in sunsetting or making other decisions on Homebrew’s behalf.
Regarding his web3 stuff, the Homebrew developers learned about them at the same time as everyone else (there was a tweet from the official account at the time). There is no relation between both projects.
Max takes every chance he gets to mention Homebrew, so the confusion is understandable but unfortunate. Even in his web3 package manager he mentions Homebrew as if it’s the old stuff.
That's opt-out, though? It's nice that they tell you, but they don't give you a choice. They're just warning you that you need to opt out. That's not the same as opting in.
They're also claiming to log anonymous data while the data they collect is actually pseudonymous as its tied to your specific machine/user, which means it has a whole different status in different privacy jurisdictions. It seems like they could make the data collection anonymous relativelt easily if they just stop tracking individual user installs, though.
They don’t collect data until after you’ve been given the chance to opt out.
> It seems like they could make the data collection anonymous relatively easily if they just stop tracking individual user installs, though.
Doing that would skew the reports to more active users & automated scripts. If you’re asking questions like “how many people will notice if we deprecate this?” you don’t want to count the guy who hammers “brew update” every other command many times more than someone who updates weekly.
> They don’t collect data until after you’ve been given the chance to opt out.
That does still make it opt-out, though, even if it's a friendly opt-out. Microsoft's dotnet tools do the same thing.
> Doing that would skew the reports to more active users & automated scripts. If you’re asking questions like “how many people will notice if we deprecate this?” you don’t want to count the guy who hammers “brew update” every other command many times more than someone who updates weekly.
I know why they're collecting personal identifiers and I completely understand their reasoning behind it. Every website and service does this in some fashion because most requests are automated. You should still ask before you submit any telemetry, though.
You shouldn’t use brew for development tools and runtimes really, you should should use a language specific version manager or asdf. Brew for system stuff.
In what world does it mean “something you dont like”? It means “overzealous” and has been used in that way in recent times. Ever heard of the word “grammar-nazi”?
So im getting flagged here for using a word as it is supposed to be.
Way to go for censorship and suppressing opinions then, since i am pointing out a valid criticism of the homebrew project.
It is so overused that it has lost almost all its meaning. It's getting very boring to see it being used around loosely and carelessly, especially when the word is used to describe things that people don't like.
Excellent! Thanks to the brew team for this — it was always frustrating to install something and have to wait for a full, slow update first.