I think some of the pain has been Apple's fault. Requiring conda for some official packages means you always have two competing ecosystems on one machine - which is just asking for pain.
What packages are you referring to? I’ve been doing professional Python dev on a Mac for the past three years and have never had a reason to use conda, so I’m curious what I’m missing.
It’s only the last 5 years, give or take, that you can install C-compiled packages with pip. When I started out, I had to apt/port/pacman install packages, then run pip install (or python setup.py even) to install dependencies. It’s hard for newer python converts to even comprehend the pain it used to be. Conda came with the compiled package concept, and pip came after. Dealing with dependencies these days is a breeze, relatively speaking. On windows you even needed a visual studio license, just to pip install numpy!
Just because the egg/wheel format existed, didn't mean that they where actually used and actually worked. Being able to reliable pip install packages like numpy and gdal only started 3-4 years ago. There is a reason that Christoph Gohlke's python package site exists and was popular.
I started using numpy around 2007 and didn’t see reliable binary installs with good numerical performance until I discovered anaconda, much later (2016?). Maybe I hung out with the wrong people, or libraries. Some libraries did not have compiled wheels a few years ago, and M1 macs still run into the very occasional issue.
You’re running install-time code if you don’t, so I disagree. The wheel format is better suited even for pure python distribution. You can read more details on that site.
Conda environments are the most isolated from the host os, outside of using docker or a vm. They’re also the most heavyweight. As a package manager, you can install a lot of non-python stuff. It’s more similar to apt or yum than pip. You can even do things like run bash on windows with m2, which I usually find preferable to wsl.
pyarrow for example, if you don't have a wheel then good luck building one without using conda. It's not impossible but the developers don't really support it.
Official dependency and build tooling is not properly geared for C extensions. You'll be looking at compiler errors to figure out what dependencies you are missing.
Conda ships Intel MKL for linear algebra, too, meaning numpy.dot (matrix multiply), scipy, fft, solvers, … run 2-10x faster. Well, used to, I think the open source alternatives have improved since.
