I tend to just give up on a package if it requires a C toolchain to install. Even if I do end up getting things set up in a way that the library's build script is happy with, I'll be inflicting pain on anyone else who then tries to work with my code.
I know this is unpopular opinion on here, but I believe all this packaging madness is forced on us by languages because Windows (and to a lesser degree osx) have essentially no package management.
Especially installing a tool chain to compile C code for python is no issue on Linux, but such a pain on Windows.
Every language tries to re-implement the package manager, but it ends up breaking down as soon as you need to interact with anything outside of that specific language's ecosystem. The only solution for interacting with the "outside" (other languages, toolchains, etc) is a system level, language agnostic package manager of some kind.
Linux distros package management is far from perfect but it's still miles ahead of the alternatives!
I very highly recommend people to learn how to write and create Linux packages if they need to distribute software. On Arch for example this would be creating PKGBUILDs, Gentoo has ebuilds, and other distros have something similar to these things.
> it ends up breaking down as soon as you need to interact with anything outside of that specific language's ecosystem.
It works OK in nuget on Windows, due to the different approach taken by the OS maintainer.
A DLL compiled 25 years ago for Windows NT stills work on a modern Windows, as long as the process is 32-bit. A DLL compiled 15 years ago for 64-bit Vista will still work on a modern Windows without issues at all.
People who need native code in their nuget packages are simply shipping native DLLs, often with static link to C runtime. Probably the most popular example of such package is SQLite.
> I very highly recommend people to learn how to write and create Linux packages if they need to distribute software.
I agree. When working on embedded Linux software where I control the environment and don’t care about compatibility with different Linuxes or different CPU architectures, I often package my software into *.deb packages.
C tends to work in those cases because there aren't a significant number of interesting C dependencies to add... because there is no standard C build system, packaging format, or packaging tools.
When juggling as many transitive dependencies in C as folks do with node, python, etc., there's plenty of pain to deal with.
It feels so suboptimal to need the C toolchain to do things, but having no solid way to depend on it as a non-C library (especially annoying in Rust, which insists on building everything from source and never installing libraries globally).
I make a tool/library that requires the C toolchain at runtime. That's even worse than build time, I need end users to have things like lld, objdump, ranlib, etc installed anywhere they use it. My options are essentially:
- Requiring users to just figure it out with their system package manager
- Building the C toolchain from source at build time and statically linking it (so you get to spend an hour or two recompiling all of LLVM each time you update or clear your package cache! Awesome!),
- Building just LLD/objdump/.. at build-time (but user still need to install LLVM. So you get both slow installs AND have to deal with finding a compatible copy of libLLVM),
- Pre-compiling all the C tools and putting them in a storage bucket somewhere, for all architectures and all OS versions. But then not have support when things like the M1 or new OS versions right away, or people on uncommon OSes. And now need to maintain a build machine for all of these myself.
- Pre-compile the whole C toolchain to WASM, build Wasmtime from source instead, and just eat the cost of Cranelift running LLVM 5-10x slower than natively...
I keep trying to work around the C toolchain, but I still can't see any very good solution that doesn't make my users have extra problems one way or another.
Hey RiiR evangelism people, anyone want to tackle all of LLVM? .. no? No one? :)
I feel Zig could help here. The binaries ship with LLVM statically linked. You could rely on them to provide binaries for a variety of architecture / OS, and use it to compile code on the target machine. I'll probably explore this at some point for Pip.
