The reality is that drivers are not added, as you say. Most companies release an out-of-tree BSP targeting a specific kernel version. They often contain blobs and are often not GPL. Linux doesn't support a stable kernel ABI/API (https://www.kernel.org/doc/Documentation/process/stable-api-...) and the only way to avoid the associated issues is to mainline drivers, which most companies don't want to do (don't want to open source their IP, don't want to invest in maintaining it etc.)
Android GKI/KMI addresses the issues related to this. GKI is relatively recent and OEMs don't offer 5+ years of Android updates because they haven't adopted it yet.
> and the only way to avoid the associated issues is to mainline drivers, which most companies don't want to do (... don't want to invest in maintaining it etc.)
They wouldn't have to, that would be done for them, for free, if they got their code into the kernel...
That Linux note about why they don't want a stable API is ridiculous. This shows how unfriendly is Linux to out-of-tree software. They want third-party developers to refactor their working and tested drivers every time they decide to change something.
I think that at least open source community should move to "never break things" concept. So that you can write and test code once and never touch it anymore. Refactoring your code because kernel/library has changed its interface is just a waste of time which gives nothing in return, often time of unpaid volunteers who could do something useful instead.
This should apply to libraries, applications, programming languages, browser extensions. Take upgrading your Python 2 code to Python 3 as an example. You need to put a lot of efforts while gaining nothing. This is not how software should work. You should write a code once and it should work forever without any maintainance.
Correct me if I'm wrong, but I thought that the actual code itself is stable, just that the compiled kernel API/ABI isn't.
So if you open source your drivers and get them accepted into the kernel, then you don't need to rewrite/recompile them for each new kernel version. Just like AMD did with their drivers. And I think this is part a conscious decision that forces people to open source drivers.
But this is honestly just guesswork based on what I've read, would love to learn more!
The Linux developer community promised almost 20 years ago [1] that no release of the stable kernel will ever break something that worked in a stable kernel before. AFAICT, this promise holds. If you upstream your driver, the community will take care (cf. AMD), if you don't your users experience occasional pain (cf. NVidia).
> So if you open source your drivers and get them accepted into the kernel, then you don't need to rewrite[...] them for each new kernel version.
You don't, but someone does. In the note linked above they give an example of how the USB interface changed from synchronous to asynchronous, this must have required some refactoring in all drivers that used USB
That someone updating the drivers is exactly the same person who makes the change in the subsystem that they depend on. They cannot break what is already in the kernel, but they can break what's outside; it is after all an internal API.
What reasons are there other that corporate shenanigans?
Sure it takes more effort to mainline the driver and address review comments, but if you aren't designing a throw-away product, it's effort well spent.
Those reasons tend to be petty ones that benefit the hardware company and noone else. Seems fair that the hardware company should pay for the consequences of their decisions and not the people advancing the kernel.
I know a lot of people who are unfriendly towards sloppy work, yes. if the driver is of a good enough quality, they will be happy to take it off your hands and look after it for you!
as a user, I hate apps that aren't in the repository. it takes effort to make sure it doesn't clobber something important just because the developer wanted to use the latest and most broken version of some library. thankfully nixos allows me to run whatever version of every dependency and then nuke the whole mess once I'm done with it.
That's not going to be fixed. This is a typical M-N problem. There are too many distros and it would require lot of work to port every app to every distro. Instead, there should be a "platform" (set of APIs) that app can work with and the same app would be able to work in any compatible distro.
To be fair, if the app isn't in the nixos distro, it's rarely that useful. The only one I had to figure out was a proprietary logic analyser interface, but it wasn't too hard.
> Take upgrading your Python 2 code to Python 3 as an example. You need to put a lot of efforts while gaining nothing.
Just the python's 3 string/binary distinction and explicit conversion squashed an entire category of bugs. Yes, it required to make explicit what was implicit before, but still, that's huge achievement in my book; saying that it is nothing is a very, very shortsighted.
Android GKI/KMI addresses the issues related to this. GKI is relatively recent and OEMs don't offer 5+ years of Android updates because they haven't adopted it yet.