I played with it for a while as the base OS for my KVM hypervisor servers.
A couple of years ago, I tried to find all the source archives for the old LFS releases. I ended up using the Debian and kernel.org archives to find most of the old package versions. Packages from before 2004 started disappearing like as86 and the PowerPC patches. I was never able to find the PowerPC patches because they were never mirrored on anything but FTP sites. The Internet Archive did not pick up any of those mirrors.
https://archive.org/details/linuxfromscratch-releases
I eventually replaced the Musl-LFS build and replaced it with a buildroot set of configs and scripts when I had issues replicating my work with later package versions. Buildroot allowed me to more rapidly test and deploy my hypervisor config. I wanted a simple /etc/rc init like what was in SunOS 4, no GRUB, and QEMU with VDE networking.
I ended up with SYSLINUX for BIOS MBR and gummiboot(before it was converted into systemd-boot) for EFI because my EFI machine (Dell R810) did not allow for direct booting of EFI binaries.
Well, what I remember from it (about 20 years ago now) is mainly this: make/compiler error -> search -> modify obscure file -> repeat. For days in row. At that time I didn't exactly know a lot about programming so most of it was just dumb copy-pasting without knowing what was going on. I do think it was useful (hard to tell - I seem to be able to fix pretty much everything in the SW depertment these days but that's probably a sign of attitude in general, much less thanks to LfS), but also lead to a bit of a distaste for linux. I.e. after having spent weeks like that with lfs, and then again when deciding to go for gentoo, and then again with debian, there wasn't much fun in it anymore. Only a 'been there, done that, now give me something which just works' feeling.
Does systemd vs init.d come into play here? Those are the primary boot/startup frameworks, correct? Is LFS only init.d? Or do you make your own bootstrap process along the way?
The first time, it is whatever is suggested. Then whatever works for You. After doing LFS or similar, You have most of the knowledge to choose. Me, I use rc files, but then I'm a Slackware Linux/OpenBSD snob.
(Oh, init.d drives me up the wall. Learn which kill signals do what You want)
Edit: s/ if / of /
Heads-up: You linked to Beyond LFS, the book for extending an already finished LFS system. The book for installing LFS is https://linuxfromscratch.org/lfs/read.html, which also has a regular and a systemd version.
I have my own LFS and no systemd... neither init.d. I have only an idiotic parallel startup SH script (busybox), already overkill. The main trade-off is I do compile as built-in the linux SH script support module.
LFS is great to learn how a Linux based system works without all the sweet, sweet magic modern distributions give us.
On a personal, nostalgic, note: running LFS on your laptop in my university was the ultimate Linux nerd cred. A step below, immediately following, there was Slackware.
The reason I feel confident enough working with Linux today is because I started back in the early '00 using Slackware, which recently had a new release[1].
Don't get me wrong, compiling your own kernel, solving shared libraries compilation hell, messing up your init.d and starting over, leaving your Pentium 4 compile for several hours was a massive waste of my 20s. But, boy, did I enjoy it!
>Don't get me wrong, compiling your own kernel, solving shared libraries compilation hell, messing up your init.d and starting over, leaving your Pentium 4 compile for several hours was a massive waste of my 20s. But, boy, did I enjoy it!
Apropos to this, I remember back in early 2010s arch linux IRC had a bot which used to make fun of Gentoo for its compile time. One of them was something like "Watching compile text fly by is fun for the whole family". I still think about this some time. Which is funny because in day to day I don't use any compiled languages :P
Edit: forgot to quote relevant text.
PS: If someone does remember the exact quote, please tell me, it would make my day.
Thank you so much. I don't know where the fun for whole family part came from though. Probably me mixing two quotes together. But you just gave me back some of my teenage. Thanks again.
> Don't get me wrong, compiling your own kernel, solving shared libraries compilation hell, messing up your init.d and starting over, leaving your Pentium 4 compile for several hours was a massive waste of my 20s. But, boy, did I enjoy it!
I had the same experience. I'm surprised how far I went with tasks I wouldn't particularly like to attempt now, purely motivated by the reward of tinkering with things without a sense of fear.
Haha, agree! So fun seeing this posted after so many years of doing this myself! Let's get this upvoted and into the hands of the next generation so to speak!
So much you can learn and like you say "dispel" from doing this. Even if you're 'just' running a bunch of scripts with the right amount of curiosity paired with it it can teach so much! Like trying out a bunch of optimizations on the C compiler and actually getting programs to fail because of it and such. Fond memories!
Personally I "moved on" to Gentoo at some point. And I remember "arguing" w/ my dad about compiling my own operating system from scratch. Of course I was that "teenager with lots of time on my hands" and he was working on production systems w/ uptime guarantees. Thanks for the fun memories in crazy times!
I still believed compiling everything with proper arch and cflags would make the system run a bit better than using distributed binaries for hardware's from 20 years ago.
Another great memory, thank you! I do remember setting up distcc just for trying it out (and fun) and knowing how to do it. It was completely not really worth it for what I was doing, because the only other machine I had was the old Pentium 200 MMX (! ;)) I was using as my router. But it sure was fun to do and learn from. So many things "kids these days" probably won't experience as there's a cloud option for that that already does this for you and better.
Then again, I suppose there are so many things we never got to play with (and instead played w/ the aforementioned). Like when I told my dad I was playing around with vmware and xen and such and he'd just be like "Oh yeah right, virtualization, like what we've been running on our IBM mainframe since like forever". But sure, have fun, explore, learn!
If anything, Linux and all this free software has done a great job of taking this out of the hands of huge corporations and having to work for them just be allowed to play and learn. Anyone could do that now.
Hehe yeah I can imagine! I mean on some level "infuriating". "Kids these days".
Like me telling him how there are people on usenet and mailing lists that don't know how to quote properly! And he'd just chuckle and tell me how quoting a whole thread could actually be good. I was furious (not him). How could that be? FWIW, I still think I was right for the usenet and mailing list use case, where anyone could just see the whole thread. He was right as well though for corporate conversations like I learned a few years later at my first job where you were glad that everyone in the thread did a full quote and 5 levels down in the first forward you got you actually saw what was actually being asked and could shortcut a whole escalated discussion with a simple "Err guys, the original question was just X, here's the answer. Can we all just go back to work now?".
Pros and Cons. Everything has at least two sides. Nothing is like it seems. Ever.
One of the best things I ever did was learn how to make an instruction decoder. Logic gates were easy enough, and flip-flops made sense; it's easy to see how to use flip-flop gates plus a clock signal to make a clocked computer, but getting all the way to instruction decoder made it all come together. I forgot the actual game that taught me, some zach-like (but not actually a zachtronics game) on steam. Still, it really means it all click.
I'm a hardware engineer, and enjoyed playing this in my spare time. You create small components from nand gates with an extremely simple made-up HDL and gradually integrate them together into larger and larger functional blocks until you have a basic but working CPU.
Is MHCD "game fun logic", or will I understand some meaningful concepts via play? I'm not looking to get into CPU design in the real world, but a teachable moment crossed with "how things work in abstract" is interesting.
Linux has changed so drastically from the 2.6 kernel days I'm a bit out of touch myself. We were still using the 2.6 kernels in highly optimised environments a year or two ago.
Setting up a cross-compiler with LFS helped me understand portability. There's so much that we don't consider about the relationship between the lines of code and where it'll be running. ARM is sort of making a comeback, so perhaps it'll become more important.
One of the most fun ways to see how 'simple' it all is, is to add 'init=/bin/sh' to the kernel command line. The kernel starts, you get a shell. It's then possible to do a lot just with shell built-ins (e.g. 'echo *' for ls).
I did this to get Linux on an original Bondi Blue iMac. I had to cross compile a kernel with USB support. I was also naive and didn’t read the gcc build instructions so I only did a single stage bootstrap. That was the most oddly unstable system. I finally realized my error did a proper three stage bootstrap and rebuilt everything.
I did it out if necessity and there was no excellent documentation like LFS, but it was still doable and extremely educational. I highly recommend it.
I did this a very long time ago, really helped me understand what goes into building a running linux environment. I also had expensive dial-up internet at the time, so had some lessons in patience too!
I still recommend this to people interested in learning more about linux and operating systems.
Question for Linux gurus: if I use Ubuntu as Windows(i.e. only run software and some cli experience without touching anything else), what's my next stepstone to LTS? I don't really know much about Linux and frankly Googled my way out if I had to, say run a bash script without typing "bash", or to find where a piece of software is installed.
I do write programs in Python and SQL but never touch compiling things, except in hobby projects which involves VSCode and some gcc.
I'd advise to try to install a Linux to a VM, and if you have a bit of experience with VMs, try LFS in a VM. Try VirtualBox if you have no prior experience. With a VM you're in complete control, you can try anything, fuck up like nothing matters and restart, pause, make snapshots and use them as checkpoints. Googling commands is not a problematic thing at all, and if you run a VM then you have the safety of the host OS to listen to your favorite music and google anything you come across.
You could start out with LFS in a VM, and would likely manage if you kept up.. Most is copy paste anyways... A youtuber called Ncommander did a livestream a few months ago.
If you want a gentler slope, maybe something like (in this order, and keep doing stuff on a "earlier level" on the list):
1) Install packages via your packagemanager from the cli and other sysadm tasks
2) Change settings for stuff by files and not by gui
3) Install things not packaged by your package manager
4) Try arch (no compiling but a lot of setup)
5) Try gentoo (lots of setup and lots of compiling)
6) Try LFS in a VM (NEVER on a system, that you actually need)
NCommander had a multi-part stream of building LFS up to Firefox. He's gotten up to building Xorg and the Firefox prereqs so far. I don't know when he plans on doing the final push to building Firefox.
I'd say get more used to Linux first. I tried LTS with very little prior Linux experience 10 years ago and failed miserably. Small documentation issues like a wrong version or an incomplete command line argument took me hours to find and fix.
Maybe using Arch or even Gentoo would be a sane next step.
Thanks, I'll try out Arch. Actually I dubbed into Arch many years without any Linux experience and failed before the 10th step. Now that I have more Linux experience maybe I'd fair better.
To get the full benefit of Arch you should try it the whole way: not just installing packages, or installing pacman to get stuff from AUR, but also making your own PKGBUILD to understand the different steps involved in getting a "foreign" source code to turn into a usable software in an automated, reproducible way.
In typical Linux fashion, this website is a complete UX nightmare. Where do I click? Why are there so many links? Why does the navigation bar on the right change? I'd like to learn Linux from scratch, but this is making it very hard for me...
Of course there may exist interpretations that bring your statement to the level of debatable, but if you keep them inexplicit, should the reader divinate?
Depends on what your goal is. If you want a useable OS, it's not worth the time. If you wish to learn about compilation and the "core" of Linux, then go for it
The outdated process is pointless. There are way too many rinse and repeat compilation steps. Focus the manual effort on the concepts that need to be taught and automate the rest. I feel like they make the default runbook overly tedious to give you a false sense of accomplishment.
> Even benchmarks showed Ubuntu was faster than Gentoo.
Source/details? I would easily believe that Ubuntu was nearly as fast, or perhaps even exactly as fast, but I can't think of any way for a custom-compiled program to be slower than a distro-provided binary.
I actually can't find exactly what I was searching for when making that comment, it may have been 10 or so years ago.
In my searching, did find this which shows trading blows with others... yet to see a comparison showing that compiling everything leads to performance gains...
In fairness, I'm sure Gentoo doesn't always win; assuming Ubuntu compiles with something like `-O2` and some lowest-common-denominator march/mtune, plenty of programs won't meaningfully use newer CPU features nor benefit from O3, or will barely benefit (below the noise floor of the benchmark). I mean, at the end of the day you're taking the same-ish programs and building them with the same-ish compilers; if the only difference is that one distro compiles for a newer exact CPU target, it'll win sometimes but not always. I was just surprised at the notion that it could ever lose, since anything Ubuntu could ask gcc to do, Gentoo could do just as well; unless it's something like Gentoo happening to lag a gcc version at the exact time of the comparison, it shouldn't be able to be worse than Ubuntu. (Unless you count the overhead of doing the compile, in which case it can be quite easy to beat:])
Bad benchmark possibly - Ubuntu presumably has some optimizations enabled by default, and portage could've been set to build with no optimizations (not used gentoo before, so I could be misunderstanding something).
Could also be talking about boot time maybe - isn't SystemD supposed to boot faster than OpenRC much of the time?
Boot time crossed my mind, too, but Gentoo is happy to run with systemd so that's not really "Gentoo vs Ubuntu" so much as "this exact variant of Gentoo vs default Ubuntu" which is kind of fair. (Assuming, of course, that OpenRC is slower to boot than systemd, which would also be an interesting benchmark)
Overdoing it does seem quite likely - if I didn't know and was trying to do a test I could easily see myself just enabling all the optimizations and moving on.
I would as well. My understanding is that SystemD does more parallelization/magic to decrease boot time, but is a bigger system to try to comprehend and has more surface in general.
I believe parallelization is a configurable option on OpenRC though, I'll add that test to the list to try some time.
> Could also be talking about boot time maybe - isn't SystemD supposed to boot faster than OpenRC much of the time?
The first time I switched to systemd, years ago (when Debian moved to systemd), I noticed the boot becoming much faster. So, yup, it was a thing. But now, years later, I got rid of systemd and honestly I don't see my system booting much slower. I mean: if Debian takes, say, 0.3s to show me the login prompt while Devuan (the Debian fork with all the systemd stuff stripped away) takes 0.4s, who cares...
(fwiw I always boot into text console, log in there, then manually launch the graphical display)
I don't think that the "faster boot time" is an argument in favor of systemd anymore, at least not on a modern system. And in case you need to spin a container in a heartbeat, you may be using some ultra minimal Linux distro like Alpine anyway.
Never messed with LFS. I used Gentoo (and then Arch) when I had the luxury of more free time. Very fun to tinker with.
With Gentoo, waiting for your DE or web browser to compile was a great excuse to step away from the computer and go for a walk. The little things in life.
I built a Cross-LFS setup for my SGI Indy, some 10+ years ago. It was an awesome experience that took about a whole week due to the Indy's 150 MHz CPU and only 128 megs of RAM. But it was totally worth it :)
This is missing CLFS, Cross-compiled Linux From Scratch, another amazing LFS project. It's unfortunate that the project apparently has been abandoned some years ago.
It actually teaches you better to learn the inner workings of Linux if you follow Gentoo (or the likes) that makes you actually type in each of the steps to get the OS running like setting up swap and configuring kernel options but LFS tells you what the OS is made of like what packages are required to get a minimal system going.
I remember doing this because I thought it would teach me to be proficient with Linux. After hours of spamming compilers and watching things build, I gave up. I realise now what I was trying to learn was how to use a Unix system, ie shell commands and the inner workings. This will teach you more about how to build your own Linux distribution.
For me it taught, that how open independent systems, talking same language, can come together and build something truly open. If you don't like one library, you can write/find another and just plug that in. Such beauty.
In fairness, I struggle to imagine that someone who finished LFS wouldn't be at least decently familiar with unixy systems; it's a load of practice running stuff in a shell and if you're paying any attention it makes you look at most of the core packages installed on most systems, so you should walk away with passing familiarity with all those parts.
Then you should try slackware, install a minimal system and compile the rest. It helps if you have programmer experience since, although there are build script for every package you need to find the dependencies yourself by looking at error messages.
This one seems to be popping up every few weeks. I really enjoyed successfully bootstrapping my own Linux box from scratch after a few unsuccessful tries. Then you have this box that helps you realize how difficult it is to maintain a Linux box without a package manager :P
> Then you have this box that helps you realize how difficult it is to maintain a Linux box without a package manager :P
Indeed. I will say though, I'm seriously considering using this for a custom built appliance, probably for audio work - I always get annoyed at purpose focused distros because I can't quite fit it all in brainspace, and then it updates and changes under me. Audio specifically because I can never remember if my laptop is set up with Pulse/ALSA/JACK/OSS/Pipewire, and always break things when I try to set things up (I'm kinda dumb).
But perhaps a custom LFS which never needs to be changed or connected to the internet after installation...
There is a rust version of a lot of utilities. Unfortunately (if we believe rust evanghelists, that rust is the best thing since sliced bread) 1) they are in an early stage of development and 2) they are mostly reinventing the wheel having different options than the standard utilities.
Or How to waste a few days copy/pasting configure commands, watching the output of make to be left with a minimal, insecure and unmaintainable system. Only think it will teach you is why distros and package managers exist.
> to be left with a minimal, insecure and unmaintainable system
Minimal yes, unmaintainable maybe, but insecure? Really depends on your threat model; courtesy of "minimal", there's not much attack surface to go around.
> Only think it will teach you is why distros and package managers exist.
And more importantly, how a distro works and what a package manager (and package build process) has to do. LFS isn't a terribly practical output for many people, but building it is a good learning experience regardless of if/how you use the built system.
I agree it's important to understand how linux eco system work, if you are interested.
I don't think LFS does that, it's simply page after page of .configure..make..make install. It would be more valuable to read a sysadmin book on unix/linux and play around with a minimal distro.
Do you think distribution managers (the staff that work at canonical, etc) are just born with the ability to create distributions, rather than it being a learned skill?
Just because you don’t see the value, doesn’t mean it has none.
Agreed. I went through about 70% of the runbook last year before running out of time and giving up. I used Gentoo in the early 2000’s and building an LFS system had been on my Linux bucket list since then.
Maybe it’s just me but the instructions seemed to focus too much on what to do and not nearly enough on why. I came out of it thinking that I learned much less than I expected.
> Maybe it’s just me but the instructions seemed to focus too much on what to do and not nearly enough on why. I came out of it thinking that I learned much less than I expected.
https://github.com/dslm4515/Musl-LFS
I played with it for a while as the base OS for my KVM hypervisor servers.
A couple of years ago, I tried to find all the source archives for the old LFS releases. I ended up using the Debian and kernel.org archives to find most of the old package versions. Packages from before 2004 started disappearing like as86 and the PowerPC patches. I was never able to find the PowerPC patches because they were never mirrored on anything but FTP sites. The Internet Archive did not pick up any of those mirrors. https://archive.org/details/linuxfromscratch-releases
I eventually replaced the Musl-LFS build and replaced it with a buildroot set of configs and scripts when I had issues replicating my work with later package versions. Buildroot allowed me to more rapidly test and deploy my hypervisor config. I wanted a simple /etc/rc init like what was in SunOS 4, no GRUB, and QEMU with VDE networking.
I ended up with SYSLINUX for BIOS MBR and gummiboot(before it was converted into systemd-boot) for EFI because my EFI machine (Dell R810) did not allow for direct booting of EFI binaries.