What ate Lisp's lunch wasn't necessarily the Lispers dreaming of a perfect system. Rather, Unix was far more accessible in terms of availability than the contemporary Lisp systems coming out of Symbolics and Xerox at the time Unix really started to take off. Before the breakup of AT&T in 1984, Unix was available to universities under comparatively generous licensing terms. While source licenses were very expensive for companies and (after 1984) for universities, binary licenses were relatively inexpensive, and beginning in the 1990s we saw open source Unix-like operating systems such as Linux and the BSDs. Unix was not restricted to a particular architecture; it ran on a wide variety of hardware.
Contrast this with Symbolics Genera and Interlisp-D, the premier Lisp operating systems of the 1980s. A Symbolics workstation can easily cost five figures in mid-1980s dollars, and cheaper alternatives such as the MacIvory boards and OpenGenera on DEC Alpha machines weren't released until later in Symbolics' history. Interlisp-D originally ran on Xerox workstations that also cost five figures, though it was later ported to the Sun SPARC architecture (though I have no idea what Interlisp-D licenses cost). Both Symbolics Genera and Interlisp-D missed out on the open source revolution of the 1980s and 1990s. To this day Symbolics Genera remains proprietary, though thankfully Interlisp-D was made open-source recently (https://interlisp.org/). There are open-source Common Lisp compilers such as SBCL, but they are not full-fledged operating systems.
Part of the reason why Unix took off had less to do with Unix's design and had more to do with Unix's lower costs. DOS and Windows had even lower costs than Unix before open source Unix clones appeared, and they were (and, in the case of Windows, still is) widely used. While Unix's design characteristics certainly play a role, we shouldn't ignore the impact of cost and licensing.
In an alternative universe, imagine had RMS had set out on building a FOSS Lisp operating system (GNU Emacs doesn't count) instead of building GNU. RMS was a Lisp hacker at MIT who started the GNU project due to his frustrations with the proprietary Lisp machine companies like Symbolics that were spun off from MIT's AI lab, so it's not unlikely to think about an alternative situation where RMS decided to clone a Lisp OS instead of cloning Unix. I wonder if a community could have rallied behind an open source Lisp OS to be a challenging contender to Unix back in the 1980s?
Lisp machines were mostly just a single type of computer: a graphical workstation for Lisp developers. Those were mostly in R&D, typically in AI - largely financed by DARPA and similar, hoping for the AI software providing leadership in tech and military. The market for their software&hardware was on the leading edge, which later was taken from offerings growing on cheaper platforms.
They had one/two operating system to fork others from: single user, no terminal story, no security, needing large amounts of memory, needing specific expensive hardware (graphics, disks, custom boards, ...), not easy to port, needed complex (and hard to debug) memory management, scarce number of Lisp system programmers accumulated in a small number of companies, no public source story, ...
Many open source operating systems are copies or forks. There was not much original research. Writing a new operating system (say, a portable Lisp OS with device drivers, multi-user capabilities, security, terminal and GUI usage) would have needed a lot of work and there are not enough combined Lisp AND systems programmers to do that - educating them takes a lot of money.
Even today, the state of the 70s/80s Lisp OS hasn't been reached again: there is no comparable operating system written in Lisp and the old ones exist only as emulators of the past.
The tech base of the early Lisp system was too focused, there was no room to mutate and grow to different platforms in different incarnations. There were attempts (like embedded Lisp hard- & software) but that had no effect in the market.
UNIX was developed for a completely different market: simpler base software, very portable, multi-user, client&server, modular programs, terminal, ... a bunch of companies used UNIX as a base for their workstation offerings (SUN, SGI, HP, IBM, NeXT, ...) - the rest is history.
But Lisp OS like software on the UNIX systems was working well in the 80s and early 90s. They still needed large amount of RAM, large amount of virtual memory, responsive GUIs, advanced GC support, ... some high-end machines could offer that and they fixed enough OS bugs to run large Lisp systems. Every larger UNIX vendor had some Lisp story as part of their general offerings.
Contrast this with Symbolics Genera and Interlisp-D, the premier Lisp operating systems of the 1980s. A Symbolics workstation can easily cost five figures in mid-1980s dollars, and cheaper alternatives such as the MacIvory boards and OpenGenera on DEC Alpha machines weren't released until later in Symbolics' history. Interlisp-D originally ran on Xerox workstations that also cost five figures, though it was later ported to the Sun SPARC architecture (though I have no idea what Interlisp-D licenses cost). Both Symbolics Genera and Interlisp-D missed out on the open source revolution of the 1980s and 1990s. To this day Symbolics Genera remains proprietary, though thankfully Interlisp-D was made open-source recently (https://interlisp.org/). There are open-source Common Lisp compilers such as SBCL, but they are not full-fledged operating systems.
Part of the reason why Unix took off had less to do with Unix's design and had more to do with Unix's lower costs. DOS and Windows had even lower costs than Unix before open source Unix clones appeared, and they were (and, in the case of Windows, still is) widely used. While Unix's design characteristics certainly play a role, we shouldn't ignore the impact of cost and licensing.
In an alternative universe, imagine had RMS had set out on building a FOSS Lisp operating system (GNU Emacs doesn't count) instead of building GNU. RMS was a Lisp hacker at MIT who started the GNU project due to his frustrations with the proprietary Lisp machine companies like Symbolics that were spun off from MIT's AI lab, so it's not unlikely to think about an alternative situation where RMS decided to clone a Lisp OS instead of cloning Unix. I wonder if a community could have rallied behind an open source Lisp OS to be a challenging contender to Unix back in the 1980s?