> I'd bet that that C, C++, Java or even Python has a numbers advantage by one or two orders of magnitude.
That's a numeric argument, not a qualitative. Common Lisp is not popular in the mass market, but not because one can't easily write large applications. Something like Common Lisp actually was designed for the development of large and complex applications.
If you develop a large C application, you could shrink by porting it to Lisp. Lisp's code density in larger applications is much higher than C and large C applications tend to reimplement half of what Common Lisp brings out of the box (objects, code loading, runtime scripting, automatic memory management, error handling, ...) - see Java.
There are systems written in Common Lisp which are much larger than 1MLoc. Some applications like Macsyma easily reached already >100kloc in the 80s - vor example the commercial version with the GUI UI. Lisp Machine operating systems had around 1.5MLOC in the end 80s.
I use a web server, which has a few 10kloc code. My development environment I use is a few 100kloc Lisp code. There are a a dozen Lisp implementations/compilers maintained, which have all >10 kloc, some have several 100kloc (especially the larger commercial ones).
Historically there have been a bunch of databases, cad systems, etc. written in Lisp, which all had/have >100kloc, PTC sells a CAD system largely written in Lisp with >7MLOC Lisp code.
Ok, given all that, why do you think "Common Lisp is not popular in the mass market"?
Remember, I'm not arguing that Lisp is a bad language or that you can not write large applications with it. My point it just that a large Lisp codebase is significantly more difficult to understand and modify compared to ..say.. C++ or Java.
Your density argument is unhelpful here, because you typically don't want dense code full of user defined abstractions if you want to maintain it for a long time.
> Ok, given all that, why do you think "Common Lisp is not popular in the mass market"?
I think this question leads to nowhere, and even so if you make up a reason. You would have first to understand the target market of Lisp and the players interested in such a market. Lisp was originally designed for symbolic AI stuff (planners/schedulers, maths programs, theorem provers, natural language systems, knowledge representation and reasoning, knowledge-based engineering, ...) - that has been its core market and this is not a mass market. It's a market of applied research&development. Over time the language has been applied to related domains and a bunch of technology has been transferred: some of the underpinnings from Smalltalk and Java were largely influenced by Lisp - even the first garbage collector for Microsoft's .net was developed in Lisp and then transpiled to C++. One can also say that Javascript is a distant Lisp dialect (eval, symbols, data syntax, closures, object system, ...) - what it basically lacks is the whole s-expression infrastructure.
Is Porsche successful in the mass-market? Probably not - they are a small company compared to most others - they are not even in the top 20. But they are a very profitable car company. Should they give up because their market share is so small? They have world-wide sales of just 250000 cars/year. Toyota has ten million.
Is a Porsche 'difficult to maintain', because it is not a mass market car?
Just as in software, there are other more important factors. Porsche's target market are people who want to own and can afford a high-quality luxury sports car. But that's not the larger mass market.
> My point it just that a large Lisp codebase is significantly more difficult to understand and modify compared to ..say.. C++ or Java.
That's your claim. Have you ever worked with a Lisp program beyond 100kloc and extended it? It's not that difficult. It just looks different from what you have seen. In Java you work with the dead code in a text editor. In Lisp often you extend the running program - thus maintenance is done while you interact with it (similar how you write scripts for Excel, write code in Smalltalk, ...). For example for my Lisp IDE I never got a new version during the last two decades, beyond the usual new releases. All I got was patches which are loaded into the application - maintenance gets easy.
But where is the factual evidence - all I heard so far were guesses based on your popularity claims? I think you would need to look beyond that. There are technical reasons why Lisp is not popular like C: like Lisp is usually not a systems language, thus it can't compete with C in that area - which was designed for that -> this leads to the fact that most infrastructures are C-friendly and low-level. Apple writes their base OS in C and C dialects. On the iOS store the majority of applications are written in Objective-C - since that's what Apple supports (plus a bit of Javascript and Swift). Other large corporate players have invested zillions into their own infrastructure (SUN/Oracle -> Java). Now your language can be as great as possible, it will always be second tier or it needs to find a specific niche (Javascript -> webbrowser, PHP -> web sites, ...). The specific case for which Lisp was developed doesn't have a corporate sponsor (in the 80s it was DARPA for Lisp) and the niche markets for it are relatively small.
> Your density argument is unhelpful here, because you typically don't want dense code full of user defined abstractions if you want to maintain it for a long time.
That's another assumption. But domain-level maintenance is best done in a domain-level language. Much of the value of a software is at the domain-level.
The user defined abstractions are in any software. It's just how they look like to the user. Typically one prefers domain level user interfaces in maintenance, instead of having to deal with the machinery. That's why we write 'make' files (and the equivalent), instead of writing the code directly in C to compile software systems. It's easier to maintain the information in a 'make' file, even though you have to learn how to write and read them. That's the same for Lisp - it's easy to write a script to compile a bunch of Lisp files. But most of the time we prefer a domain level Lisp tool:
(sct:defsystem cl-http
(:pretty-name "HTTP Server"
:default-pathname "HTTP:SERVER;"
:journal-directory "HTTP:LISPM;SERVER;PATCH;"
:initial-status :experimental
:patchable t
:source-category :basic)
(:module pointers
("SYS:SITE;HTTP.TRANSLATIONS"
"SYS:SITE;CL-HTTP.SYSTEM"
"SYS:SITE;W3P.SYSTEM"
"HTTP:LISPM;HTTP.TRANSLATIONS")
(:type :lisp-example))
...
(:serial
showable-procedures
"PACKAGE" ; HTTP Packages
"PRELIMINARY" ; Showable procedures
"VARIABLES" ; Variables and constants
#+Genera lispm ; Load Lisp Machine specific code
"BASE64-ENCODING" ; Base 64 utility RFC-1113 and RFC-1341.
"MD5" ; MD5 Digests based on RFC 1321
"SHA" ; SHA Digests based on Internet FIPS 180
"TASK-QUEUE" ; Thread-Safe Queuing Facility
"CLASS" ; Server Class Definitions
"URL-CLASS" ; URL Class Definitions
"HTTP-CONDITIONS" ; HTTP
...
)
If you happen to know some Lisp, that's easy to maintain: During maintenance we just add the new configuration info - that's much easier, then to change a low-level program.
Where it gets more difficult is when we need to change the implementation of these tools. Like when you need to fix a bug in the 'make' program itself.
Every program has these abstractions at the domain level: either they are internal or external domain specific languages (DSLs) or they are complex frameworks at the OOP or functional level - which makes them very difficult to use. Lisp happens to be able to have an integrated way to expose these complex machineries in maintenance-friendly domain-level constructs. For an example see GNU Emacs which comes with a million lines of domain-level Lisp code.
Also, how many of these codebases attract new contributors regularly? That's a pretty direct metric for readability of a codebase.