1) SBCL working very well on the big three PC platforms. 2) Bindings + easy installation for popular libraries. 3) Education (pg's articles, everything on planet.lisp.org) 4) Solid debugging, perhaps with things like breakpoints.

There is much activity on all 4 problem items that I identified above, so Lisp IS getting more popular. I don't know if programming.reddit.com can be a scientific gauge of how popular Lisp is, but it seems to be mentioned increasingly often.

Of all the problems Lisp has, parens are much less than the problem that it takes a gigantic effort to use Lisp with any popular libraries, once you step outside of the way that the larger community is using it.

The only (!) thing Lisp has to do is efficiently and absolutely abstract away XML, CSS, and JavaScript. No Lisp is going to get any uptake otherwise. We could do this with Arc.

One thing it took me a while to realize about the Lisp world is why there tends not to be standard frameworks or libraries for some of this stuff. The answer is that, for certain kinds of problems (basically, anything involving metaprogramming), it's actually easier to solve your problem yourself than to learn someone else's framework and then use it to solve your problem. A good example would be unit testing... there are probably a dozen unit testing frameworks, none standard, none much used. The reason is that the frameworks don't add much value over what a programmer can easily do for himself, with all the advantages that implies. I think this may also be why web app frameworks are less prominent in the Lisp world.

This is not the case with all libraries, of course. CL-PPCRE adds huge value, to pick an obvious example, but not in the class of problem I'm talking about.

This is very different than abstracting away HTML, CSS, and JavaScript, which is where I think the real problem is.

I shouldn't have to learn 3+ of different languages (HTML, CSS, JavaScript, etc), AND 3+ different variations of each (Mozilla, WebKit, IE, etc), AND multiple versions of each, AND another abstraction layer (Parenscript) just for basically macros, just to write a web application.

If so, it's a fair distinction. Probably it would be more accurate if I said abstract over rather than abstract away. We can get rid of a lot of repetition this way, but not a lot of the details that remain after that. Still, that's a big deal; better than anything else I've seen by far. So while "abstract away" may be an overstatement, I think that "just lets you write HTML, CSS, and JavaScript using a Lispy language" is an understatement. It doesn't just let you write those things - you don't need Lisp to do that. Similarly, saying "just for basically macros" (my emphasis) reads like an oxymoron to me. Macros are a big deal!

If you can do better, I definitely want to know. But my definition of "better" includes being usable in a standard browser, and "usable" includes performant.

http://www.defmacro.org/ramblings/ui-dsl.html

Most of lisps problems these days stem from social issues and logistical issues. The core tech is excruciatingly sound, and the language itself is great.

To someone out in the startup circles, it almost seems like Lisp and Scheme focus more on being a standard than being a pragmatic and global platform. While noble, this is not the path to massive popularity. Some people might be okay with that, but it seems to me like you could serve both goals and end up with an overall better product.

You do get that this is exactly what Lisp hackers who build web apps do?

HTML, on the other hand, is easy to abstract. I'm glad I did it in Weblocks, it saves me an enormous amount of time. ASP.NET uses this strategy (use the DataGrid control for a while, and then try coding without it).

JS is somewhere in the middle.

I was actually complaining here about this a few days ago. What activity on this item are you referring to?

Breakpoints as such are probably superfluous, since it's so easy to insert "(break)". But I wish I could single-step through s-expressions and have them highlighted in the editor, and I wish there were an easier way to inspect state. Actually it doesn't seem that sldb is so far away from what I want (with the exception of single-stepping and highlighting), yet I always find myself reverting to the equivalent of printf debugging.

http://www.sbcl.org/manual/Debugger-Policy-Control.html#Debu...

Edit: I seem to recall sldb works better with SBCL (which I'm not using). Perhaps that's my problem?

I think the sad reality is that most programmers aren't very smart, so smart languages will never become popular.

I'm not sure why it matters though, there are plenty of CL libraries (and jobs), so if you like lisp you can get by just fine.

This is nonsense. You can't just 'press keys and go home' and make working software. Most of the programmers I've worked with thought a great deal about programming.

Anyway, Lisp (and many other languages) gives you a lot of tools to make programming more thinking-intensive and less typing-intensive. Since typing is easy and thinking is hard, it follows that many people will prefer the language that makes you type.

JMHO.

In order to understand good Lisp code you need to learn new tools for expressing procedural concepts. Tools that are better, for that task, than natural language. Most people would think there is no such thing. Most people will always stick to whatever feels more intuitive at each stage of their formation as programmers. This is often a handicap to learning to think in more powerful ways.

No wait: on second thought, I surrender to how completely dumbfounded I am by this comment.

I'm calling Kenny...

Perhaps what you mean is that when a construct is unfamiliar to a programmer then they have to think harder about how to use it?

http://www.google.com/search?q=blub

No, I don't mean unfamiliar. Things like parentheses and prefix notation are unfamiliar to many people, but once learned, they don't require much more cognitive processing than other notation. There are several things in Java that are similarly weird until you learn them.

Certainly if you're habituated to the object-model style of programming, other forms of abstraction may seem weird. The irony here is that twenty years ago, the people making your kind of argument were directing it all against objects.

As a side note, treating programming objects as if they were physical objects works as long as there's a good fit between the two. As soon as you need the objects to behave in ways that physical objects don't (and believe me, in any complex OO system, you will), you find yourself tied up in knots that it will take a lot of thinking to extricate yourself from. (Speaking of complexity that isn't intrinsic to the problem...) Not accidentally, the tools that people reach for then tend to be ersatz versions of metaprogramming (reflection, code generation), and Greenspun is off to the races...

It's not. It's just a matter of practice. Most people find OO modeling natural (even when it is extremely tortured and difficult) because they've acquired a comfort with it born of many hours of practice. You might argue that you're not referring to familiarity with the paradigm, but I think you are, and don't realize it.

I assure you that a functional style has equal power to abstract as a OO style. You're right, our brains have a good ability to think of things as sets of objects. But the lisp style leverages the linguistic propensity to give words special significance. We're also very good at learning languages and parsing sentences in context, so the lisp linguistic style leverages similar natural cognitive abilities.

Not me personally, though.

But the lisp style leverages the linguistic propensity to give words special significance. We're also very good at learning languages and parsing sentences in context, so the lisp linguistic style leverages similar natural cognitive abilities.

Linguistic processing does require more cognitive effort, especially when it involves something like a macro that can change the context in which it is invoked. For exactly the same reason, most people are more productive with a graphical user interface than with a command-line one. This is directly related to the fact that abstract linguistic processing is more difficult than manipulation of the tangible, physical objects you find in a GUI.

I can't resist pointing out again how laughable this sounds to anyone who remembers the long debates about how hard it is to "think in objects" and how objects "would never work for the masses". (Actually, that was right, because the "most people" in question are still largely writing much the same code as they always did and always will; now they just surround it by "public class".)

I'm not disagreeing with you at all; I just feel compelled to make sure that somebody else sees this irony!

You don't have to think about how the macro is written or what it's expanding into when you use it, you just have to know the semantics of the macro. "When I pass such and such, it will produce code that evaluates this in that way". Similar to a function or a language primitive right? If you're writing a for loop in Java, do you have to care how it's implemented in the JVM?!

As for writing macros: you're writing a function that takes an AST and returns an AST to use as replacement. This function will be called at compile-time to change your nice high-level macro call into lower-level code. There are a couple pitfalls (variable capture, multiple evaluation) but they're always the same and you'll learn about them quickly enough that it will become a sixth sense.

Or the performance characteristics of that library.

Or the fencepost cases of a loop.

Or which exceptions to handle.

...

I agree that this is impressive, but remember that you get to cheat: You build the thing, measure how far off you are, and stick delay lines in the short branches until everything lines up. Admittedly, if you're off by a hundred meters the required delays would be embarrassingly large, and the architect would get teased a lot, but even then I don't think it would pose an insurmountable problem.

Meanwhile, if the essence of your argument is that Lisp was used by mainframe guys and wasn't pushed hard enough in the microcomputer world, I can't help but agree. God, if only I could travel back in time and replace all those useless Pascal books I read as a teenager with Lisp books...

they call that Agile Microwave Engineering

oh, and you must have two microwave engineers working on each delay line.

My hypothesis is that Lisp is unpopular for the exact same reason some of us love it: concise code. Everybody praises concise code, but humans love looking at things that have repeated patterns. The more you compress information, the fewer repeated patterns there are to look at.

I wish PG well with Arc, but I don't think popularity will come from fixing anything that's wrong with Lisp. By my hypothesis, Lisp only becomes popular by fixing what's right with it.

Another barrier is having to learn emacs/SLIME to have a decent development environment. That is a big piece of yak shaving for the average developer. Hopefully Cusp in Eclipse or something else will help.

Perl, PHP, Python, Ruby -- none of these emerged from the womb with a full-grown development environment. They managed to grow to the point where people wrote Eclipse plugins or what have you. Lack of a "mainstream" development environment is not a reason for Lisp not to grow in popularity.

Ralphc's point wasn't about there being a clear winner in the dynamic language space, but that each of the languages mentioned had one canonical interpreter.

In that case, I think that there might be a better chance of a sub-set of Lisp dialects becoming popular if they were described as separate languages. It will be interesting to see how Arc develops from MzScheme, and whether or not it becomes more popular with non-Lispers.

Edit: I don't mean these are "failures" now, just that they're doomed in the long run. Does anyone think they'll be able to stand any of these languages in 2018? I don't they'll have evolved much by that point either. The older a language gets, the harder it is for it to evolve. And if it tries to make too big a leap, people simply don't go for it (PHP5, Perl 6).

"I think that, like species, languages will form evolutionary trees, with dead-ends branching off all over. We can see this happening already. Cobol, for all its sometime popularity, does not seem to have any intellectual descendants. It is an evolutionary dead-end-- a Neanderthal language." - http://www.paulgraham.com/hundred.html

Python and Ruby especially are growing in popularity, and I think it's awfully premature to consider them evolutionary dead ends at this stage in the game.

   common lisp: (lambda (x) (+ x 1))
   C#3        : x=> x + 1
   ruby       : { |x| x + 1 }
   python     : lambda x: x + 1
   Haskell    : \x -> x + 1
   

No, the problem with trying to get a lisp under your belt has alredy been mentioned. Compared to the alternatives;

- choosing an implementation is a research project (clisp? sbcl? allegro?) - documentation is weaker - libraries are harder to find and install (ASDF vs Rubygems, Python eggs, etc.) - emacs/slime is an investment to learn - examples are few and far between - the language itself (the words, not the punctuation) is inelegant (mapcar, setq, cdadr)

I've tried really hard to learn lisp, and it turns out that actually, while there may be some platonic lisp that IS great, actually programming common lisp, now, is a massive effort in struggling in the dark.

You obviously put a lot of sincere effort into learning Lisp and I agree with much of what you say. After all that work, I wish I knew an easy way to communicate the joyful side to you.

[1] http://www.gigamonkeys.com/book/practical-a-simple-database....

[2] http://www.gigamonkeys.com/book/practical-building-a-unit-te...

I really should get that book in paper form.

To find out for myself, I plan on one day doing something large and messy with arc. I pick arc cause I figure in the eons it will be before I get around to it, arc will still be around (s/arc/a lisp/)

OTOH, some languages are particularly good on small things, but don't offer much help for large problems.

The "I did": It seems very sensible to use s-expressions for everything. That lispy syntax meant that I started seeing ways to encode almost everything as lisp. I even started keeping a todo list in s-expressions;

(do (buy bread) (tidy (kitchen living-room bathroom)) (get life))

The classic XKCD cartoon had it perfectly: "I felt a great enlightment. I saw the naked structure of Lisp code unfold before me. The patterns and metapatterns danced. Syntax faded"

I didn't: when it came to actual common lisp, cracks started to appear. The crazy mini-language for looping. The arbitrary-sounding names. The feeling that, if I ever wanted to do anything remotely windows-specific I would face years of horror.

I've blooged about it more at http://www.stevecooper.org/2008/02/13/impractical-uncommon-l..., if you've got too much free time. ;)

I think what happens to most of us who love Lisp is that, at a certain point, the initial frustrations are eclipsed by how much more productive we are (and liberated, if I may put it that way). That point comes sooner for some than others. In your case, not soon enough. But Lisp will no doubt still be around if you ever do decide to get back to it.

once Arc gets its own independent implementation i will probably start using it more heavily and seriously

Another thing you'll find is that for trivial problems, Lisp might take a few extra characters, but it scales much better. Try making your Python lambda print the value of x first. Try telling your Ruby lambda that x (and x+1) fits in a machine word so it can compile the function into only a couple of CPU instructions. Try writing a metacircular evaluator in C#. :-)

When you're just getting started, in any language, it doesn't really matter which implementation you pick. (FWIW, CLISP is byte-compiled, SBCL is a native compiler, and Allegro costs $600. Not unique to Lisp: you can find expensive proprietary C++ compilers, too.)

As a Lisp and Ruby programmer, I disagree that Lisp's documentation is weak. I would be very happy if Ruby's docs were even half as good as Lisp's.

I type "sudo apt-get install library-name" for both Lisp and Python libraries, so no difference there. (Ruby gems are less frequently packaged, and aren't compatible with the FHS, so they're a little harder.)

Lisp examples seem to be more often found in (dead tree) books, while Ruby and Python examples are more often found on the web. Since Ruby and Python libraries (and even syntax) are still in flux, and Lisp predates the web, this kind of makes sense. If you can't buy a couple books, it can be rough, though some good Lisp books are becoming available on the web for free.

Those symbols you picked are a little weird, but when's the last time you actually saw a cdadr? It is at least consistent (which is more than can be said for some languages!). I don't find "unshift" or "__gt__" any better.

I don't want to sound like a pure apologist. I admit that Slime is an investment (if you don't already know Emacs). Apparently there is a Lisp plugin for Eclipse, but I know nothing about it. It is a shame more IDEs don't support Lisp well, though I can see why they don't.

Finally, it could be simply that Lisp doesn't map well to your mind. (I don't mean this as an insult -- everybody's different. Maybe Smalltalk or Prolog or Sisal is your cup of tea.) I don't think Ruby maps well to mine, though I know some people who seem like Matz reincarnate -- which could explain why the docs are so sparse! The core of programming, to me, is having fun building good abstractions, and if Lisp isn't doing it for you, by all means, find (or invent!) something that does. Cheers!

I think what's happening currently in programming is that lisp-isms are filtering through to other languages. What used to be unique lisp-juice is available more generally, so the switch to lisp isn't perhaps as compelling.

Anyway. I'm off to write that metacircular evaluator in c# ;)

- Exactly.

Last year I started reading about Lisp etc, and wanted to play with it a little bit.

These are the exact same obstacles I faced.

If you ask about "IDEs" for Lisp, you're met with downright hostility for not embracing the glory of emacs with open arms - anything else is just unfathomable.

Well, what if I'd like to just, you know, start using the language, and not learn a whole new world of an editor first?

"Why does emacs + SLIME have to be the only viable option?"

That's not a question you can ask, of course, but the answer would be something like "roll your own" anytime you need anything.

(Oh, and Xach is a prick. He and Krysztov(or whatever) are practically high-fiving each other on IRC every time they alienate another new would-be Lisper)

As for the implementation research, I sort of came to the conclusion that SBCL was the way to go, but documentation (for anything?) was pretty scarce, and ASDF feels like an obstacle too.

Later on, I realized that Scheme is more elegant as a language, but it's more or less in the same situation as CL.

Compared to these two, something like Python feels much more accessible to me (being a novice at all), because I get the feeling that I can just start using things whenever I want to make that effort.

But that's just it - with Python, there's only the effort of learning the language as you use it.

You learn the standard library as you go, and suddenly you notice you've accomplished a lot of things and had fun while at it.

- At least that's how I imagine it will be when I get around to working on my own super-secret web-startupy project.

With Lisps, you need to make a lot of effort just to get to the beginning, and that's a big problem.

I'm unconvinced that installing lisp libraries is so very horrible; I've had far more trouble with python ones, myself.

Maybe it's because I don't normally use Eclipse.

If someone is interested in other alternatives, here's one: http://phil.nullable.eu/

- Phil's "ABLE" editor. I tried it too, but it didn't work too well on Windows (something about CLISP, iirc).

..open a file at the top window by pressing c-x-f and typing in hello-world.lisp <enter>

..yay, start coding..

c-c-c to eval forms c-c-k to eval/compile the entire file q to close down "popups" (exceptions .. etc.)

yawn .. what's the problem? .. are somewhat, "limited"?

Please don't call me a weed.

You point out that the more effort you expend, the better you get. The problem is the rate of reward. Getting a productive lisp environment is hard work. Competing implementations, sometimes-compatible libraries, very little documentation -- these are things that make lisp, holistically, hard to adopt. Every other production language makes it easier to get to the programming. I can't think of a worse-supported production language than lisp.

Also, the benefits you mention are either not that uncommon, or that valuable. The only thing lisp has as a unique feature is it's macro system, thanks to it's syntax. It's the only thing that can't be adapted straight into another language. Otherwise, the features you mentioned are basically available elsewhere; Visual Basic has closures. C++ has multiple inheritance. Python has multimethods. Ruby has class redefinitions.

The thing is, I wanted to believe. I stocked up my library with PCL, SICP, the little schemer, and On Lisp. I learned emacs. I stuggled with asdf. What I found, though, did not seem to live up to the promise of a hidden pearl.

Sorry :(

"You point out that the more effort you expend, the better you get. The problem is the rate of reward. Getting a productive lisp environment is hard work. Competing implementations, sometimes-compatible libraries, very little documentation -- these are things that make lisp, holistically, hard to adopt. Every other production language makes it easier to get to the programming. I can't think of a worse-supported production language than lisp."

I agree.

"Also, the benefits you mention are either not that uncommon, or that valuable. The only thing lisp has as a unique feature is it's macro system, thanks to it's syntax. It's the only thing that can't be adapted straight into another language. Otherwise, the features you mentioned are basically available elsewhere; Visual Basic has closures. C++ has multiple inheritance. Python has multimethods. Ruby has class redefinitions."

Well, it's uncommon and valuable to have all of them in a coherent language. Especially since they're pretty orthogonal features that complement eachother well. The whole is greater than the sum of its parts.

"The thing is, I wanted to believe. I stocked up my library with PCL, SICP, the little schemer, and On Lisp. I learned emacs. I stuggled with asdf. What I found, though, did not seem to live up to the promise of a hidden pearl."

I'm sorry that you've come to the conclusion lisp is not right for you. Maybe in a couple years you can try again and most implementation/library/documentation/etc issues will be resolved?

Nah, that's ok. ;) CL isn't really for me, now. As I said, common lisp, right now, is too much work. Who knows what Arc 2020 will be like? ;)

Exactly. Common Lisp is close to a superset of the features of other programming languages.

Even in cases where Common Lisp lacks a feature, if it's something that can be expressed by changing syntax, macros can get you pretty close. I remember a thread on comp.lang.lisp where several people had a go at adding pattern matching to Common Lisp, for example, and were able to get a pretty long way towards that goal in a short amount of time. Another example: seems like half of the Lisp books out there present an implementation of Prolog in Lisp.

"use Lingua::Perligata;

If you allow a language to mutate its own grammar within a lexical scope, how do you keep track of that cleanly? Perl 5 discovered one really bad way to do it, namely source filters, but even so we ended up with Perl dialects such as Perligata and Klingon. What would it be like if we actually did it right?

Doing it right involves treating the evolution of the language as a pragmatic scope, or as a set of pragmatic scopes. You have to be able to name your dialect, kind of like a URL, so there needs to be a universal root language, and ways of warping that universal root language into whatever dialect you like. This is actually near the heart of the vision for Perl 6. We don't see Perl 6 as a single language, but as the root for a family of related languages. As a family, there are shared cultural values that can be passed back and forth among sibling languages as well as to the descendants."

I'd say Perl is getting comparable macro-level functionality. Perl also has one likable feature that is less present in lisp; different idioms look different. When I hear Paul Graham talk of (a i) being an array index or a function call, I start to think about concepts like Hungarian notation to keep my code more readable.

A uniform syntax structure isn't required for macros.

sounds like XML

> "A uniform syntax structure isn't required for macros."

it makes it a lot easier!

i don't understand what you mean by 'pragmatic scopes.' does perl 6 have macro functionality?

I picked up Python in a day. I'm quite proficient at it a year later. A simple syntax + a great standard library is the best way to get things done, if not the best way to learn to write software.

nobody, proficient, lisp. I, deeply conversant, lambda calculus. You're comparing apples and oranges.

I learned lisp (pretty deeply, I'd qualify myself as intermediate then and now) in what... 5 or 6 months? And I was alone. And I didn't know emacs. With some coaching and pointers I'm pretty sure I could have made it in 4. Aren't you happy?

For what it's worth, I think learning Lisp is useful for getting (re)acquainted with lambda calculus if nothing else.

Lists confine one to rigid hierarchies, which have to be compensated for with dirty (but sexy) hacks like in-language macros. Meanwhile the index of hashtables is arbitrary, which allows you to do naturally the things you have to patch in Lisp.

Though hashtables are more powerful and easier to grok, for one reason or another nearly all (popular) hashtable-oriented languages are total crap (C++, Java, C#), but in some regard a step up from deformed languages like C (in all seriousness, how does anybody get by without first-class functions and hashtables?). My tentacles have only found two decent hashtable-oriented languages: JavaScript >~1.5 and Io.

Anyway, why is Lisp unpopular? Because it's harder for most programmers to think in lists than hashtables. Then, why does the lang have a cult following? Because it's well crafted and consistent, which seems to cause some people to overlook its shortcomings, even to the extent of seeing design flaws as features.

But as far as I'm concerned the Language War is over anyway. JavaScript won.

I'm not sure whether or not you'd find it "decent", but you might consider adding Lua to your list. It's small, relatively fast, and uses hashtables as its composite data structure. It also has some other neat stuff, like tail-calls and coroutines.

The basic structure is the hashtable. You can use any first-class value as a key (number, string, function, another table) and similarly any first-class value as a value. Creating a table is done with the table constructor "{ }" ('> ' is the repl prompt):

  > a = { }

  > a = { 'red', 'green', 'blue' }

  > a = { [1] = 'red', [2] = 'green', [3] = 'blue' }

  faves = { bob = 'pizza', george = 'cake', mary = 'pie' }

We can use numeric indices along with string ones in the same table:

  mixed = { 'a', 'b', 'c'; state = 'NC', city = 'Charlotte', county = 'Mecklenburg' }

  > print(a[1])
  red
  > print(faves['bob'])
  pizza

  > print(faves.mary)
  pie

  > print(faves.james)
  nil

  faves.george = nil -- the cake is a lie! And the key 'george' is removed.

In addition to the tables, you get first-class functions:

  > function a() print 'hello' end
  > a()
  hello

  > a = function() print 'hello' end
  > a()
  hello

  > function a.foo() print 'world' end
  > print(a["foo"])
  function: 0x807f2e8
  > a.foo()
  world

  > a = { color = blue }
  > function a:fave_color()
  >   print('My favorite color is ' .. self.color) -- .. is concatenation
  > end

  > a:fave_color()
  My favorite color is blue

What brings even more power to the table is that we can define custom behavior on each one. We can set a 'metatable' which defines how the table responds to subscripting, the various mathematical operators, and being called in the functional position. In a quick script I worked on I implemented a prototype-based class system in under 20 lines of code, all with the power of metatables. Lua tables are very powerful and though similar to the ones in Javascript are even cleaner and more pleasant to work with.

Anyway, it's all nifty stuff. Other features you get: fast incremental garbage collection, first-class functions, asymmetric coroutines (isomorphic to one-shot continuations), lexical scoping, closures. Lua is also one of the fastest interpreted languages and has an excellent API for binding to C if you want to do something performance sensitive, or want to use a library written in C. The language itself is written in 100% pure ANSI C. It runs on everything from embedded microprocessors to mainframes, in lego robots and on space satellites. And the community is very friendly.

If you want macros, there are several community-run variants of the language with macro facilities... I haven't gotten very deep into those though.

Good documentation for getting started:

  http://www.lua.org             -- official lua home page
  http://www.lua-users.org       -- lua community wiki
  http://www.lua.org/docs.html   -- lua documentation
  http://www.lua.org/manual/5.1/ -- lua language reference
  http://www.tecgraf.puc-rio.br/~lhf/ftp/doc/hopl.pdf -- a fascinating article on the history and features of lua

Really I don't get the difference you state between list/hash programming languages, I've work a little with EcmaScript and I don't know anything about Io, but I'll put it in my to-do list. Can you, please, extend it? What's the main difference between them?

Because you say that the primary abstraction of a "list" programming language is a tree, which I think not. Since a tree is a directed graph without cycles, you wouldn't have loops. So, at least is a directed graph. But then, which property has the "hash" language graph that the "list" language graph has not? It's an undirected graph? It think it is not possible, because if it's undirected how do make your program to go "forward" and not "backward"?

I hope I'm not too obfuscated and my questions make some sense.

I'll just go through my probably plebeian understanding. Arrays are to lists as hashtables are to objects. An array, in my mind, is a list that only contains one type and is indexed with enumerated integers. On the other hand a list can contain any type, but is also indexed with enumerated integers.

In JavaScript:

  array = [1, 2, 3]
  list = ["one", [[array], 3]]
  array[0] == 1
  list[20] = 23 // list indices aren't necessarily a linear enumeration

  hashtable = {
    "today":20080403,
    future:function(x) { return this.today + x }
  }

  hashtable["today"] == hashtable.today
  hashtable.method(23)

RE: trees and graphs -- I was getting at the relationships between nodes, not the actual computations, but I'm not comfortable enough with the terminology to explain exactly what I meant.

It's a fun thread :D

I'm basically just asserting that objects (should) == hashtables. This is quite literal in JavaScript. Other languages bend the metaphor in different directions, and obscure it to that no one even knows what "object-oriented" really means beyond particular idiosyncratic syntax in this language or that.

PG of course talked about this before, in Why Arc Isn't Especially Object-Oriented:

> I've done a lot of things (e.g. making hash tables full of closures) that would have required object-oriented techniques to do in wimpier languages ...

I'd argue that he was employing genuine object-oriented techniques, but just didn't have classical syntax and didn't consider what he had an "object." Other languages make a point about it, and use special syntax, which fogs the whole thing. Perhaps some people in "OO" mindsets have the kind of naivete that C-only hackers I've met have about first-class functions.

Actually, I just realized the whole reason C++, Java, C#, and co. have "methods" in the first place is just compensation for not having first-class functions you can stick in a hashtable.

I think in the "hash" languages, you usually consider the method to "belong" to an object in some way.

With multi-method dispatch, the relationship between objects and methods is more fluid. So I think there is a difference here, too.

It's not that "list" languages only use tree structures for everything, just that trees are the more "natural" choice in those languages.

In those languages, everything is an object, that is everything construct you define, whether data or code stared as an object, a set a behaviors and properties. (Assuming it is fully OOP, unlike Java.)

In common lisp, everything is an object as well, with it's own set of properties and behaviors. Cons cells are basic, for example, but you can still add properties to them and define methods for them. Encapsulation is not enforced, but that's for what closures and packages are.

Lists are dominant in lisp for another reason: the language itself is represented in them, not just the data and the methods, but the pre-compiled language, so that you can use lisps meta-programming facilities to generate lisp code itself. In most other programming languages, the code is represented to the compiler as text and to use such meta-programming facilities would require string parsing. Macros aren't a 'hack' but an actual paradigm shift. Attempts to do the same thing in other languages have largely been very clumsy, witness C++ macros. (Template Haskell apparently has managed to do it properly though.) Nearly Lisp's entire syntax is for defining the structure of the code; everything else is done with operators, functions, and macros.

If languages like Java or Ruby were represented in hashmaps the same way that Lisp is represented by lists, they would probably be incomprehensible. if you were to attempt to use a structure to represent the language, it would probably end up being something of a tree format. Code, is naturally hierarchical, even class definitions, and if I were to do the same kind of thing that one does with Lisp in C++ or Java, I would end up using lists. So I don't think that this comparison is really correct.

(BTW, I love C. It's not deformed, it was designed like that to A- make it easy to implement and B- give the programmer as low a level access as he needed. You are meant to define your own data structures and implement them in an efficient way using algorithms that make sense for the usage. You are not confined to a preset, possibly inefficient implementation. This is a level of control not available in a lot of other languages which is why C is so commonly used to write interpreters and compilers for other programming languages. Those highly efficient Python hashtables are implemented in C (and maybe a bit of assembler))

Huh? Let's review.

>>>If that's true, it follows list-oriented languages are inherently inferior to their hashtable-oriented brethren.

Javascript code is semi-formalized strings or ASTs.

The careful reader has noticed that lists that represent code are ASTs with context-dependent field names. Since the nodes provide the context, said dependence isn't a big deal.

I don't know how many javascript programs manipulate their ASTs. (Lisp programs with macros are manipulating their ASTs.) The vast majority of javascript hash table operations are on data. (Yes, lisp code can be data, but not all lisp data is code.) In that, they're no different than any other language that has decent hash tables, such as lisp.

What design flaws of Lisp are seen as features?

Sure, my experience is only anecdotal and not at all large. But always I've tried to introduce someone to Lisp (I'm talking of about 5 people, none of them have converted...) they seem incapable to see further than the parenthesis and the prefix notation, but specially the parentheses.

I tried to explain them the benefits of being such a regular language, how the source code is a list which makes it really easy to do meta-programming, the power of macros, the fact that using a good text editor (Emacs + SLIME) with an excellent indentation you really don't have to care about parentheses: you follow the indentation not count parentheses!. Not to speak about the prefix notation, which they seem allergic to. But all have been in vain.

Lisp is so esthetically different from the other Algol-derived languages that it, simply, does not fit their brains, which I think is really a pity. Four of them are Perl programmers, being Perl their blub. They belive and act like there is no language better than Perl, they can do everything with perl. Whey I explain macros and metaprogramming they reply that Perl can do that with eval and strings. I've tried that and is, at least, sadomasochistic!

So, after this rant (sorry), I've to restate: the main problem people have with Lisp are parenthesis and prefix notation.

PS: Some of them also worship Microsoft and argue that Windows XP is the best operating system out there far better than Mac OS X (which none of them have tried for more than 5 minutes in a row), and that Linux is only good for severs. Some times I think that they suffer from severe brain damage, but apart from that they're pretty clever. So... it's an enigma.

Unfortunately for lisp, that's not the case. Whitespace is different than lines. There's a reason most people can't look past the parens - they're hard to look past.

Can I write really ugly and hard to visually parse Lisp? Yes. Bad code can be written in any language.

No one looks at C code the first time and thinks it's clear to understand.

Specifically, reading a newspaper that has parens instead of whitespace would be harder.

(Specifically,(reading(a(newspaper(that(has(parens(instead(of(whitespace (would(be(harder.)))))))))))))

In terms of actually being a problem, I'd say it depends on the code, for sure. I could create that same example in C, but end it with }}}}}}}}}}}}}}. However, in my admittedly limited experience with lisp, it seems more likely to be highly nested, tempting people to put more levels on a single line.

I also agree with you that C shouldn't be held up as the paragon of readability.

Python is pretty darn good though.

Example from taken from http://www.gigamonkeys.com/book/

  (defun test-+ ()
    (let ((*test-name* 'test-+)
          (another-var 'foo))
      (check 
        (= (+ 1 2) 3) 
        (= (+ 1 2 3) 5) 
        (= (+ -1 -3) -4))))

OTOH lisp encourages deep nesting, which makes it harder to be readable. Here's a rougly equivalent program, if I read the lisp correctly.

  void test-+()
  {
     *test-name* = test-+;
     another-var = foo;
     assert( 1+2 == 3 );
     assert( 1+2+3 == 5 );
     assert( -1 + -3 == -4 );
  }

And flat programs are easier to make legible, because you're less tempted to put multiple things on a single line to avoid going another level deeper in your indenation.

If you look at languages in wide use, they're all procedural - everything from ASM and basic to ruby and python. Similarly, all the pseudocode I've seen has been procedural. If you ask a lay person to come up with a way of making 20 pb&j sandwiches, you're going to get a list of steps for each sandwich.

I'd argue that functional thinking is not natural, it's learned. Prefix notation is not as natural as infix notation. It's not just Algol, it's likely a reflection of natural language, and the forces that shaped natural language.

My prediction is that lisp is going to become more popular in one of two scenarios. The first is that parallel processing becomes more necessary, and because functional programming is easier to parallelize, people will learn it.

The second is that lisp is redesigned with usability in mind. The fact that it's not bad as long as you use emacs+SLIME is not a ringing endorsement. One of the advantages of, say, ruby is that you don't need an advanced editor to be productive. The design goal was to make a language that's nice to use, so it is nice to use.

If lisp were redesigned with usability in mind, I could see it being more popular. However, the people who like and use lisp enough to try to redesign it likely aren't bothered too much by the syntax, so it's not something they try to fix.

You really don't need to rely on Emacs+Slime, an editor with a good support for indentation will suffice. IIRC, pg codes in Vi (or Vim). The idea is that, like in C/C++, good indentation is enough to determine the nesting so you don't need to count parentheses or curly brackets.

The last difference, infix vs prefix, I can see that we are more used to infix notation. But we can't forget that the Algol derived languages use a mix of them (operators and functions are different!) while Lisp always use prefix notation, and incidentally put the parentheses surrounding all the expression (and being the first element the function name).

If the indentation is sufficient, why not just remove the parens completely and rely on indentation?

Lisp does have an advantage in being regular, which might not be worth giving up, and Algol descendants are kind of a mishmash, to be fair. To be honest, I'm not enough of a lisp user to be able to know what a good solution is for this one. Maybe changing some function names would be useful.. for example 'add' is better than '+', which I read as 'plus'. Repurposing infix symbols as prefix symbols seems problematic.

If I were redesigning the language, one of the more important things I'd change is the keywords and builtins. For example, head and tail seem a lot better to me than car and cdr. A lot of lisp's keywords and functions come from math, and I'd probably change them to reflect mainstream algorithm design instead.

http://en.wikipedia.org/wiki/Logo_%28programming_language%29

And for the names used for the keywords and builtins, I agree with you.

About the +/add, I really don't see too much problem.

SQL?

For example, a few months ago I was working on a blog post about how nice it would be if Perl had macros. I had a perfect example picked out, but I decieded to try an implement it without macros. It ended up working perfectly.

The docs explain the original problem, and my solution: http://search.cpan.org/~jrockway/Context-Preserve-0.01/lib/C...

Also, if you read your friendly local CLOS implementation, you will notice a lot of macros. Moose is a CLOS-alike for Perl, and it obviously didn't need macros. (Actually, I would like to port Moose to Lisp, 'cause it's much more sugary than CLOS, and it has traits. Mixins FTL.)

Anyway, Perl is nearly as good as lisp, so be careful when choosing your examples. (Incidentally, I am currently working on a lisp variant that compiles to Perl, because I like lisp syntax better than Perl syntax, but I like Perl's CPAN and regex engine.)

Actually, for the last 3 years I've been using it at work, including first-class functions (and Catalyst ;)), and I've it enjoyed quite a lot. I've some minor issues with it's syntax (for my taste too much sigils, but I understand why they are there).

What I meant to be masochistic is trying to use eval with strings to emulate macros, and even trying to defend it.

On the other side, the CPAN is amazing and I find difficult to live without it!

BTW, I hope you'll publish here the Lisp to Perl compiler! Lisp + CPAN sound really awesome and productive!

http://git.jrock.us/?p=Perlisp.git;a=summary

It's pretty useless right now though :) I started rewriting the reader, but didn't like any of the parsing modules on CPAN... so it's sitting at that stage right now. When I get some tuits, I'll probably just port over SBCL's reader, or perhaps emacs'.

Thank you very much.

1. They fear the unknown; 2. They fear fanatics.

Mac OS is The Unknown to them + Mac users tend to be pretty rabid fanatics = Let's bash them!

I know this doesn't describe everyone who uses Lisp, but these fanatics are very vocal and the reason I stopped reading comp.lang.lisp and decided to try other languages. I am not against being excited about a language and trying to convince other people to use it. I love Haskell and tell people about how wonderful it is all the time, but I understand that they have reasons for using the tools they do. I am more interested in the theory of a language than in how good it is at solving some practical problem, but some people don't care at all about theory and just want to get things done. I don't understand why, but I accept it. I think that the most important thing the Lisp community can do to be more popular is to be friendly to newcomers and those who disagree about Lisp's superiority. There are some very helpful and friendly people, they just need to be more vocal and drown out those who are not.

People simply stopped paying attention because of the failed promises of AI - and in a nod to the parent, some of the talent base probably shifted to other technologies, and Lisp was left without an ecosystem. It's starting to build up again.

Beautiful case-in-point: I went to a developer's conference yesterday put on by one of the big software vendors that my company buys from. I was carrying a copy of PG's excellent On Lisp to read during dead times. Having that book out in the open started several conversations with people I wouldn't have met, incidentally. Each of those conversations went something like this:

He: Lisp! Who uses that anymore? Didn't that die out a long time ago?

Me: Well, it was big during the 80's because of AI, but it's in the middle of a comeback that started sometime around 2000.

He: really!? I thought it was a dead language. Who uses it?

Me: There are a few companies out there that use it. There are even commercial offerings of Lisp tools. There's Franz, Inc. which sells AllegroCL, for example. There are two or three others I can think of off the top of my head. There are some open source projects that are very active, too.

He: but it's just a scripting language, right?

Me: No, it's really a general-purpose programming language that can be used for any number of things. It's currently being used for writing web applications, among other things, but it could be used to write any kind of software, really. Whether or not it is interpreted or compiled is dependent on the implementation and how you want to use it.

I'm confident, given time, that all these concerns will be addressed and the rate of adoption will continue to increase. That's why I think now is a good time to get involved if you aren't already: be ahead of the curve.

I disagree with your conclusion that Lisp is somehow going to take off though. After being around for what seems like forever, its still a (and I will say it) categorically unpopular language. If I were a betting man, I'd bet on JavaScript.

Basically, there are two Lisp eras - the big AI era and the new pg/Open Source era. The recent renewal of interest in programming languages, as well as publicity by writers like pg, Yegge, Raganwald, etc, have led more people to become interested in Lisp, but it's on an informal, person-by-person basis. Some of these people (like me) have chosen to stick with it and new Lisp communities have formed by gradual accretion. But this process has started later and been more distributed than other languages (PHP, Python, Ruby, Perl).

In researching things and trying to figure out Lisp setup problems, I've noticed that there are fewer technical "How do I ..." questions now than a couple years ago but more usage questions. The technical problems are being solved and now more stuff "just works" - not completely but much more than in the recent past. I've noticed this for CLisp, SBCL, ASDF, and other tools. There's more documentation, more tutorials, etc. Now there are a couple young web frameworks (UCW, Weblocks), generally accepted libraries for common tasks, etc. Lisp is growing in popularity and is maybe 1-2 years away from being "tasteable" - ie you can try it out without a big hassle and a research project. I think that will be an inflection point, after which growth will come much faster.

It's like asking, "Why are Arctic foxes (Alopex lagopus) not very popular on the African savanna?" Because they didn't evolve in response to the demands of that environment. The bat-eared fox, Otocyon megalotis, did.

The differences between Alopex and Otocyon aren't so massive that they seem like totally different kinds of animals. But it would be tricky to predict exactly what they should be, and develop the right kind of fox in a lab. Some adaptations are obvious (thick fur), other less so (specialized circulatory system). I suspect the same is true for programming languages. It's tricky to determine exactly what makes Scheme better suited to the classroom than to developing general applications. Perhaps the best way to solve the problem is the way PG is doing it with Arc, which is to take a very young, malleable dialect of Lisp and plunk it down in the environment to which you want it to become well adapted, and then continue development in response to the demands you encounter.

    (mapcar #'why-is-lisp-unpopular? 
            (list system-administration
                  enterprise-applications
                  web-applications
                  embedded-software
                  mobile-phone-apps
                  blogging-software
                  compiler-writing
                  configuration-files))

After reading and doing the exercises from the Touretzky book and The Little Schemer, plus some articles here and there on the 'Net my experience was:

1) Lisps have their annoyances like all languages. Lots of little non-intuitive keywords to memorize, the dotted pairs, quite the baroque computation model (variables having sub-variables, etc.), the FOR syntax, DO, etc. Not a deal breaker, but annoying. (Touretzky seems to fixate excessively on some of these weird traits of Lisp.)

2) If your Math chops are rusty, you are going to have to work twice as hard. Mine are, and I had to. If you are fresh from taking your Computability Theory course, then recursion and infix are going to be second nature. If you haven't been doing Math for a while, it's going to be hard. Another reason to learn Lisp while in college.

3) You can learn the syntax in 2 hours, yes, but to really know Lisp you need to really understand recursion, continuations, thunks, macros, multiple dispatch, all the fine points of object orientation (to understand the CLOS) and who knows what else. Basically, you need to have a deep understanding of, well, all of CS? This is not something that can be learned on your spare time unless you have lots of it.

In my case, I am starting The Seasoned Schemer now, and taking it slowly, under no illusion that I am going to learn Lisp any time soon. I may well be that I am one of those "stupid programmers" that are unsuited for Lisp. Oh well...

Also, it would be easy to compensate any error by controlling the phase of the fed signals.

See http://en.wikipedia.org/wiki/Ballistic_Missile_Early_Warning...

I was thinking of that very link when I wrote this. Dunno know why I didn't look more carefully for it.

The number L = lambda/10 is actually part of the standard definition for a microwave system. Any circuit that has a physical dimension greater than L is too "big" to use standard circuit analysis, because the voltage and current values you measure at different points will be different due to propagation delays, even when those points are connected by a wire with no reactance and negligible resistance at lower frequencies.

The world's biggest microwave system (biggest machine period, actually) is the US power grid, even though it oscillates at a measley 60 Hz. Lambda is about (.6c / 60 Hz) = 1800 miles.

This is proof that, electrically as in so many other things, the Midwest is 180 degrees out of phase with the rest of the country.

Anthropomorphically, you could say that Perl retreated to its base of support among sysadmins, (Common) Lisp took shelter in big-think research organizations, and Java reinvented itself as New Cobol.

But that doesn't really satisfy either, because it doesn't tell us why nobody tried to similarly re-purpose Lisp. That's especially odd, in fact, because now Everybody Knows that a lisp core is really simple, and Lisp was still on the radar in academia, where people can be rewarded for re-implementing old ideas.

The problem with the AI winter argument is that it's half an answer that satisfies you enough to stop asking questions. If you invoke the collapse of commercial AI to explain Lisp's current state of disfavor, you then have to explain why the collapse of the dotcom bubble didn't have a similar effect on Perl or Java.

Anthropomorphically, you could say that Perl retreated to its base of support among sysadmins, (Common) Lisp took shelter in big-think research organizations, and Java reinvented itself as New Cobol.

But that doesn't really satisfy either, because it doesn't tell us why nobody tried to similarly re-purpose Lisp when DARPA et al lost funding. That's especially odd since Everybody Knows that a lisp core is really simple, and Lisp was still on the radar in academia, where people (called grad students) can be rewarded for re-implementing old ideas.

I think there is a pretty good explanation (with evidence, even!), but it's not a satisfying one in this community: the end of the Cold War caused a major reallocation of scarce resources away from expensive one-off projects for the government, and towards mass-market stuff for consumers and businesses (see below), and Lisp suffered from some initial disadvantages in the new environment:

* Limited availability on PC platforms

* Windows Apps were sexier. The internet was sexier.

* Glut of new CS students while the bubble was inflating were mostly taught in Java. (Many of them are sensitive about arguments that lend further support to the idea that their education has turned out to be less valuable than they thought at the time. That may just be my bias showing, though. I think that Math should be relabeled CS, and that "programming" is for autodidacts and trade-schoolers).

* CS is a new field, and industry tends to dumb down research in new fields for a generation or so after disruptive technical innovations. That way, they get enough of the low hanging fruit that it further disruptive innovation is unlikely and their capital is safer. See below, particularly the chapter on the relationship between GE and MIT before WWII.

Social arguments are hard to make here though, because so many people are so hooked on the "weird and scary and only for elitist pricks" narrative. That's kind of ironic, because high level corporate executives, if they heard the current public stand on lisp -- "it may be really productive, but it's too foreign for most people and I don't feel like learning it" -- could probably force adoption from above, especially now that cost-cutting is the order of the day.

http://www.amazon.com/Leonardo-Internet-Technology-Culture-R...

I'm not saying AI Winter is the definitive explanation, but AI collapsed so thoroughly that there were barely any AI companies still around. Even at the worst of the dotcom collapse, use of the web was still growing among the general populace, and the principle leaders like Yahoo, Amazon, and Ebay are still around.

Your argument about Lisp losing interest because of the PC boom is really interesting, and I think it has some merit. Still, doesn't it beg the question of why people didn't just port Lisp to the PC and keep going? That's what happened with Unix -> Linux.

Other language syntax (C or Erlang, e.g.) is arbitrary and tedious by comparison.

I think it more reasonable to compare the popularity of problem domains (e.g microwave engineering vs robotics) or techniques (e.g. S-Parameter Simulation and Lisp).

http://commons.wikimedia.org/wiki/File:PAVE_PAWS_Radar_Clear...