I know at first it looks confusing, but the lisp syntax is pretty much the best. I can teach it to someone in 1 minute:
(<function or macro name> arg1 arg2 ...)
That's it. This is 100% of the syntax.
If dealing with a function, the evaluation will always be arguments from left to right are evaluated first, and then the function is called with the result of evaluating the args. This is 95% of the cases.
If dealing with a macro, go read the documentation for it, because every macro is a DSL, and evaluation can happen in different ways. This is the 5% case.
All the complexity of learning a lisp comes from knowing more functions, what they do and how to use them, and learning each macro's DSL.
Can this syntax be improved on, I'm not so sure. Clojure added some special syntax for some more literals, such as [] for vectors, {} for map, etc. And I do think in one way they help, but in others they hurt.
Maybe adding something that distinguishes a macro from a function, but once you know the names of them, the name is enough to know.
Maybe make it so there is a more visible demarcation between the function/macro name and the arguments. Not sure how though.
All that to say, if you give the ayntax a good chance, like 2 to 3 months of 40h a week for your brain to more clearly see where the functions start and end, I'm sure you'll like it.
That really depends on the current package at the time the form is read, and then how `list` is defined within that package. Strictly speaking, you don't even know which 'list' symbol is referenced without that context. (And in that context, list could be bound to a macro, or it could be unbound entirely.)
1) A list of 1.0 and the pound char.
2) I'm guessing a complex number.
3) A list of 1 and an explicitly evaluated 2.
4) Hum, would it just be a symbol?
5) I am surprised. Maybe just another symbol?
6) More complex numbers.
7) Another complex number?
8) A string
9) An empty vector.
Without the complex numbers, which I admit I've never used, it's not that bad. They're just shortend, you can use list, vector, quote, unquote, unquote-splicing and quasiquote functions instead.
I think there's a function for char too, name->char I think.
The ones that are weird are 1@1 and 10#.
Anyways, there's a bit more syntax it's true, but all of that is literals. I'm not sure that counts as syntax, I guess it does. So ya, there's a little more than I said.
Maybe it's easier to say I can explain the semantics in one minute and the syntax in two minutes?
5) # at the end of a numbers are to express inexactness, and are equivalent to a zero digit, so that express is 100.
6) The first one in that list (#i+1) is actually just 1.0, no imaginary part. The #i notation is for inexact numbers. The other two are the complex numbers i and 1+i.
In "'(1. . #\#)" the "1." is a number. You are correct that you don't need digits after the decimal point. And #\# is just the pound character. `(1 ,2) is a proper list, and `(1 ,@2) is an improper list. You'll want to check out unquote-splicing:
Scheme has a really nice way of representing numbers. The socalled numerical tower can be hard to implement for the language implementors, but it is really nice to have as an user.
That something is simple and easy to explain doesn't necessarily mean that it's the best. Simplicity can hurt readability, for example, by utilizing too few distinctive visual patterns - fewer than human brain can easily distinguish while scanning code - and then having to reuse those same patterns for different things, making it necessary to pay more attention to finer details of the code to understand what it does.
To give an analogy, imagine that we replaced all our punctuation with S-expressions. It would be simpler in conceptual sense, but do you think you could read such text faster than the one that distinguishes between commas, periods, question marks etc?
I disagree. IMO it's just about what you've learned before. Note that parsing natural language isn't obvious until you learn to read - a task on which you spend years of your life.
I'd argue Lisp is no less readable than C for people who haven't seen a programming language before. It's just more likely your first language will be C-like rather than Lisp-like, and this colors your expectations.
Funny things, people in our industry deal with Lisp-like syntax all the time. Consider HTML or XML. If you grok how they work, you'll also understand S-expressions. Except S-exps have less clutter.
XML is not like S-expr in the sense that I'm referring to - quite the opposite, in fact. In XML, special syntax - <> - is used to visually distinguish what in S-expr is a head item from the rest of them. Furthermore, there's a big visual difference between attributes and child elements, as well.
Although once you get to a higher level of structure, XML shows similar deficiencies to S-exprs. For example, consider XSLT - that's a programming language that uses XML as syntax for both its language constructs, and its data. There are good things and bad things about XSLT, but I have never heard anyone claim that readability is its strong point. In contrast, XQuery - which uses its own Algol-style syntax for language primitives, and only uses XML-like syntax for inline data, is much more readable.
That's the wrong comparison. S-expression don't compete with natural language any more than C does. S-expressions compete with JSon and XML. And IMHO they win hands-down.
I didn't claim that S-exprs compete with natural languages. I'm saying that C is to S-exprs is like natural language is to an artificial language where all punctuation is replaced by a single symbol + words.
> Simplicity can hurt readability, for example, by utilizing too few distinctive visual patterns - fewer than human brain can easily distinguish while scanning code
There's a bit of a tradeoff. I tend to agree with you that there is at least an initial readability hit when looking at most Lisp code. However, that's something that can be managed with a bit of experience reading/writing the syntax and some care taken with the way you write the code.
The benefit of the Lisp syntax comes in that its regularity makes it easier to make tools that manipulate it in higher order fashions. (Paredit mode and macros being two examples that immediately come to mind.) Not only does the syntax make it easier to write those tools, the syntax makes it easier for developers in the moment to understand what those tools actually mean and how they operate on code.
The trouble with all this for Lisp is that the costs are immediately evident, but the benefits all accrue on a longer time scale. It's a style of language that rewards patience and long term investment in a world that does not necessarily favor either.
IMO, the ease of manipulating Lisp code comes from that code being internally represented as a tree. But that doesn't mean that the syntactic representation has to be so closely mapped to that underlying tree. Syntactic sugar is good! In fact, it's so good that even Lisp has some - you still write '(1 2), even though your macro will see (quote 1 2).
My point is, essentially, that raw S-exprs, or even S-exprs with the usual Lisp extensions, are too "bitter" - there's not enough syntactic sugar there to have them go down smoothly.
For a very extreme opposite example, consider R. Under the hood, any R program is represented as S-expr style tree (the corresponding data type is "call" - but it's basically just a linked list of arbitrary objects, including other call objects, and its interpretation is exactly like a Lisp S-expr - first item is what we're invoking, the rest are arguments).
Then on top of that they have created a C-style syntax. But that syntax is pure syntactic sugar - there's an exact one-way mapping from it to the actual call tree, and there's a canonical back-mapping as well. And you can skip the syntax and write everything in pure calls if you really wanted to.
Now, I'm not claiming that this is the sweet spot. But I'm pretty sure that S-exprs aren't it, either. It feels like there can be some middle ground, where the original tree structure is still fairly recognizable, but also with enough sugar that common patterns (e.g. variable bindings, function definitions, conditionals etc) are easy to parse visually.
> '(1 2), even though your macro will see (quote 1 2).
'(1 2) is (quote (1 2)), not (quote 1 2)
> that common patterns (e.g. variable bindings, function definitions, conditionals etc) are easy to parse visually.
Actually those Lisp patterns are not difficult to parse for humans. It's just that it looks a bit unfamiliar for non-Lisp users. The human visual system is capable to deal with that quite easily, it just might take learning. Humans can for example read book pages very fast even though there are very little visual clues in long sequences of text. It's just that humans have read a lot of those books...
The visual pattern for variable binding in Lisp is:
LET var value
var value
var value
expression
expression
expression
That pattern is easy to spot visually. In actual Lisp code it is:
The top-left item defines the structure of the rest. That item is easy to find. Here it is LET.
All you need to know is what item has what pattern. There are a few basic patterns and combination of those. Once you learn about 10 - 20 of those patterns and understand that something like `WITH-FOO` will give you a clue what pattern to expect, then it is fairly easy to read Lisp.
Typical patterns:
function call
sequence
binding list
with scope
tree if function calls
property list
association list
conditional expression
At this point we're basically both making unsubstantiated assertions.
Anecdotally, I know that I was able to learn to read C, Pascal, BASIC etc fast much easier than I was able to learn to read Scheme fast. I've asked this question to other people, and it seems to be a common thing. From this I conclude that visually parsing Lisp is more taxing. Yes, you may train yourself to do so fast - but you end up spending more time and effort on that.
Again, this is anecdotal. A proper way to resolve this question would be to have a controlled study where a bunch of people not previously exposed to any language is given a crash course in, say, S-exprs and JSON, and then asked various data retrieval tasks on input of increasing complexity.
Since you learned C, Pascal, Basic, ... you already had expectations how a program looks like, how it works and how to develop code. People also hear that Lisp is difficult to learn and exotic, this creates another negative bias. The next problem when learning Lisp is that the whole combination of syntax, semantics and pragmatics is new. Thus people might be able to read the code, but they have difficulties understanding the meaning, given that prior exposure to other programming language might not help here. They also may not be familiar with he particular interactive style of development and not see the advantages of working with code as data.
The typical old saying is that everybody can learn the basics of Lisp in a day, unless one already knows FORTRAN, then it takes three day.
The best way to get over these hurdles is to read and write a substantial amount of code. Luckily Lisp has some great books to learn from.
A page of C that conforms to common coding conventions does next to nothing. I can read it fast: "oh this is yet another reinvention of getting a line of arbitrary length from a FILE stream". Look; there is the realloc, the getc, the doubling in size of the buffer. There, I read it in 3 seconds! Three weeks later, I spot the failure to null-terminate the line in a corner case ...
> IMO, the ease of manipulating Lisp code comes from that code being internally represented as a tree.
I don't think this is sufficient to explain the ease of manipulating Lisp code, After all, most languages are processed through an internal tree representation. (This fact is inherent to the grammars used to specify and parse those languages.) The only real exceptions I can come up with are concatenative languages like Forth, PostScript, and RPL and in these cases the syntax is truly primordial. (Which also carries its own advantages.)
Where Lisp's s-expressions both shine and suffer is that they bring the structure of the initial syntax tree right up to the textual surface of the language. With some exceptions (like the quote syntax you mention), there are syntactic textual delimiters at either end of a Lisp ASP node, which is not even usually the case in an infix language.
Just to illustrate, take these two (hopefully!) equivalent expressions... one in infix and the other in an s-expression.
(> (+ x (* 2 z)) (- y 3 u))
x + 2 * z > y - 3 - u
Given those two, and an editor command that selects a sub-expression or broadens a selection to include the enclosing sub-expression, where is it more obvious where the sub-expressions begin and end? The parenthesis in Lisp clearly delimit the nodes.... which makes it easier to identify the tree structure at a glance.
Of course, it also makes it less-usual for human readers, who are well trained in infix. (Although I should point out that reading the infix requires being aware of hidden precedence and associtivity rules for the operators in use.... which is often managed by conventionally requiring extra parens, etc.)
This is the trade-off I was referring to.
From a couple posts down:
> At this point we're basically both making unsubstantiated assertions.
Anecdotally, I know that I was able to learn to read C, Pascal, BASIC etc... A proper way to resolve this question would be to have a controlled study where a bunch of people not previously exposed to any language is given a crash course in, say, S-exprs and JSON, and then asked various data retrieval tasks on input of increasing complexity.
The way this is written almost implies that there is a single continuum on which these syntaxes can be ranked into better and worse. I really think this is something that's much more subject to opinion and personal point of view than that. As positive as you've been about infix syntax, I can state that, for me, the prefix syntax tends to feel more right.
In other words, I don't think this is as much either/or as it is both approaches have strengths and weaknesses.
Another thing is the ease and regularity of formatting into multiple lines:
(> (+ x (* 2 z)) (- y 3 u))
(> (+ x (* 2 z))
(- y 3 u))
(> (+ x
(* 2 z))
(- y
3 u))
Algol like languages are expected to use indentation for major structure, and expressions to be small, all in one line. Breaking infix into multiple lines is never satisfactory.
S-expressions are more of a grammar, you'd need to change the English grammar to something more like it, specifically make the action first.
Send money to John's account -> (send-to (account-of "John") money)
Find all empty boxes in the warehouse -> (find-in warehouse empty boxes)
If Bob is home tomorrow, tell him John was looking for him. -> (if (is-home bob tomorrow) (tell bob (fn [him] (was-looking-for him bob))))
Now you'd have to define a lot of those arguments, and the implementation of these functions would be complex, to interpret all the relational rules of natural English language. But I feel this would be s-expressions applied to english. Things would be grouped in parenthesis, with the action first, and the set of things involved in the action following. Conditions would come first, different tense would be different functions, subject and complement would follow as arguments, and actions can involve other actions by passing them as arguments.
Now on the subject of readability, I think you're right. Spacing, font, whitespace all helps readability, and maybe the syntax lacks a little in visual separation and organisation, though you can solve most of it with indentation. But I think we must also consider comprehension. Code is more than about reading the text, it's all about understanding the underlying behavior. I think LISP syntax is good for that, in fact, it's much closer to English in that sense. It can have very powerful embedded semantics. But just like English, it requires you learn the definition and rules of a lot of words, or in LISP, functions and macros. That's what people mean when they say LISP syntax is both simple yet powerful.
Now it's not easy to learn, just like English isn't, but once you know it, it's much more expressive, and the rules are simple, but build off one another to form complex systems. I think this is actually very close to a natural language in a way.
S-expressions are less of a grammar. (send-to (account-of "John") money) has a tree structure that is known even if those symbols have no assigned meaning or lexical category. We know that structure by following just the tiny grammar that governs the parentheses and symbols, instead of the huge (and not completely knowable) grammar which governs sentences like "send the money to John's account".
Linguists do in fact use parenthesized notation ("labelled brackets")
Right, so parenthesized notation is basically an "AST dump" of a natural language sentence.
But when it's dumped to paper, the end result is a visual representation, which still needs to be parsed to consume again (and have the AST inside your brain) - you just have to do it according to the rules of that parenthesized notation.
So, when you see a sentence in this notation, do you find that it takes you less time to read than the original notation, or more?
For me, it's definitely more. And I think the reason why this is widely used in linguistics is not because it's faster, but because 1) different languages have different syntax parsing rules, and having a single language-agnostic uniform notation is desirable, and 2) unlike normal syntax, this notation is completely unambiguous even in corner cases - in particular, it can explicitly represent arbitrary nesting without ambiguity in a way that commas do not.
English prose, which presumably is very readable, uses very little in the way of visual patterns. Spaces between words, periods between sentences, line breaks between paragraphs, and a tiny handful of punctuation characters are enough to write an entire novel. Almost all meaning in English is represented by word order rather than punctuation or visual patterns.
Exactly like a programming language, then. But I disagree that all the things that you've listed are "little". Again, try ditching all punctuation, and see how much your reading speed suffers - even though it's the same words.
Just like PLs, natural language has to be parsed to determine word and sentence boundaries, before you can understand their semantic meaning. It's that parsing speed that spaces between words, periods between sentences etc affect. And similarly with PL syntax, it's what all those ()[]{} things do.
Prose is full of redundancy. You can skip lots of words and still get the gist of what's going on. The words themselves are also full of redundancies - rmmbr prse wrttn wth mssng vwls?
Lisp syntax is like no syntax at all, it's all just direct parse trees. It is unviable without more secondary formatting than syntactic forms since it is really difficult for the eyes to balance parantheses on their own.
That LISP-style (non) syntax is successful just goes to show how little syntax really matters (I.e. the null case works), I guess.
I get what you're saying, but that's a pretty extreme view. Parenthesis are obviously syntax, even if they look similar to the notation used to represent ASTs in other languages. They are minimal syntax, but they're still syntax (also, lisps have `"@' and sometimes {}[]).
Just because this is the internet, I'll also disagree with your second point. Lisp programmers use white space and macros to make programs more readable. Despite this, lisp's adoption trails many less powerful languages that offer more intuitive syntax. Since lisp's metaprogramming facilities allow it to effectively simulate pretty much every other style of programming, the only explanation that I can come up with for this situation is that intuitive syntax matters a lot (at least for language adoption).
> the only explanation that I can come up with for this situation is that intuitive syntax matters a lot (at least for language adoption)
Where intuitive is, as always, defined as "things I've seen before". Which makes the argument boil down to "popular things are similar to other popular things".
Currently the two most popular types of syntax are C-like (C++, Java, C#, JavaScript, etc.) and Python-like (Ruby, Elixir, etc.). Prolog suffered from the AI Winter similarly to Lisp, and thus Prolog-like languages (e.g. Erlang) still get complains about their "arcane syntax", just like Lisps.
Macros aren't so much syntax as they are semantics (inlined for performance, but we can still think of them as special procedure calls). Lisp has been relatively successful in its various forms, it should get some credit.
The DEFCLASS macro implementation will check some of that automatically, but much of the syntax has to be check by the macro in the implementation code.
> we can still think of them as special procedure calls
Lisp syntax is two stage: at the first level it is a data syntax. It provides via s-expressions a textual representation for data. It also helps to format data to make it more readable to a human reader:
but the tabular print may help with reading. S-expressions know nothing about Lisp syntax. All they do is describe how to textual represent data like lists, symbols, numbers, strings, arrays, ...
At the second stage is Lisp program syntax. It is defined on top of s-expressions.
This is the syntax for DEFUN, which defines functions:
It does format it differently from the similar structured list above. As a Lisp programmer I expect the second formatting for valid Lisp code.
The Lisp programmer uses the s-expression syntax as a clue, but actually follows the special formatting/indenting of Lisp code to read it. The experienced programmer doesn't see the parentheses, but follows the head symbols (here DEFUN), the special grouping, the blocks, the indentation. As a Lisp programmer I know that after the name the argument list follows and then optional type declarations and documentation string. After that I expect the body forms as a sequence of forms.
Have you ever programmed in a lisp language? Balancing parentheses with your eyes is not something I've ever had to do. If you have to do that you're using the wrong editor
But then you need x years to teach them all those mini macro languages like loop etc.
While I personally have pleasant memories of the time I used lisp, I learned to like languages with some syntax and the support editors and IDEs provide for that languages.
> If dealing with a function, the evaluation will always be arguments from left to right are evaluated first, and then the function is called with the result of evaluating the args. This is 95% of the cases.
> If dealing with a macro, go read the documentation for it, because every macro is a DSL, and evaluation can happen in different ways. This is the 5% case.
> If dealing with a function, the evaluation will always be arguments from left to right are evaluated first, and then the function is called with the result of evaluating the args.
The order of evaluation depends on the dialect. Common Lisp evaluates the arguments from left to right, but e.g. in Scheme the order of evaluation is deliberately left unspecified.
Exercise 4.1 in SICP [1] asks for an implementation that evaluates operands from right to left. :)
Other than that, I don't know of any implementations that do. I also don't know of any implementations that guarantee left to right order. Many manuals specifically mention not to rely on the order of evaluation (e.g. Chez Scheme). Since I try to avoid depending on unspecified behavior if I can avoid it, I might not notice when an implementation uses a different evaluation order.
I would expect one of the optimising compilers (MIT Scheme, Chez Scheme, Stalin) to be your best bet to find a real implementation with non-left-to-right order. And it might require a specific set of circumstances to trigger, so trivial examples might work ok.
And of course, just because current implementations use left to right order, doesn't mean that that can't change in the future.
I don't think that's the syntax people find hard. I think its the semantics.
The functional way of doing things confuses people. The fact that your program is a single nested chain of function calls I think is what people struggle with most. That and the dynamism.
The whole thing just needs time getting used to, because it's unfamiliar.
The thing about evolutionary advantages is that things that helped us in pre-agricultural age aren't necessarily so helpful in the so vastly different environment of a technological civilization.
I'm not so sure, because "not helpful" in the day-to-day experience in the technological civilisation is not the same as not helpful in the long term.
E.g. the progress of the technological civilization itself might not even be in our long term advantage as a species (we might think it is, but that's not the same as it being -- e.g. every too developed civilization might collapse over complexity, or kill itself, irrationality or not).
Another Algol-style language with a macro system in this vein is Honu (https://www.cs.utah.edu/~rafkind/papers/dissertation.pdf). It goes slightly further, allowing macros to do things like define new infix operators, and its layered approach (with an "enforest" step interleaved with parsing) is worth a look.
Since there's been interest in the D-expressions paper, I wanted to add that Jonathan Bachrach (the first author on the paper) is actually my Ph.D. advisor, and we just released a new programming language a few months ago called Stanza based on similar ideas. Stanza is a spiritual successor to Dylan of sorts: it has an optional type system, coroutines, a multimethod-based object system, programmatic macros, and an s-expression-based natural syntax. You can check it out at www.lbstanza.org.
Dylan did so much right. Well - the syntax was too verbose - but optional static types, pure OO, multiple dispatch.. that's an amazing set of features that would make me very productive.
There is a solid person who frequented here who was the last guy working on OpenDylan and mentioned he largely abandoned that for a new passion for Rust. Besides him, was that the last person really shepherding it?
That is me. I haven't seen much development effort since I switched to Rust. But ... I would still mentor someone who wanted to work on it or do something with it.
I was never at that level but still was very interested by your selfless work on OpenDylan. I was going through a Lisp adoration phase (not that it ended, really; one of many cycles I have) and found Dylan as "Apples prescient Lisp without parens from the future" elevator pitch and was surprised its tooling was better on Windows than Mac, basically because of you driving a very smally community of volunteers.
I've been working on a Lisp for the last couple of years. There's one major downside to reader macros / syntax extension: Your editor probably won't understand it, and in extreme cases won't know how to indent or read it.
Note that Lisp macros do not suffer this problem because it still uses the standard `'(,"foo") syntax. (In fact it's somewhat remarkable that Lisp syntax can be described in shorthand so compactly.)
Reader macros are a different beast. Let's say you want to add support for [] as indexing. You can do this:
> (let l '(a b c)
l[1])
"b"
And this:
> (let l '(a (b) c)
l[1][0])
"b"
And since the reader is under our control, it's easy to use a dynamic expression as the index:
> (let (l '(a (b) c)
i 0)
l[(+ i 1)][i])
"b"
If you try this, you'll find most Lisp editors work okay-ish with it.
Now let's say you want to be able to do this:
> (if x) {
(print "hello")
}
This is entirely possible since we control the reader. The rule is, if the following expression is a curly-body, it gets converted into `(do ,@body) and inserted as the last argument of the previous expression. So it becomes:
> (if x (do (print "hello")))
However, this fails hard inside of standard Lisp editors. You'll lose indentation right away. Nothing understands curly notation. For example if you turn on Clojure mode and try to auto-indent, here's what you get:
> (if x) {
(print "hello")
}
The alternative curly style is just as broken:
> (if x)
{
(print "hello")
}
So if you expose this power to users, suddenly you have an editor problem: How can the editor know what to do with tokens, short of running the program?
Even in Emacs, which is Lisp-based, it's not really possible to execute other Lisp variants. (I actually got both Scheme and Arc to work inside of emacs, which was amusing. No external programs; literally scheme, but running in elisp.)
But even if it were possible, you wouldn't want to. An infinite loop would freeze your editor. Not to mention it should always be safe to open a file for editing, yet this breaks that safety guarantee.
I think the solution is something like:
> (define-reader ("{" s curly-body)
...)
Within the define-reader block, `s` is the stream, and you can read tokens from it and return a result, or stuff the result into the previous token's last argument, or throw an error, etc. And since it ends with "curly-body", Lisp editors can be designed to detect this token and turn on "curly-body syntax" for any file that imports it.
It's worth it. Would you believe this is Lisp?
> require("fs").existsSync("/tmp")
true
It's equivalent to:
> ((get (require "fs") "existsSync") "/tmp")
But ten billion times more readable. And Lisp!
I have no idea why Lisp doesn't let you call functions like
foo(1 2 3)
instead of
(foo 1 2 3)
It's an unambiguous transformation in every case. Nobody smashes an atom into a left-paren, ever. If there's an atom followed by a left-paren, or a right-paren left-paren combo, it means you want to call it as a function.
It leads to some amusing situations:
(list 1 2 3)
list(1 2 3)
But there's an easy solution: If it looks confusing, don't write it that way. Just like every other language feature.
Even though Lisp is such an old language, I feel like we've barely scratched the surface. Clojure was a promising demo of what's possible. It's time for the next step.
The thing is, lispers don't want more syntax. They've rejected Dylan, sweet-expressions, and a million other experiments. There's a sizable contingent who complain about Clojure using [] instead of all ().
So really all you are talking about is making a nice syntax that enables macros, and that doesn't require lisp. See Elixir.
If lispers aren't going to adopt extra syntax, your only hope is to attract non-lispers, and then you've got the issue where your lispy language has to be better than existing languages at doing something, and that's hard.
Why go to so much trouble for something that's been rejected repeatedly? Macros don't require lisp syntax and that's the main advantage.
Lisp already has a lot syntax, but on top of s-expressions. Lispers usually want to keep that two-stage syntax: s-expressions for data and Lisp on top of s-expressions. This enables the simple s-expression data representation and manipulation to be applied to programs, too.
Lispers did not really reject the non-s-expression Dylan syntax - it was never developed for them. Apple developed that syntax and Dylan to target non-Lisp developers.
"I have no idea why Lisp doesn't let you call functions like
foo(1 2 3)
...If it looks confusing, don't write it that way. Just like every other language feature."
I used a similar ethos to create a whitespace-sensitive lisp with infix operators without compromising macros. The rule for whitespace-sensitivity was: if whitespace sensitivity was even slightly ambiguous, you lost it. Inside parens you lost it. Inside macros, you lost it. But it was still useful in a wide variety of situations: https://github.com/akkartik/wart/blob/master/004optional_par...
However, when it came to foo(1 2 3) I ended up just saying no. It has absolutely no benefit except being familiar to non-lispers (where whitespace sensitivity actually makes Lisp easier to read for anyone outside an editor). Particularly if you permit infix, (foo 1 2) and (1 + 2) have a nice correspondence in syntax. Function calls just become a form of prefix operator. More details: http://arclanguage.org/item?id=16924 (particularly the last couple of paragraphs)
I think a lot of the reason lispers look down at attempts to make indentation significant is that it's been bundled in the past with other syntax sugars to make Lisp more familiar to non-lispers. The goal shouldn't be familiarity, it should be power.
I don't know if you're actually looking for a solution here, but cider performs macro identation by delegating to metadata on the macro itself. This lets user-defined macros get correct formatting (for whatever value of correct the author of the macro decides).
You can reprogram the reader. But by default the reader knows nothing about a programming language. All it implements is reading of data: numbers, symbols, strings, vectors, arrays, ...
The reader does not know about functions.
Something like
(defun foo (bar) (+ bar 10))
is for the reader just another list. It does not know that the first symbol has to be DEFUN, that the next item needs to be a symbol, that the next item is a list of arguments and that then a sequence of body expressions follows.
Something like
(defun (bar) foo + (bar 10))
is also a valid list, but it is not a valid Lisp expression. The reader does not know that.
FWIWI The term "Reader macros" has slightly different meanings in Common Lisp and Racket. In Racket reader macros are used to modify the existing reader. Languages that have "almost" the same syntax as Racket can therefore use reader macros. Languages with new syntax (Scribble, Pascal, etc) don't use the Racket reader, but provide their own.
Special data. Lisp programs are encoded as data. Not every data is a program.
The reader does not know anything about Lisp. All the reader knows is data. It does not know about LET, nothing about LAMBDA, has no idea about DEFUN, does not know what an argument list is, it has no idea what a class is, ...
The only thing about the Lisp language the reader actually knows are abbreviations for QUOTE and FUNCTION. Everything else in the reader is about data representation: syntax for lists, numbers, symbols, characters, strings, bitvectors, structures, ... The reader has no parser for functions, definitions, special forms, macros or other Lisp language constructs.
It's pretty trivial to add macros to any language. The only thing that separates a macro from a regular function is time. In a normal function, the parameters are evaluated before being sent to the function. In a macro, the function is called before the parameters are evaluated. What separates lisp from other languages when it comes to macros is that the language is very simple and provides lots of tools for operating on the same data structure that you write your code in.
Of course you can add macros to any language, that doesn't mean they'll be anything like Lisp macros, because you're still writing code that has many different ways of expressing function calls. Importing a file? Oh, don't call a function. Instead write this special syntax which will call that function for you. Declaring or setting a variable? Same shit. What if you wanted to do something really fancy like create a class, oh yeah, there's a special way of saying that too. So now when you go to automate this stuff, you've gotta teach your computer all the special different ways to write everything. Lisp isn't lisp because it has macros. Lisp isn't lisp because it's simple. Lisp isn't lisp because it has a lot of really smart ways of doing basic things that we take for granted (like explicit vs implicit scope on local variables). Lisp is lisp because it has them all.
You can add the same extensibility that lisp has, macros, reader macros, operate on symbols, closures, whatever. You can do some of the really smart things that lisp does like explicitly scoping your variables or multiple dispatch. You can even have a really simple language, with a really simple syntax. But it's not Lisp's individual features that make it what it is, it's how these features interact with one another to give you something that's truly hard to replicate.
Yes, of course you can replace a block of code with another, but how easily? For example, yesterday I was thinking about a variable that represents the result of evaluating the previous form. So instead of having to read a method chain backwards: (wax (buff (rinse (soap car)))) it could read forward: (soap car) (rinse ?) (buff ?) (wax ?) Now whether or not this is a good or bad idea is completely fucking irrelevent. The point is that it's an idea, and until an idea is tested, we have no idea how bad or good it is. In lisp, in 5 minutes, I can write a macro that wraps around a block of code and implements this idea so I can actually play around with it in the real world instead of just in my head. You literally just take a bunch of forms in, put a let statement around the whole block, and then put a call to set the variable around each individual form (If you're not familiar with lisp. a form is kind of like a line of code, it's usually a function call but it can be a value as well). In lisp I can have very abstract programming ideas and then immediately implement them, without having to fight through the language to do so (ie without having to write a parser).
It's cool when other languages take stuff from lisp, it makes it easier to write lisp like code when I have to use other languages, but I've read a lot of articles on "Lisp macros in X Language" and none of them seem to understand that you can't just implement macros, you have to make the structure that code is written "first class". You have to give the programmer the tools they need to operate on a piece of code as naturally as they would a string, a number, or an array.
> It's pretty trivial to add macros to any language.
Not true.
> The only thing that separates a macro from a regular function is time. In a normal function, the parameters are evaluated before being sent to the function. In a macro, the function is called before the parameters are evaluated.
That's not really capturing it. A regular function expects to get passed evaluated results from valid code. The arguments are being evaluated. The function gets called and the parameters will be bound to these arguments.
A Lisp macro OTOH gets arbitrary stuff passed. A Lisp macro can enclose anything. This anything can look totally wild and does not need to be valid code. The macro gets this anything, it has to compute valid source code and can create any side effect it wants.
In a macro form, the macro function gets called with the code it encloses and an environment. It can then do anything it wants, but it has to return valid code. This code then will be evaluated - the generated code can be anything which is in that context a valid Lisp form: self-evaluating data, a variable, a special form using built-in Lisp syntax, a function form or another macro form.
> (If you're not familiar with lisp. a form is kind of like a line of code, it's usually a function call but it can be a value as well)
A 'form' is an expression which is also a valid Lisp expression (can be evaluated, etc.).
> But it's not Lisp's individual features that make it what it is, it's how these features interact with one another to give you something that's truly hard to replicate.
(<function or macro name> arg1 arg2 ...)
That's it. This is 100% of the syntax.
If dealing with a function, the evaluation will always be arguments from left to right are evaluated first, and then the function is called with the result of evaluating the args. This is 95% of the cases.
If dealing with a macro, go read the documentation for it, because every macro is a DSL, and evaluation can happen in different ways. This is the 5% case.
All the complexity of learning a lisp comes from knowing more functions, what they do and how to use them, and learning each macro's DSL.
Can this syntax be improved on, I'm not so sure. Clojure added some special syntax for some more literals, such as [] for vectors, {} for map, etc. And I do think in one way they help, but in others they hurt.
Maybe adding something that distinguishes a macro from a function, but once you know the names of them, the name is enough to know.
Maybe make it so there is a more visible demarcation between the function/macro name and the arguments. Not sure how though.
All that to say, if you give the ayntax a good chance, like 2 to 3 months of 40h a week for your brain to more clearly see where the functions start and end, I'm sure you'll like it.