Programming by modifying ASTs sounds very interesting and I really like Lamdu examples. I see, however, many problems, as not being able to introduce even slightest modification to the code using terminal, not being able to use textual tools, and problems with source control interop.
The last one is probably the most serious, as it doesn't seem to be enough to plug new diff tool into git. There are still places like GitHub, which are hard to modify. That would mean one would need whole new ecosystem for such language.
Having underlying textual format, possibly based on s-expressions might help. You could modify it using AST editor, but still be able to read the code as text and diff it. Other option is to have two-columns editor, with one column being text source and the other AST, making it possible to edit any of two and have the other updated.
On one hand, requiring a custom IDE to work with code just seems like a recipe for disaster.
But on the other hand, when I started Clojure full time, I switched to using Emacs + Paredit + CIDER (née nrepl.el). And come to think of it, I still open Xcode whenever I'm about to crank out any Swift or Objective-C code.
I think the difference between those situations and Lamdu's approach is that you still can access the files using any other editor you want. You just don't want to, because that given editor is best(ish) for that given language due to IDE features that help edit it more conveniently. Whereas this Lamdu stuff isn't just a matter of convenience, it's part of the language/environment/experience itself.
So yeah, I would never touch this, because it's basically got lock-in that requires its own custom everything, from version control, to needing to rewrite plugins my editor has but Lamdu doesn't (if that's even possible!), etc.
Isn't this the "sunk cost fallacy"? i.e. we've spent X effort-years creating text-based tools, might as well keep going with them, even if there's a potentially better way of programming?
Imagine semantic version control that can parse your AST and tell you which methods, expression, classes, functions etc were modified, instead of just syntax changes like "this line of text is different because you added a semicolon."
I'd love to see more semantic-focused tools, eventually leading to generic solutions that work across any AST. Polyglot IDEs like IntelliJ IDEA can come close, but only because they implement their own parsers, to derive an... AST!
Spreadsheets are not on the same level of complexity as programming and never can be. (In this way, I suppose I disagree with Chris Granger's latest ambitions.) I use spreadsheets for the tasks they're best at, and if I need CSV or some other format, I can always export it.
But what are their advantages?
After all, their supposed advantage is their live programming, which you claim you already have with ordinary programming languages.
I can't wait until Lamdu supports something equivalent to standard Haskell 98. Right now, it is hard to do anything useful. For example, I tried to du Project Euler's 3rd problem, but I couldn't, since I needed cons and it is a sum type constructor and they are not implemented yet.
There is a built in cons (in the type Stream which is a lazy list).
Sum constructors are supported but creating custom ones currently has a very cumbersome ui.
Yeah, in Yairchu's euler solution, it is called "NonEmpty".
Lamdu has a nominal type called "Stream" (nominal types are basically like Haskell's newtypes). You can refer to them via their names in holes. The purple arrows represent packing/unpacking depending on which side they appear.
The Stream nominal type has a single type parameter `a` which is the item type, and is defined as:
Stream a = () -> (Empty + NonEmpty { head :: a, tail :: Stream a })
(Note the + above is type-level plus, a sum type).
So inside the Stream you find a function from unit (representing lazy computation) to a sum type of 2 cases:
Empty - equivalent to Haskell's []
NonEmpty - a cons, equivalent to (:)
The NonEmpty case has 2 named parameters (head, tail). Maybe it will become an infix (:) in the future.
When you pack a Stream nominal, Lamdu auto-completes the lazy suspension (half circle character), and then lets you fill in `Empty` or `NonEmpty`.
When you unpack a Stream nominal, Lamdu offers completions that force the suspended computation (apply with `()`) and then do case analysis (represented by a colon) on each of the sum constructors.
There's the "Stream" nominal type, shown below in Haskell-like text syntax:
newtype Stream a = () -> (Empty | NonEmpty { head :: a, tail :: Stream a })
To construct a "Stream", we type "stream" in a hole and pick "Stream« ◗ _", which wraps a "deferred computation" ("◗ _") in the nominal "Stream" type constructor.
Now the type of the value inside the hole within is just "Empty | NonEmpty { head :: a, tail :: Stream a }", now type "nonempty" and pick it.
I don't want to be a skeptic, this question is serious, but has anyone produced a large project that used live-editing? I've never seen any example, they are all small code snippets. Any example would be appreciated.
Games/game-engines often can change scripts or even C code out on the fly, so they are at least partially build live.
Working in Smalltalk images can be "live-editing", if you run your application at the same time.
Both of these examples work on a method-basis: you change the code, trigger a rebuild the method and it is used from then on, not automatically after each change/keypress.
There is a bunch of live-coding stuff for creating music, but that often is used with smaller programs and writing the entire program as part of the performance.
I think here the more important thing is that it doesn't work on source code that is then parsed and compiled, but directly on a tree structure representing the program. Live-coding is sort of a side effect, because you a) don't have to reparse everything and b) always can have a state that is a valid program.
I use Webpack's Hot Module Replacement [0] along with Redux [1] and React. It's extremely useful for iterating on user interfaces. You just save and your styles and markup update without having to reload.
Furthermore, Redux has a devtools library [2] (you can view a gif of it in action in the repo) that lets you do time-travel for debugging purposes. I don't use it much, but it can be occasionally useful.
I would argue that most browsers qualify in this regard these days (at last partially). Live editing has transformed how I work when creating web apps.
I did it for a while ~2006 using ASP.NET, although it wasn't as in-depth as this. It was kind of cool, but after a while it was easier to just use text and view it in a browser.
I think it would be more useful to build a tool that helps visualize the overall structure of a program. The usefulness of such a tool would increase as the size of the program increased.
I'm interested in hearing about this too. If it counts, I certainly use R and Python in a live-ish manner but without a lot of the IDE support to truly make it live.
I submitted this because it IMHO is one of the more polished and approachable examples of an AST-based environment, which came up in another thread today.
I'm not an expert on language design, but it seems to me that some of the guarantees of that language are equivalent to actually executing the whole code, which might become really slow from the user experience perspective. Please correct me if I'm wrong here.
It runs the expression you write in the repl, it's as big as you make it.
It'll also be sandboxed against thrashing.
It's also asynchronous, showing results it computed so far. If example is too big you just get feedback that it's still computing.
One question about a thing I'm not really sure about w.r.t. Lamdu, from quickly skimming the page and the thread: is it possible to draft something syntactically and semantically incorrect first, and only start making it correct later?
"we made the syntax for "deferred computations" (aka functions with no arguments) very lightweight - simply a preceding ◗ symbol."
This is a nitpick, but an important one IMO - this syntax is only "lightweight" in an environment where the tooling types "◗" for you. Elsewhere (comment boxes / SO / IRC / etc), you're stuck figuring out how to type U+25D7 or breaking flow to pick it from a menu...
Why should we break compatibility with plain text based tools to achieve what Lamdu is set to achieve? AST could be an intermediate representation but source should be text. Parsing/generating source to and from AST is some work but doesn't look like an unsolvable problem. We would got all the benefits without actually throwing away all the existing tools. Am I missing something?
In Lamdu, each subexpression has its identity that survives as it is edited, moved, etc. This helps merge changes with far fewer conflicts.
Lamdu also allows attaching English names to identifiers, in addition to Chinese names, Hebrew names, etc. This approach can hopefully rid the world of duplicated libraries where the code is nearly the same but the names are Chinese. This kind of rich metadata doesn't survive well when serializing to text.
Lamdu will also maintain(and distribute) code indexing alongside the code such that the algorithmic complexity of refactoring large projects is not O(project size). Text is not a good data structure to perform these operations.
Lamdu also maintains many invariants about the code (e.g all type errors are localized and marked as such), what would Lamdu do if you try to load text that doesn't maintain the invariant?
We also believe the entire textual tool chain could be so much better if it were rewritten to work with asts.
So for us the only value in textual integration is the "old world" of programming, and we're creating Lamdu much because of our dissatisfaction with that world.
Some of this issues could be resolved while staying within plain text, well to some extent at least. There're various attempts to encode metadata into plain text source code. Loading partially incorrect code should be possible, that's what many IDEs do successfully.
Plain text is the most portable data format ever invented, any incentive to replace it should have very convincing benefits.
Loading partially incorrect would mean that the invariants are not, in fact, invariants.
Encoding the metadata in the text would lose the benefit of using text. At that point you may as well store the code as XML or indeed as we do, in a key value store.
We could layer our data on top of text but this would lose the algorithmic benefits of a good data structure, and wouldn't actually be editable reasonably with a simple text editor.
> "Live programming" means that the code is being executed as you type it.
Not only.
The article references related work, but somehow it focuses on a single level of liveness[0], which is instantaneous feedback. It is great when you recompile your game update function while running it, but as far as day-to-day coding is concerned, I find it sufficient, and in fact less distracting, to have feedback only when I want it.
I have just spent a little (too much) time playing with this.
How are you meant to implement the reverse function without a helper function? I tried defining an anonymous lambda function in a let statement, but recursion isn't allowed it seems. And interesting project though! well done
Erlang supports hot module loading as an in-the-field deployment concern. This is more like development tool. Similar mechanisms with completely different experiences.
It's been clear to us that we need animations for structural editing so that it's clear to the user what's going on, and we're not familiar with a GUI library that provides us with what we needed, so we made this custom one.
Currently it's part of Lamdu's sources, but we could make it an independent library if there's demand.
I like it the look of it very much, and I will have to test drive it tonight. I have tried so many livecoding environments mainly for generative art and music. At the moment, I am learning Extempore, however, it does not have the updating values and widgets in the editor; it updates on the fly the music, 3D or 2D content you are creating/modifying. For me, a marriage of this with Extempore, would be best, I think. I'd have to try to know for sure. Livecoding music and art is awkward still, even for the 'pros'. I am also using 8th a Forth-like language for true cross platform work, and I would like to integrate Paul Batchelor's Sporth [1] into it for my explorations. However, none of these have the code-side relate to the live updates. Atom and Emacs work well with Extempore. Good luck!
Wikipedia says: 'The syntax [of an AST] is "abstract" in not representing every detail appearing in the real syntax.'
So relating to Lamdu, you could say that what you see is the abstract representation. The representation is a bitmap, of course you can't evaluate that.
When you write C++, you actually just write characters, right? Does that mean that you are not writing a program? No, those characters may comprise a program in the end. I see Lamdu the same way: Even though you only manipulate a graphical rendering of the program, it still is all you need to write that program. Just like with C++, you don't see the actual representation of the program (because you can't really 'see' a data structure, you need to encode it), you only see the characters, but you can write the program anyway.
Lamdu is a purely functional programming language so far, I believe. You don't need input, since you can, in the one expression you have available, call a function with arbitrary parameters. You can say that it does not allow for long-running interactive programs, but that is easy to add later once the foundations are in order.
Isn't this more or less similar to what happens when developing in Java on Eclipse? To my knowledge, Eclipse does exactly the same thing: it compiles the code in real time, it loads the AST in memory and then it uses it to provide you powerful development tools.
Your "live AST modification" looks a lot like debugging Java applications in Eclipse ;)
Still, this seems a great idea! Even if it forces you to a single language/IDE combination (but that cannot be helped, I guess)
I like the colours and am curious if it's a known colorscheme available for other environments (Vim, Emacs, terminals) or is it designed from scratch. Thanks for pointing to the config, it could be used to replicate the colorscheme for other editors.
It's not a known color scheme. For a while we had an ugly color theme, and then I sat with a designer friend and we fixed it. I agree that it's pretty now :) Apparently colors needed to have consistent saturation levels and stuff..
Yea, this project fixes all the problems nobody ever has.
"See the value of some binary logic in real time"
"You can't make syntax errors"
All of this and more, and it works on a small subset of a language that nobody uses!
These are the sorts of things that would be really interesting to a student in their first programming class. I can't imagine anyone with any experience needing or even wanting this.
So I read that page, and I think the explanation is even worse than what I interpreted it to be the first time.
What they call 'blindly manipulating symbols' I call 'automation'. That is the entire point of computers in the first place, to very quickly perform tasks so that a human doesn't have to. That means describing the solution to those tasks. This is no different than mathematicians using equations. We don't need to do the entire process of solving every problem if they are the same except for changes in some parameters, we can write an equation and plug the parameters in. In software, we can write an algorithm/program that solves some specific problem, so long as the solution is the same and only the inputs are changing. Since programming languages support branching, we can make these algorithms as general as we think is useful.
Then you need to do a better job of running the code in your head and understanding what it does. Code that is 'written by debugger' is easily recognizable in that it is crappy in pretty specific ways.
If you often have to run your code so that you can verify that it is doing what you thought it would do, you don't have any idea what is going on and it's going to be a disaster.
Of course you need to run it to catch mistakes that anybody will make, but not to check that the code does what you think it will do.
For example, in some language you have my_list.sort() method. What does this do to the list? Does it sort it in place? Does this lock the list from other changes? Will concurrent changes to the list corrupt it? What is the expected runtime of this? If the list contains strings, how are they sorted? By unicode code point? By binary utf-8 encoded value? Randomly?
You should just know all those answers in whatever environment you are working in. If you don't you should study up on it. Trying to get a project finished by repeated guess and check is going to take an eternity and you are going to do a poor job.
Monolithic code is certainly one possible cause. Another is having to manually recompile, flip windows or whatever then open the app, get to the point where your code change has made an effect then test it. That's a lot of context switching out of the mindset of changing that code.
The last one is probably the most serious, as it doesn't seem to be enough to plug new diff tool into git. There are still places like GitHub, which are hard to modify. That would mean one would need whole new ecosystem for such language.
Having underlying textual format, possibly based on s-expressions might help. You could modify it using AST editor, but still be able to read the code as text and diff it. Other option is to have two-columns editor, with one column being text source and the other AST, making it possible to edit any of two and have the other updated.