Hacker News new | past | comments | ask | show | jobs | submit login
Astmaker – A DSL in Rust for programming language designers (david-delassus.medium.com)
89 points by linkdd on May 17, 2023 | hide | past | favorite | 33 comments



Rust seems to be good for language designers thanks to a combination of features - enums that can contain data (aka tagged unions) and exhaustive matching on all variants of that enum with the match statement. This makes it easy to model all the token types of the language. Good performance helps as well.

I reckon other languages could replicate this combination, it’s not very hard. Dart added this feature recently. I just don’t know why it’s taken so long for tagged unions + exhaustive match to become widely adopted.


I prefer having a GC, makes things easier. Also a copying GC can be better for some of the stuff you do in a compiler (immutable data structures).


Easier for quick prototypes, for sure (although I switched to Rust from Haskell due to even worse compile times in Haskell and my disdain for monad transformers). Also, Rust can't currently do nested pattern matching through Boxes, which can be annoying, coming from Haskell.

But for a high performance compiler, manually managed arena allocation will beat out a generational GC.


box_patterns exists on unstable. Unfortunately it's still not merged.


Chez Scheme would disagree.


How so?


Rust's enums/matching come from its ML lineage (OCaml, etc), and that family of languages has strong roots in programming language work ("ML" stands for "Meta Language"), so it makes sense!


Exhaustive pattern matching and tagged unions (sum types) are both great. Haskell's had them since its inception.


well, sadly, you have to turn on an option to get exhaustivity checking. Which annoys me, because sometimes I forget to turn it on for a small project/script.

I need to come up with a haskell "new project" template thing


Wall!it’s great.


[A clarification to the less familiar: --Wall is a compiler option that stands for "all warnings".]


And in ghc is enables coverage checking for pattern matching. The most amazing compiler feature in most compilers

Thx for expositing my remark :)


> Rust seems to be good for language designers thanks to a combination of features

Not only the type system is quite nice, the ecosystem is full of nice crates:

  - lexing: logos
  - parsers: rust-peg / lalrpop / nom
  - utilities: string-interner / linecol / snailquote
I also wrote about this here: https://david-delassus.medium.com/writing-a-simple-lisp-inte... (friend link, no paywall)


I came cross this post when I was working on my own LISP parser :) Very cool

Though personally I think writing LISP interpreters with parser libraries should be considered sacrilege


> Though personally I think writing LISP interpreters with parser libraries should be considered sacrilege

Agreed, though the point of the article was to explore the Rust ecosystem. When learning, a little bit of blasphemy isn't a bad thing :)


One question I always ask myself when I see a crate that uses macros is "how much will this crate disrupt my IDE experience?" If it breaks rename and goto definition, then I'm not sure it's worth it. Especially with AST definitions where I will be going back to my AST definition a lot.

Also, I find the whole visitor pattern abstraction a little pointless in Rust. I can't help but feel that applying a visitor pattern is taking an idiosyncrasy of object oriented programming and forcing it into Rust. Basically abstraction for abstraction's sake. I'd much rather write a bunch of `visit_expr`, `visit_stmt` (really `type_check_expr`, `type_check_stmt`) functions that pattern match on the data. It's much more searchable, and condenses the information at the pass level and not the data structure level. I care much more about having all of my type checking methods in one place, than I care about having all of the transformations of `Expr` in one place. I suppose if you're doing a lot of passes over the same representation and you need to schedule them in an efficient way, a visitor pattern could work. But how often are you doing that? Maybe if you're writing an optimizer but more often than not I'm doing a transformation pass from one IR to another.

With that said I can sympathize with ASTs being a lot of boilerplate. It took a lot of typing to thread location data through my AST.


The rust-analyzer extension for VSCode is quite good for this.

Though, I should try to use `quote_spanned!` instead of `quote!`, but I'm kind of a newbie with proc-macros.

As for your take on the visitor pattern. In my previous code, I had the `visit_expr`, `visit_stmt`, etc... And it's a lot of boilerplate, I wanted to reduce the amount of copy/paste and facilitate refactoring, hence my DSL.

I do have multiple phases during compilation where I walk the AST and the "only" difference is the return value of those `visit_*` functions and their body. Here, I just have to use a `model!{}` macro for each "phase", which is less boilerplate code.

In the end, it's just static dispatch through the use of a trait that I called "Visitable".


The macro's input syntax could be improved by more directly matching the generated code ([C-EVOCATIVE][0]).

Rather than

  pub node Expression =
    | BinOp -> Node<BinaryOperation>
    | UnOp -> Node<UnaryOperation>
    | Num -> Node<Number>
    ;

  pub node BinaryOperation = {
    lhs: Node<Expression>,
    op: BinOp,
    rhs: Node<Expression>,
  }

write

  pub enum Expression {
    BinOp(Node<BinaryOperation>),
    Node(Node<UnaryOperation>),
    Num(Node<Number>),
  }

  pub struct BinaryOperation {
    lhs: Node<Expression>,
    op: BinOp,
    rhs: Node<Expression>,
  }
[0]: https://rust-lang.github.io/api-guidelines/macros.html#input...


You can define a tree-walking interpreter in an ML just by recursion and pattern matching.


I'm really upset Haskell is so slow/GC'd/etc. Every other language feels like a chore coming from it, particularly for parsers and interpreters.


Having a garbage collector is good in a compiler.


Medium sucks, I don't like reading stuff there.


https://david-delassus.scribe.rip/astmaker-a-dsl-in-rust-for...

courtesy of a Show HN project from a while ago


Just for the record, this will also bypass the monetization I get from my articles. But I don't mind, thank you for sharing this so more people can read my work.

I'll keep publishing on Medium though :)


Then don't, I'm not forcing you.

I use Medium because I can monetize easily the articles I write, though it's not much (currently around 10-15$/month).

I'm not doing it for the money, but it's still nice.


All programmers are language designers and there is actually no distinction between programming and language design. This peculiar belief that there is a distinction is the root of most of the most serious problems in this industry.


You can express the same idea in english and german and french and ...

But if you try to invent your own language (like Tolkien did), you are not "expressing an idea" (aka: speaking), you are creating a framework to express ideas.

When you write a program, you follow a grammar to express an idea (the solution to the problem you are tackling). When you design a programming language, you are creating that grammar (using another language to implement it afterwards).

Also, here "programming language designers" designates "people who are working on their own programming language". Those are the people I'm talking to when writing this Rust crate and blog article. The webdev doing CRUD microservices could not care less about astmaker.


Is it just me or is this paywalled?


https://david-delassus.scribe.rip/astmaker-a-dsl-in-rust-for... will help that problem, and strip a bazillion tons of JS out while it's at it


Same here, unfortunately.


HN removes the friend tag in the URL so I can't share the "no-paywall" link unfortunately.

Though, this is fine by me. I don't mind people bypassing the paywall, but it's a source of (rather small) income for me :)


So, what's the story with the friend tag -- it's a paywall bypass for everyone who happens to see this submission on HN but Medium gives the finger to anyone who would otherwise discover that link organically?

"My blog but secret" is a very, very strange promotion strategy IMHO


As a non-paying member (or anonymous visitor) you get to read 2 "member-only" articles per month.

For "member-only" articles, you get a remuneration proportional to the "member reading time". For members who subscribed via your "onboarding" page (I don't have one I think, I don't know, I should look into it), you get a remuneration proportional to their reading time on any articles, not just yours.

The friend-tag on an article's link removes the paywall, allowing you to increase views when sharing the article. The more views your article has, the more Medium will promote it. But many platforms removes it as it may have trackers associated to it (I haven't looked into that to confirm or infirm).

EDIT: Fun fact, the Javascript subreddit even forbids medium links because they consider the content to be spammy 99% of the time.

For Medium, they want to incite people to buy a subscription, hence the paywall. But they also want writers to generate the content for them, hence the friend-link and remuneration model.

The remuneration is quite small, so I don't think they are loosing money. But short clickbaity articles about how console.log works can generate a LOT of views, and writers spamming those can earn up to 300$/month (or maybe more), hence the bad reputation of Medium. I'm not interested in doing that, I mainly write to share about my work and just enjoy the small remuneration as a reward for my open-source work: it's not entirely useless if I can get a meal per month out of it ^^

EDIT: Out of transparency, by writing an average of 1-2 articles/month on topics I enjoy, I managed to make an average of 10-15$/month. The ones about Rust are the more profitable ones, go figure :P




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: