The argument here boils down to: I haven't seen it work in places where I've been employed so it must not work.
Reality check: I've worked on multiple large research teams that successfully developed DSLs still in use by domain experts -- military sigint in one case, doctors in another case, industrial engineers in another.
We developed these languages for specific applications where the domain experts needed to program extremely complex systems to perform tasks in much shorter timeframes than would be possible if programmers needed to get involved.
Why did these DSLs succeed? Funding, manpower, and experienced project managers who delivered constant access to domain experts.
Delivering a successful DSL targeted at narrow domains with non-programmer users is much more resource-intensive than normal software--it's more on the complexity level of a compiler or small OS--but that doesn't mean it can't or shouldn't be done. Plenty of systems couldn't be automated otherwise.
Anyone who thinks DSLs for the general public can't work has never used Excel. The real trick is creating a domain model that "clicks" in the heads of your users.
I wouldn't go as far as that, but yeah. Adding up a column is not the same as actually coding in Excel or even using a few math functions in a cell. And considering how many people take classes just to handle the basis spreadsheet functions, much less anything complicated, the naive accessibility of Excel can be overstated.
I've never tried to make a DSL for non-technical people because every single time I've seen a client get an expensive product aimed at their industry and designed to "not need a programmer" to customize, they end up paying to have me or a coworker to not-program customizations for them in an excruciating interface.
On the other hand, I think a lot of this side stems from insistent terminology in treating programmers like some tiny priesthood separate from the general public. That woman who can actually put together fairly complicated SQL queries? She has learned to program in SQL. That guy who put together a ramshackle VBA atrocity that gets the job done? He has learned to program in VBA.
All the mess comes from not accepting that. No DSL or templating interface or whatever will let people do programming without learning how to program to some little extent. Acceptance fails when the "public" doesn't want to give/can't spare the time and effort to do so. On the flip side, when it doesn't fail because the "public" does work to learn to program in some case, developers take that to mean that this can work in any case, so long as the public is approached properly with the right DSL/graphical environment/etc.
> designed to "not need a programmer" to customize
> they end up paying to have me or a coworker to not-program customizations
These things aren't contradictory, though. The software doesn't need a programmer to customize. They likely tested that, and proved that regular people can set it up just fine. Regular people just aren't normally motivated to do so, when it's so easy to just spend money in order to not have to learn something new.
IMHO there's a clear division that forms between people very early in life, and it can be predicted which side of the line someone falls on as early as the third grade: there are humans for whom solving puzzles that involve "wrapping their brains around" a new mental skill is a fun diversion; and there are humans for whom that same experience is aversive and something they will flinch away from. And—importantly—this has nothing to do with how good they are at doing it!
We can endlessly invent new ways to make hard things easy, but no matter how easy we make them, as long as they're at all novel or unfamiliar, the people for whom absorbing new mental models is an aversive experience will just do everything they can to avoid it. It's like people with social anxiety doing way more work of some other type to avoid socializing, but with "forming new mental connections" in place of socializing.
But the fact that a large part of the population is like this, doesn't mean that there's no value in these customizable systems. They still increase accessibility, bringing tasks that were impossible for a non-programmer into the realm of possibility! It's just that for most people, the line isn't between "possible" and "impossible"; it's between "trivial" and "not worth the trouble." The question only becomes one of possibility when you're forced into a corner of solving the problem yourself, no matter what, unable to even quit and walk away. When people are in that situation, they're thankful for these systems. But only then.
Good DSL are hard and expensive to design and maintain. Look at the good example of DSL in comments here. They are not small project with small workforce.
If not very limited, a DSL is an ambitious project (if very limited a GUI will surely do a better Job).
Developers should think twice before writing a DSL for "non-programmers"
Nonsense. Making a good DSL is easy. I’ve built production DSLs that users adore.
What’s hard is finding developers who know the domain and its users in the first place. You can’t solve a problem if you don’t understand the problem space. And most devs don’t understand shit except how to turn it off and on again, nor are they willing to learn. So if your average professional university-educated programmer can’t even get a simple turnkey CRUD app right, what chance of them delivering a tool that successfully hands those keys to its users?
..
Go read the “Collaborative Work Practices” chapter of “A Small Matter of Programming: Perspectives on End User Computing” by Bonnie A Nardie that describes how basic users, expert users, and trained developers can develop their own software tools together, where each layer supports and educates the layers above and below it.
And then count the number of programmers you know who’d embrace working like that. Sadly I doubt you’ll run out of fingers.
The counterpoint would be Visual Basic. It was meant for non-engineers/managers to be able to write programs, but that never really happened. It was always easier for a manager to write an email 'I need an app that does X' than to write the app themselves.
I think later-legacy-VB (eg 6) suffers harshly from failing to properly separate primatives.
Imho (open to disagreement), one should think incredibly hard before adding primatives to a DSL. Because what that's essentially saying is "My abstraction was incomplete over the target domain, and now I'm going to hack something in." And now you have a "domain + some other stuff" language, which starts to look like a general purpose language.
SQL vs VB is an excellent example of focus.
(Although I do think VB.net added sanity back to the mix, by saying more clearly "These are VB things" and "These are .net things", and if you want to do the latter in the former, just make a library call)
I saw VB6 used by many non-programmers. It often wasn't pretty, but got the job done (helped in no small part by a rich ecosystem of COM/ActiveX components).
There are many more success non-popular stories around DSLs. Last year, in the Community event of JetBrains MPS, a bunch of big companies (Siemens, Bosch, Workday, etc.) showcased their DSLs built with MPS technology. Here are the videos: https://www.youtube.com/watch?v=102hoaAdctk&list=PLQ176FUIyI...
That may be true in some sectors but not all. In capital markets (banks, investment funds etc) a lot of coding (via DSL or not) is done by folks who are not developers/programmers or self-declared coders. They just get it done and don’t bother calling on IT which is often counter-productive. Wether its SQL, VBA, R, Python or some some horribly-convoluted Excel worksheets, there is plenty of technical work getting done by non-developers.
Visual Basic definitely opened up programming for a lot of people that otherwise never wouldn't have written software. You will never be able to create something everyone can use (unless it can do mind reading and even then I am not sure) but tools like VB, Access and Excel have enabled a lot of people tow write software.
COBOL is specialized for business applications by supporting records (i.e. punch cards) and decimal numbers (as opposed to the binary floating point of FORTRAN).
Visual Basic is specialized for using COM (with a high-level interface to its features), I suppose - otherwise it's definitely general-purpose.
> Anyone who thinks DSLs for the general public can't work has never used Excel.
I would say Excel is exactly the opposite of a positive example. Unless you mean being used as a sort of electronic scratchpad, which works okay.
But apart from that there's too much freedom in Excel to be used to make anything properly. Every time I've ever seen a non programmer use it, they make a pile of spaghetti. You never have any assurance that you're actually calculating what you want.
Even highly qualified domain experts (eg in derivatives trading) will end up building utter nonsense in Excel, and then use it to trade millions of dollars worth of exotics. Whenever they need something fixed, it takes an enormous effort to trace through all the cells to make sure things are correct.
Excel is a great example of a domain-user tool that's very hard to use correctly, and even harder to know when you're using it incorrectly; that's not "snobbism".
Totally agree. Whole companies run on Excel spreadsheets maintained by non programmers. Maybe they are not up the quality of good software engineers but they are good enough. When I was contractor I saw in one company the IT department taking over all Excel spreadsheets and creating "proper" software. It was a total disaster. Instead of same day turnaround users had to wait requirements and then wait for months often. Costs went up, productivity down. Good software engineering practices have their place but if something works we should leave it alone.
I don't think it is snobbishness. Some programming systems definitely encourage hacky spaghetti more than others (if you don't believe me try using LabVIEW) and Excel is definitely in that category.
There are ways that they could make it less so, like only allowing cell references by name, using a saner scripting language than VBA, allowing multiple cell grids on a single page/sheet, etc.
"Millions of successful businesses are built on Excel" is no argument at all. You can build a successful business with pen and paper.
I'd love to hear your take on the rise of "notebook" environments. :)
Directly stated, I don't feel any dissonance agreeing with both of your posts. I do think it is snobbish behavior. I also believe some environments make for worse results on some problems. There is usually room for both.
Generally my feeling is somehow similar to the one expressed in the blog post, ie. that DSLs don't work for non-technical people. If someone is able to use a DSL then he/she is also able to do regular programming, and many people don't have such abilities. And regular programming languages are better tools for the ones with the abilities.
> If someone is able to use a DSL then he/she is also able to do regular programming, and many people don't have such abilities.
I'd disagree with this characterization. Because a good DSL abstracts away things that are not relevant to the domain (e.g. in Excel, memory pointers and allocation) while retaining those that are (which data transformation you want to apply).
It doesn't follow that someone who understands the concepts in the DSL must also understand the superset of all regular programming concepts. Hell, most Java coders couldn't even write a memory manager from scratch!
Which is exactly where their power comes from. Because each time you shrink the required knowledge to do work, you broaden the pool of people who can perform that work.
> SQL is used widely, but it does not work that well for people who are not programmers.
Is the key phrase that shows the author has no idea what they're talking about.
I can count 20 analysts, off the top of my head, that I personally interact with on a weekly basis at work that disprove the point.
They have incredibly deep knowledge of their particular datasets. They know more than enough SQL to produce efficient queries for their needs. And they wouldn't understand exception chaining if it bit them in the ass. But they still run their part of the business better than I would.
Author is making an "anything shy of perfection isn't worth doing" claim, which misses the forest for the trees. The objective of programming (DSL or otherwise) is to get work done.
Not to be ideologically perfect.
Which is one reason Python is wildly successful, despite all its Oops design bits.
> (In author's defense of DSLs) There are AWK, RSpec, EJS, Emacs Lisp, bash scripts and a lot of others that are cool because their domains are steady and they are used by professionals.
The pompous twit... so data retrieval isn't a steady domain and business analysts aren't professionals?
> Which is exactly where their power comes from. Because each time you shrink the required knowledge to do work, you broaden the pool of people who can perform that work.
That's an excellent point.
It seems the same concept could also apply to broadening the scope of complexity that an individual could handle. The fewer things that the user needs to have in mind, the more mental space is freed up for other things, like domain complexity.
> Because a good DSL abstracts away things that are not relevant to the domain (e.g. in Excel, memory pointers and allocation) while retaining those that are (which data transformation you want to apply).
I agree, I wrote this small example on using DSL for E2E testing, where you abstract away a lot of asynchronous calls, headless browser specifics and other low level concepts to expose a simple API so domain experts can help you to write tests https://github.com/davps/tic-tac-toe/blob/master/src/App.tes...
What is the criteria you use to determine they’re successful?
If you have an interactive debugger and a way to test the code, I could see it being feasible. If not, then there’s plenty of examples of DSLs that are in use that people are forced to use and squander time and brain power due to a lack of concepts that real languages have long since addressed.
Since others have mentioned Excel, how many mistakes have been made in long complicated formulas in little tiny input boxes with no way to comment or even explain what on earth the formula calculations are doing? Yet that’s still not addressed, and it’s a nightmare when you come across complicated spreadsheets for this reason.
It actually doesn't matter. Yes, we can all agree that Excel is a dreadful interface for a programmer. But programmers don't use it for things better done otherwise. The people using it are not programmers. If you suggest they try to accomplish their ends in more sophisticated language, they look at you with wide-eyed horror.
Every moderately-sophisticated Excel document written by a "non-tech" person is something which would previously have required either the endless back-and-forth of a middle manager trying to explain a poorly thought out spec to a programmer or, more likely, simply have been done by hand till the end of time.
Maybe it could have been better accomplished by that middle manager learning a real language. But it's neither here nor there, because that was never going to happen. Excel, for all its faults, actually convinced people that they could build simple programs (by pretending they weren't). That is the criteria by which it is successful.
The question I'd ask is: why do you think it's important that all of these features be included? Why should it have a debugger? The people using it don't want one - make it look any more like programming and they'll go back to asking IT. The people who'd appreciate a debugger - they aren't programming in Excel. Who is served by making Excel a more sophisticated language?
When an Excel document grows enough legs, just sometimes someone will decide that it's useful enough that they should build a proper service to replicate its functionality. That's the point at which so-called "real" programmers want a debugger in Excel: because someone with talent but lacking experience has built a monstrosity that now needs to be properly replicated and made maintainable.
Especially when the original developer's manager starts asking pointed questions about how their one "non-programmer" could build something in Excel that then takes a whole team ten times as long in a proper programming language.
Having a debugger available to help reverse-engineer the spec would be really useful.
You could innovate and make spreadsheet debugable, or allow comments.
The argument that users don’t want a way to know what the logic they’ve created does is taking the point of view the users are idiots and don’t know any better rather than the developers aren’t thinking about how people are to reason about problems. If anything it’s the fault of the developers for not thinking clearly about how problems are solved in general and are locked into thinking in terms of their computer’s existing poor approximations of reasoning.
There are important differences here. One is the level of engineering effort, but another is the nature of the user of the DSL. Doctors, engineers, and SIGINT specialists are all types of people who are accustomed to doing some type of specialized technical work themselves rather than depending upon others to do it for them.
> experts needed to program extremely complex systems to perform tasks in much shorter timeframes than would be possible if programmers needed to get involved
I always think of DSLs as automating my own job as a software engineer. It makes it possible to have someone with a different skillset than me not need to go through me to build features.
> I always think of DSLs as automating my own job as a software engineer. It makes it possible to have someone with a different skillset than me not need to go through me to build features.
It's interesting that this sort of drives against our own motivation of professional self preservation.
In my own experience, it has generally worked out better to just empower others as much as possible, rather than trying to lock down and add friction and permission to the system.
It's a great feeling to give someone the power to do something they couldn't do previously.
Everyone has to admit there's a lot of programming minutiae that could and should be thrown overboard.
If you show your language / app / webpage to someone and they ask "Why is X preventing me from doing work?", it would behoove all of us to honestly ask ourselves "Is X important enough that we should keep it?"
Otherwise, we end up with 10,000 gotchas and only someone who studies for 2 years can use things. And I'd personally much rather that surgeon be studying medicine for those 2 years!
Which is a different way of saying that most computer UI still sucks for most users. I agree, and I think that one of the key reasons programmer-y people are successful at navigating computer tasks that stymie the proverbial grandma is this: faced with a question about how to accomplish a task inside some UI, we're just inherently more likely to follow a similar train of thought as the person who wrote the program in the first place, and reach the correct conclusion. (And case-sensitivity in a username is exactly the kind of keen (textual) detail that we're both wired and trained for.)
In other words, we're just iterating on trying things [as xkcd observed][0]. The key is that inside that first decision box, locating an action to try, we have a huge leg up on the "...which looks related to what you want to do" part. I sometimes describe this as "knowing how computers think", but really it's knowing how the people that wrote the program think.
Not that it's an easy problem, but I think we still have a lot of work to do as a field, to make stuff accessible to people whose brains don't follow the same patterns as ours.
Agreed - great insight. And this effect is exacerbated when there is genuine malicious intent on the internet. I've sat with my mum or dad on several occasions, when they've been confused by 'ads' on websites which are designed to look like dialog boxes.
It was far too difficult for me to explain how to tell that this was not a legitimate dialog box. Or understand the concept of phishing.
> The last time I had a business analyst that wanted to write SQL, he tripped over one join and asked me to do the job. SQL is used widely, but it does not work that well for people who are not programmers. HTML and XML were also originally intended for non-programmers - where are they now? They are used by amateurs or enthusiasts sometimes, but most of the time programmers do the job.
At both is my last two jobs (SaaS and e-commerce startups), the marketing teams had several individuals proficient in SQL. Hell, in some ways (complex read only queries) they were probably better than me simply because they did it more often. There existed BI tools and dashboards for many things, but the savvier and more curious individuals always wanted to dig further out experiment with queries before getting them formalized like that.
Members of the email team knew a fair bit of HTML and CSS too. And email is not what I'd consider one an easy HTML medium.
Due to these examples, I can't agree with the article at all. Must be a selection bias experienced by author.
My experience has been there are definitely people who become very proficient with sql while not knowing any other programming but they generally hit a ceiling. For most people they can do joins and some basic nesting but they start getting out of their elements if you mention window functions, ctes etc
I'm just one data point, but this group I'm thinking of uses a LOT of CTEs. Perhaps too many - they seem to always have a bunch in their saved queries, personal notepad, etc and would probably benefit from being allowed to create views so that they'd have smaller queries with fewer WITH clauses.
Seeing things like the above or spreadsheet power users really makes me think that "everyone should learn to code" education / literacy initiatives could be a very good idea if done properly.
Last time I tried to get my coworkers to use sql for a report, they straight face argued that doing all of the logic in code is clearer. So, I suspect it is selection bias, as well. Amazing how hard that is to look past.
I've met many non-programmers who are quite comfortable with SQL. There are many smallish scripting languages that should qualify as DSLs that allow lay access to domains that mystify normal people. Excel comes to mind. Mathematica used to be one of these before it became a real general-purpose programming language, if a somewhat limited one.
The author appears to be projecting their personal convictions onto everyone else.
SQL can bite back with the sometimes non-intuitive consequences of its version of three-valued logic. Unfortunately, some professional programmers have trouble with it. I once had to sort out a situation where the developers were blaming the users for not using the system correctly, when the problem was with the queries.
Spreadsheets are another good counter-example to the author's thesis. They, too, can be misused, but that is outweighed by the great many cases where they are used well by non-programmers.
Autohotkey is another one of those scripting languages. ::F4:: Send {Alt down}{F4}{Alt up} is so much simpler than whatever I would have to do with Java to remap keys. Same with stuff like WinActivate (window title) and MsgBox "hello world". It does get complicated like the OP says pretty quickly, with match modes and conditionals and stuff, but you can go years using it without getting any deeper.
I would argue that SQL is an excellent language and the only issue is with the actual inherent complexity of what goes on behind the scenes. Writing the equivalent of even a modestly complex SQL query in any other language is much more tedious and error prone.
I think it's better to call the users of such high-level DSLs, accidental programmers. It's not like they are not programmers, its just that they don't see software development as their vocation or profession. Yes, it's important to have languages that target them (e.g. Excel, Latex, SQL). Design of such DSLs is important and hard work. We need more professional activity making DSLs usable, consistent, and extensible.
I agree. I think what the author was trying to say with his article is that DSLs where end users can use the system with zero training are a hoax. You can't put SQL, LateX or even Excel in front of someone with no context or help and expect them to understand how to use it. But with a few days or weeks of training, a business user can go from being helpless to being able to query their organization's database and create fancy charts.
Much like most things in software, it's key to hit the correct level of abstraction.
Completely agree with your points, except the extensible bit.
I believe having a DSL with strict boundaries will make it easy for new users to grok than having to go through N pages of modules(/ libs / extensions) documentation and feeling overwhelmed.
Smaller DSLs that could be mix n matched depending on the problem at task might be a better fit.
Wait, I feel like I'm describing a std lib here...
Reminds me of a speech rms gave about programmable editors. Money quote:
"
Multics Emacs proved to be a great success — programming new editing commands was so convenient that even the secretaries in his office started learning how to use it. They used a manual someone had written which showed how to extend Emacs, but didn't say it was a programming. So the secretaries, who believed they couldn't do programming, weren't scared off. They read the manual, discovered they could do useful things and they learned to program."
"""
Shel wrote Mailman in C, and Customer Service wrapped it in Lisp. Emacs-Lisp. You don't know what Mailman is. Not unless you're a longtime Amazon employee, probably non-technical, and you've had to [directly talk to customers]. [....]
Mailman was the Customer Service customer-email processing application for ... four, five years? A long time, anyway. It was written in Emacs. Everyone loved it.
People still love it. To this very day, I still have to listen to long stories from our non-technical folks about how much they miss Mailman. I'm not shitting you. Last Christmas I was at an Amazon party, some party I have no idea how I got invited to, filled with business people, all of them much prettier and more charming than me and the folks I work with here in the Furnace, the Boiler Room of Amazon. Four young women found out I was in Customer Service, cornered me, and talked for fifteen minutes about how much they missed Mailman and Emacs, and how Arizona (the JSP replacement we'd spent years developing) still just wasn't doing it for them.
It was truly surreal. I think they may have spiked the eggnog.
Shel's a genius. Emacs is a genius. Even non-technical people love Emacs. I'm typing in Emacs right now. I'd never voluntarily type anywhere else. It's more than just a productivity boost from having great typing shortcuts and text-editing features found nowhere else on the planet. I type 130 to 140 WPM, error-free, in Emacs, when I'm doing free-form text. I've timed it, with a typing-test Emacs application I wrote.
"""
It's funny how some Smug Emacs Lisp-guy are confronted with why the Emacs with extension mail client was deprecated. It must have been some life changing experience chatting with non Smug Emacs Lisp-guys people actually wanting to use Emacs.
I can quite understand why you would want Emacs for Customer support emailing, with the ability to semi-automate tasks and extend extensions. If it's your full time job to write quite similar emails, it's probably worth the effort.
I made a post elsewhere earlier today to demo Multics Emacs, and just so happened to link to the very manual that rms refers to in that quote - the "Multics Emacs Extension Writer's Guide".
To add another counterexample into the mix, I’ve met many artists and designers who are proficient with Unreal Engine’s Blueprint with no C++ experience. Some have even shipped games made entirely in blueprint with no traditional code.
To add to that: Clickteam's Multimedia Fusion 2 Developer, which I learned to make games with back in elementary school. It is strictly (although not exclusively) "no code." Hell, one of my favorite indie games of all time, Knytt Stories, was made in it.[1]
I think you can add stuff like Twine and Inkle's scripting language Ink to that. Their user base is emphatically not programmers, but they are definitely domain specific languages.
The SQL example is rediculous. I wasn't a programmer when I decided to use SQL, but learned it to be better at my job. After I learned it everyone else thought what I could acheive was magic. I only had read access so I had to be able to perform advanced subqueries, windowing statements , etc. After a couple of years my skills could embarrass most programmers in the company (minus the Oracle dbas). This is because the programmers wanted to write SQL in a proceedural way using plsql and were never very comfortable with set theory.
This knowledge expanded my power within a mutibillion dollar company and I went from temp analyst to director of supply chain business system and informatics within 3.5 years. I also decided to learn python due to the data libraries, and my power grew more as I automated most routine processes in my department. I required every employee of mine to learn SQL and gave them 3 hrs every Friday for self study. Our supply chain went from being ranked 340 in the country ( for our industry) to the within the top 10.
Don't tell me non programmers can't learn SQL. Also these DSL get people like me that should of gotten a CS degree the experience and confidence to get deeper into programming. Now I write python and clojure. In fact, because I have deep domain specific knowledge in supply chain I can apply my programming in ton of different ways. I find most corporate programmers have all these skills and either don't really know how to apply it to really help the business, are too lazy and think of the business stakeholders as a annoyance, or just do their job and go home.
I think it is going to be hard to bea white collar worker in the future if you don't know how to program.
It would have helped if the author would have given their definition of a DSL at the start.
Embedded in a later paragraph a DSL is defined as having a syntax where "code that must look similar to proper English of the domain."
I tripped over that because, as a schemer, it's easy to discover there's really no fundamental difference between a "DSL" and "a program", in any programming language.
Completely agree; this is a difference of degree, not kind. Quoting myself[0]:
> As soon as you have something as apparently simple as named procedures, you're really writing a DSL, albeit very coarsely, for your business problem.
For most of the article the author seems to be discussing non-professional programmers rather than non-programmers. Or perhaps using the two interchangeably?
There is certainly a need for non-professional programmers to be able to program (e.g. in the sciences). For non-programmers, programming a task via GUIs have been effective, although admittedly in the worst case they can devolve into a mess of check boxes, drop downs and spinners.
EDIT: Of course in a business you'd hire professionals for a job (or outsource it). That's just good sense for anything your business relies on.
I'm not sure about what the article is specifically referring to, but end-user programming refers to a spectrum of people that need to accomplish a programming-like task without being a professional developer.
The paper "The state of the art in end-user software engineering" discusses this spectrum from an email user to a middle school child to a scientist: https://dl.acm.org/citation.cfm?id=1922658
Also: Not discount the power of some basic training to unlock the power of DSLs.
I work a lot around RDBMS. Most people have almost null training about SQL, relational model or how databases work. They learn it in the most ad-hoc, step-by-trick-step.
Is mostly: "Hey how do that? Look! this cool trick do the work!". You see this everywhere. Sometimes people ask you this question a lot (about excel, sql, how use email, what? Word can do some mat too?) and so on.
Eventually, I sit a few minutes with them and explain the fundamentals. It change the game. A lot.
IMHO author is missing the historical perspective of DSL approach. In the time when SQL was developed, users simply needed a much higher level of tech skills than today. One older colleague told me once how he wrote his PhD thesis using ed (unix line-editor), and it was more than 100 pages work. To us that seems crazy complicated and unfriendly, editing 100s of pages line by line using obscure single letter commands, but to them it was a huge improvement compared to typing it on a typewriter - and they all learned how to use it. Similarly, analyst in the days when SQL was invented had very different expectations and ideas about what "simple to use" means. Back then you expected that you need to type your commands as text, so developing specialized tools in form of DSLs was a perfectly logical step. Today DSLs make less sense as we can build spreadsheets, wizards, visual drag&drop UIs, AIs for speech commands and so on. Users' expectations and skills are completely different now, and it's less likely that they'll invest time into learning a new language, they want buttons to click. That's why DSLs are now more a middle step that simplifies life for programmers when dealing with specialized tasks - for those areas where visual tools are too limiting, but still you need a way to abstract the complexity - just the way the SQL does it.
Our previous PM was better at SQL than most developers on my team at the time (myself included, which was a big motivation in taking the time to properly learn more advanced SQL last year).
The debugging issues which came to light as part of the development of POP-PL led to the new Racket trace debugging facilities in the form of Medic and Ripple, which I've previously commented about here, and which unfortunately barely anyone seems to know about:
I think there are good counter-arguments in these comments, but I also think DSLs should be approached with caution: creating a non-trivial DSL that is correct in a consistent and intuitive way, which provides useful feedback on syntax errors, and is documented well enough to be used by someone else, is a lot of work.
The example given, SQL, also falls flat. There’s plenty of people who are technical but not developers and use SQL for their daily lives. I know a marketing manager who is better than me at SQL.
Heck, I ask a PM at work to debug some of my complex queries and he asks me for help with sample code for a developer product.
DSLs for non-programmers are battle-proven. The author dismisses SQL with an anecdote but the simple fact of the matter is that SQL is known and used proficiently by a massive number of non-programmers to get real work done. I guarantee you there are numerous non-programmers that know SQL better than the author, but he met a few guys who aren't so good at it so he thinks nobody else is. Terrible article.
how many time did these "non-programmers" spend on learning SQL ? wouldn't have they learned basic programming skills on languages such as python or ruby during the same time ? The problem with DSL is the S, they are Specific, and trap the "non-programmers" in a non-programming culture. If they want to do something a little different than the DSL is targeted for, they are unable to.
Excel is good example, it is REAL good tool for what it is meant for, but I seen many people doing crazy thing with excel that can be done with very basic python in a dozen of basic statements. Why ? because they people never learned to do other thing than excel sheets, "When you only have a hammer everything looks like a nail".
Yes, and it's so strong it makes the topic totally moot. The author has just stated in his experience DSL:s have turned into a mess. The same could be said for any known well established technique that is not used competently. "No, all tools don't suck. But please don't use the chainsaw as a hammer"
Yep. But they're overused and underused each day... Everything can be a spreadsheet and nobody questions if there are better ways of holding that particular data
The issue that I've seen with Excel is that it's often seen as the hammer -everything is a nail- for people that are less inclined with technology / set in their way.
They will use it both as intended, a spreadsheet for calculations, usually for reports.
But also as publishing software (think InDesign) and format their spreadsheet to look like an invoice / quote, a hand made Gantt chart (yeah with arrows as delimiters, as opposed to filling the cells), etc ...
It's both a fascinating and horrific thing to behold, something that wouldn't be out of place in a H.P Lovecraft story.
We're going to try to get the Gantt folks to modernise a bit and use a dedicated tool for planning but we already know it's going to be complicated to get the senior folks to even consider it.
> But also as publishing software (think InDesign) and format their spreadsheet to look like an invoice / quote, a hand made Gantt chart (yeah with arrows as delimiter
Meh, I think I've done all these things myself at some point.
I'm very familiar with Excel, so sometimes it's easier to throw something together using a tool I know well, rather than get (or maybe buy) and learn new software.
Dan Bricklin himself said in an interview (I think this one [0]) that when they created VisiCalc (the first spreadsheet program) with Bob Frankston, they designed it to mix data and non-data. It was entirely part of their vision of a malleable information space.
Of course at some point of using it for everything any tool become the wrong tool, but most of the time users creating sophisticated layouts in a spreadsheet actually use the spreadsheet as intended in the original, powerful vision of its inventors.
"As intended" is a suggestion, not reality. Physical reality doesn't have XML tags attached to things, saying "this is a hammer, you shall use it to drive small nails, and small nails only". The same thing applies to software. There are tools more and less convenient for a task, but ultimately it's the task that matters.
Consider invoices: before computers, they were done on the ultimate universal medium - paper. Even today, they exist as documents primarily meant to be read by humans. They have some required information and vague shape they're supposed to resemble, but that's it. It's much faster to make one with Excel than with InDesign, doubly so if you already know Excel. The part that Excel doesn't support - invoice life-cycle management - can be done elsewhere (file system is damn good at it), and often isn't even a part of the task - that aspect is handled by the accountants you CC when sending out an invoice.
Excel is really a kind of smart virtual paper with affinity for grids. People use it for a lot of weird things because it usually works damn well for those.
I think DSLs are about defining a 100% textual format. And spreadsheets are GUI apps. There's a DSL for defining a cell formula but in total it's a small part of the experience of using a spreadsheet app.
> "I think DSLs are about defining a 100% textual format."
That's an unjustified qualification. I could throw together a spreadsheet format that is all text. The spreadsheet GUI then becomes a advanced text editor that, when editing that particular format, exposes advanced content-aware controls not at all unlike advanced text editors like emacs can for s-expressions.
We can bridge the gap in other ways too, for instance the '2d' racket language that lets you do control flow using a two-dimensional ascii art grid: https://docs.racket-lang.org/2d/index.html It's not hard to see how this concept could be iterated on to become something quite like a spreadsheet, and with editor support the editor/language combination would begin to look a lot like a spreadsheet too.
> I could throw together a spreadsheet format that is all text. […] The spreadsheet GUI then becomes a advanced text editor that, when editing that particular format, exposes advanced content-aware controls not at all unlike advanced text editors like emacs can for s-expressions.
(Emacs also has an even more spreadsheet-like mode; ses-mode, but that saves its data in less-than-purely-textual format. Or, at least, less textual than org-mode.)
> That's an unjustified qualification. I could throw together a spreadsheet format that is all text. The spreadsheet GUI then becomes a advanced text editor that, when editing that particular format, exposes advanced content-aware controls not at all unlike advanced text editors like emacs can for s-expressions.
If you could invent a text format that could be efficiently edited with a basic text editor then I would agree it's a DSL. But I feel like you would lose a lot by dropping a dedicated GUI, e.g.:
- horizontal scrolling of columns, adding, hiding columns
- "smart copying" a formula by scrolling down
- selection of rows/columns/cells
I don't think anybody would use such DSL using a basic editor.
Overall it's discussion about the definitions of terms, but I don't understand why people want to capture anything having some "editable format" as a "DSL" when addtional terms like "visual programming" allow more differentiation?
Let's take the Wikipedia definition, not the examples list:
> A domain-specific language (DSL) is a computer language specialized to a particular application domain.
where computer language links to a list of things that includes "programming languages", which visual programming languages are a part of.
Other sections of the article also mention some graphical examples, e.g. UML. (which isn't a programming language, but a modeling language, also used sometimes as an input to software)
Many DSL can be replaced by python modules: it is far easier to develop and document a well written python module than a good DSL Defining a syntax, documenting it, doing parsing, syntax checking, generating good error reporting on a DSL is far harder than just designing a python API.
Of course it is a programming language, but Python can absolutely be used by non-programmers. Not as cleanly as a good programmer will do of course, but I've seen many non-programmers learning it pretty quickly for basic usage (in this case the basic usage is just to use your python module which act as a "DSL")
It also has the advantages of having tons of resources online, IDE, code coloration, etc... the random "user" of your module will in general just have to search "how to do XYZ in python" and get plenty of answer.
This is actually a great idea. You get all the readability and ease of access of a DSL with a strong language backend.
In fact, some gui-heavy software have a python interface to write some automations or plug-ins to extend functionality beyond the GUI, mainly targeted at advanced users.
There's a middle ground, where most of the users you hoped would write the DSL never become comfortable enough to do so, but do become competent to read the DSL and review proposed changes.
I'm surprised no one is talking about React JSX. I'd be very interested to hear the authors opinion on it.
Unlike the author's examples, JSX isn't intended to be used by business people, but I think JSX presents some of the same problems but instead with regard to web designers. JSX is a DSL that was intended to help simplify and streamline the process of building a web app, but the leakiness of JSX as an abstraction prevents anyone less than a full front-end engineer from working effectively with React.
My point was about the tools that let people do the job they'd ask to do programmers otherwise, but it is also a nice touch. Although, I think it is not only about JSX, virtually any modern framework markup part is not easily readable by non-developer. With complexity of the clients and trends we have now it is not fixable and won't be for a long time.
How do we make automating the work of a computer more accessible?
1. Visual programming usually turns into a hot mess. It's actually as hard to learn properly as a simple programming language and once learned - usually harder to build things in. How many devs choose a visual environment when there is a genuine option?
2. "Simple" languages - BASIC, Python, Applescript, Logo. They often become complex over time unless ruthlessly maintained.
3. DSL's - probably is they tend to be, erm, domain specific and therefore you have to learn from scratch multiple times if you want to do things outside of your initial domain.
I think a hybrid of all of the above might be interesting:
1. A visual builder as "training wheels" but always ensure a perfect transform from visual to code and back again.
2. A small language with a simple syntax (something Logo or Python-like but even more carefully curated)
3. DSL-like behaviour without being a DSL. Extensibility with elegant syntactic support. Ruby aspired to this but it's Perl-isms stopped it being a good beginner's language.
Is a thing where DSLs are expected to be for non-programmers? I mean language is right there in the name. They are programming languages and you have to have programming skills to use them.
Unless by non-programmer you mean non-developer. People for which programming is secondary or tangential activity. Like scientists.
I'm just curious... the blog post has the title "DSLs for non-programmers are a hoax" and this HN post originally had the same title. At some point it got changed; I've seen this happen before. Is this HN editorial intervention, or did the OP make the change?
Any time I've encountered Cucumber-esque testing DSLs, the amount of work that was required to plumb things up to the actual test code made me want to throw up my hands and just write xUnit tests or whatever and cut that extra work out. It's very possible that I haven't worked with anybody that really understood how this is supposed to work, and with the ten thousand other things I am responsible for, I haven't had the time or inclination to take a real deep dive into it.
I mean, they want you to write "expect(some_thing).to be_true" and even more ornate stuff.
I mean, why not "expect(some_thing)" or "expect(some_thing==true)"?
Rspec, Cucumber, etc. do have matchers that are helpful and more concise than idiomatic Plain Old Ruby but some of them are so basic and yet so ornate that they really seem like some kind of a joke.
RSpec is written and read mostly by developers. Unlike those tools that business likes to use and abuse developers with extension requests just to feel more control over the situation.
Anyone who uses a DSL is a programmer. Another way to say that "DSLs for non-programmers are a hoax" is: Good DSLs make programming easy enough that people who aren't otherwise programmers can successfully program.
Jetbrains, makers of IntelliJ IDEA and PyCharm, have a DSL-creator tool called MPS [0]. It looks interesting, lets you include language and "non-textual" elements including "math notations, diagrams, and forms." I'd like to apply it in some context.
When I read Game Engine Architecture[0] I got the impression that DSLs are pretty effective in that industry. Anyone have war stories of game DSLs gone horribly wrong?
I don’t necessarily agree. I have worked with financial analysts using Excel, SQL etc. quite proficiently. In fact, I am told that familiarity with DSLs used for data analysis/manipulation tools is considered a part of the job description by many companies nowadays.
The problem with programming for non-programmers is so many programmers have told non-programmers that they can't program. Programming is far easier in the small than we'd care to acknowledge. Don't cripple what you let your users do, enable them.
DSLs made of elements of a real programming language always seemed weird to me. When I need a language for a specific domain I just make one from scratch using either S-expressions or manual string parsing (using just string split and replace functions).
I agree with most. But I find SQL to be a very bad example for a failed DSL.
On the contrary, It is a highly successful DSL. In most large companies business analysts (with no programming education) do use SQL extensively.
> The DSL fashion I oppose [...] is the fashion of self-baked business-specific tools that are intended to be used by users that are not programmers or even tech people. Those require much more effort than a regular system and are seldom to be done well. So it is better to think twice before creating your own DSL [...]
"In the end when the serious stuff comes in it was always a better choice to hire someone who really understood what he was doing." - I know this is a side point, but it makes it very hard to engage with articles when their use of gender pronouns deliberately excludes you from being 'someone who understands what they are doing'. I understand that this is often accidental, but in 2019 I'd like to think people would be more careful about excluding a marginalised group in software.
Reality check: I've worked on multiple large research teams that successfully developed DSLs still in use by domain experts -- military sigint in one case, doctors in another case, industrial engineers in another.
We developed these languages for specific applications where the domain experts needed to program extremely complex systems to perform tasks in much shorter timeframes than would be possible if programmers needed to get involved.
Why did these DSLs succeed? Funding, manpower, and experienced project managers who delivered constant access to domain experts.
Delivering a successful DSL targeted at narrow domains with non-programmer users is much more resource-intensive than normal software--it's more on the complexity level of a compiler or small OS--but that doesn't mean it can't or shouldn't be done. Plenty of systems couldn't be automated otherwise.