I just realized something: The time investment in learning a programming language is so large, that when faced with the challenge of writing something that your language is not suited for, you set about in an effort to make the language you already know fit what you want to do, rather than learning a new language.
This leads to things like desktop apps in node.js (or server apps in node.js, amirite), or high-performance apps in Python/Ruby, or mobile apps in PHP. Given this trend, a language that is very easy to learn/teach will soon take over everything, and, if Pyret is that language, brace yourselves for the oncoming "put Pyret in all the things" revolution.
Of course you can butcher the language in all uses-cases it's not suited for but hopefully you realize that sooner than later. A complex language(e.g c++) would put off most of the students(unless they are CS students). It seems the language is heavely trading large scale development features for its specific use case(see the comment about unit testing below) so hopefully it will become "impossible" to misuse it.
This reminds me of observations on the rise of nationalism.
Once newspapers became large scale phenomena and states began building large bureaucracies, the reality that most people in most areas will only learn one (natural) language became a very unifying (and dividing) dynamic.
Computer science education idealists imagine a world built by the best students, those who finish the entire course of study they set out. It indeed seems logic that the world is shaped by the great mass of people who at the minimum useful level.
No. I think the best illustration is the continuation-based Web programming work we did several years ago: http://cs.brown.edu/~sk/Publications/Papers/Published/khmgpf... If you don't have the right underlying language (in this case, with continuations), you _can't_ have a similar API. Similarly, the reactive APIs we had in Flapjax [http://cs.brown.edu/~sk/Publications/Papers/Published/mgbcgb...] enabled entirely new kinds of primitive “library” operations; see, for instance, the drag-and-drop example in that paper.
The reason why some languages are more suited for some things than others is due to paradigms like callbacks in JS that were fit to certain applications (the web in this case). Python is incorporating newer "paradigms" like asyncio into its language, but it feels convoluted at this point. The preference of certain semantics for different application types brings forth the same problem (convolution) if we tried to implement a single API that fits all purposes.
You are being rhetorical, but over its long history dialects of BASIC have been used to write a huge amount of software. It was the first generally accessible interpreted language, and its influence on programming today is no less than Lisp or Smalltalk. It was a major driver of early personal computing, got Microsoft rolling, and in the process produced a huge crop of programmers that allowed the web to be commercialized and become what it is today.
You make it sound like it was an ill-conceived goal to make a simple and understandable language, but the reason most of us aren't writing our CRUD apps in BASIC isn't that we all moved to APL.
Certainly simplicity and understandability are very important language traits, but if you only concentrate on them, your language won't fit certain tasks well and won't get used for them.
Programming languages as Basic or Pyret certainly have their place and can empower people that didn't program before, but they won't take over the world.
i am eagerly awaiting a good linux command line implementation, at which point i will embrace the "put pyret in all the things" revolution. it's a beautifully designed language.
Thanks. This is where our “education first” emphasis comes in. I'd like that too, but it's just not our most critical need. Since it can be run on top of Node from a shell, what it mainly lacks is a REPL. One of our students took a shot at that and had half an implementation, but didn't finish it. So for someone interested in hacking on that, there is a starting point, and a REPL atop Node would a big win. I'd use it every day (-:.
it's not the lack of a repl I mind so much as the startup time. I'd love to be able to write small command line apps in pyret but the execution delay is painful. (that said, I fully appreciate the focus on education; there are lots of other scripting languages I can use!)
I agree! We're acutely conscious of it. This is precisely where, as you note, the education focus puts our resources elsewhere. Once those things settle down, this is something we're going to want very much even for ourselves.
There's no reason you can't write high-performance apps in Python, unless you have decided to tie your hand behind your back and not use any of the optimization strategies which have been developed in the decades that Python has existed.
If you mean something like programming without using the heap, most high-level languages cannot do that anyway.
Optimizing Python generally means using numpy, cython, or the C FFI. Or numba. Or theano. Or pypy. Unless you farm out what you're doing to another code generator or compiled library, Python can't be made fast. Try making matrix multiplication fast without passing through the FFI at all in cpython.
So, a question: last time I talked to one of the Brown PLT profs making this on HN (GH contributor log makes me think ... shriram?) this worked well in Chrome, but not well or not performant enough in Firefox. Has that changed?
An admiring comment: the fact you encoded into the language unit tests close to functions, which is a style choice for some in Python but not mandatory, was very cool to me.
An admiring comment: you talk about multimedia-centric must be awesome. I am language major and failed programmer many times over. I started with C++ on Solaries boxes compiling with g++ over SSH; I was one of the few who started using Linux, not Unix mind you, in Poli Sci school so it was not so bad for me as everyone who would use VS and msvcc, forget to check g++ builds, and lose points (I still did miserably). When I saw samples where you have stdlib tools for integrating into things like Google Docs and Dropbox, I thought wow; these kids are going to learn to make stuff so early, and that is so liberating.
And finally, an obnoxious comment: as a Lisp wheeny, I wish Racket and its toolkit was fast enough, as one of the few who bought Conrad's book and never got around to really amazing Scheme usage.
Still, this is wonderful work. The idea of people building such full environments in JS, stacked on top of transpilers and compilers, is amazing to me as a novice. Not that it's possible, but that it works at all. Every time Pyret comes up, I smile.
Aaarr, keep hacking matey!
UPDATE: Add my previous Q&A and with shriram. You Brown PLT and Racket people are so approachable and smart, damn you !(He says ironically while shaking his fist.)
Firefox performs better than it did, because the whole language has gradually increased performance, but still the worst of the major browsers. It's worth writing down a bit about that here, just for others to see and comment on.
We have some ideas that we suspect will mitigate the main issue in Firefox, which is that Firefox stack frames seem to be very large for compiled Pyret functions. This matters because Pyret's execution control works by (a) figuring out how many "typical-sized" stack frames fit before a stack overflow; this is a little tricky [1], (b) leaving breadcrumbs of metadata on each stack frame, and collecting them on an exception thrown when the limit is reached, restarting after yielding control to the browser's event loop. The issue is that the "typical" frame size for a Pyret function in Firefox means we only get a few hundred frames, and most idiomatic Pyret is implemented recursively, hence lots of pausing and collecting the stack.
It's clear that good support for safe-for-space tail calls is a simple way to mitigate a lot of the issue. We have a version of safe-for-space tail calls that avoids using unbounded space, but doesn't help (much) with speed.
Upcoming safe-for-space tail calls in JS provides one avenue for this that we'll be trying to take advantage of.
Actually compiling instances of Pyret tail recursion to JS loops is something we want to get working. It's a mildly nontrivial engineering challenge in the presence of dynamic annotation checks for return values (which muddy the definition of "tail call", especially since annotations can contain refinement predicates -- more function calls!), the desire to report faithful error information to students, and wanting to yield to the event loop often enough to not lock the page and get "busy script" errors. It's not extremely difficult, there's just enough tricky bits that it's on a branch right now and we're not totally satisfied with it yet.
Since we've largely focused on design of language features rather than performance for a long time now, there's somewhat of a backlog of these kinds of straightforward performance ideas that aren't in the system yet.
So, novice question: will Web Assembly help TCO? I know nothing of it really, but it seems like a long-term goal, not shortterm for the WA design team.
As an aside, I am a weak programmer, but I love what you guys have done with Racket, especially with incremental language building with #lang directives. I know you could not do that so seamlessly in Pyret, but I love your progessive lang building, progressive type addition. You get the practicalities of pedagogy in language learning by breaking it down into pieces; I wish more people got that and could have succeeded into steering me into double majoring in Arabic and CS. Ironically I was so weak at it after two semesters of C++ only for major intro courses, the enticement of crazy USG machine translation gigs my profs promised me scared me away with my impostor syndrome.
So why say this? How can people like me help with Pyret? I think its web browser JS transpilation are very interesting, and in this age of data visualization and rich multimedia interactivity, I think these tools are the future of language learning, university or not, whether people think it.
I see you guys went so deep with Racket for building blocks of abstract language building blocks, but the Racket people involved in Pyret are sweetening the deal with the notion of tools for practical incentivization with the wep API and media integration.
So, yeah ... sorry to brain dump there. Really enjoyed this post and the work of your lot over there.
Good question re. Web Assembly. The problem is we need much more than tail calls. Just as important is deep stacks, to enable a _functional_ style of “functional programming”. And that has definitely been a headache. Our current (clever) implementation strategy actually gives us tail calls as a consequence of the deep stacks, using a variant of Henry Baker's "Cheney on the MTA" (for those old enough to remember that) combined with the "stack reconstitution" technique we published in ICFP a decade ago [http://cs.brown.edu/~sk/Publications/Papers/Published/pcmkf-...] and a few other things. But tail calls by themselves wouldn't be enough.
What _could_ we use help with? Obviously, we could use infrastructure help or porting to “platforms” (e.g., a REPL on top of Node), but those are huge commitments and require a lot of special knowledge. But I think a person with only a little time can still help.
Right now, I feel the language is still a bit impoverished. We want to make it really easy to get working with data, and that's still too hard. But pretty soon we're going to be putting out better support for tabular data manipulation. Actually use it to do some fun things with data; share your successes (when things work) and your experiences (when they don't)!
How strongly is Pyret committed to the JavaScript runtime? I read jpolitz's recent comment about the contortions that you have to go through to support recursion while not overflowing the stack or blocking the UI thread, and I wonder if it would really be so bad to require people to install a programming environment on their computers. How common is it really for students to be limited to what they can run in a browser? Anyway, have you considered a non-JS runtime of some kind?
We started out as a Racket `#lang`. That is in principle always an option. However: the in-browser support is non-negotiable. We have many users in schools, which have very locked-down machines. Many of them cannot install new software. This is a real obstacle for many curricula, and our ability to run just off the browser has been a huge prerequisite for adoption of Bootstrap's [http://www.bootstrapworld.org/] curricula.
Interesting, the language re-purposes the 'where' clause to declare unit tests within function scope. In general, I like the idea of putting testing front and center for a language targeted at teaching programming.
Conversely this is what I like least about it. It works nicely for small functions, and even reads like documentation.
But past that it will balloon and just bloat the program you're trying to understand. Tests are better left in their own file. That's not to mention test infrastructure (mocks etc) should you need it.
To add to what you said, this is exactly the kind of situation where we design features that _may_ not scale well, but are remarkably useful in teaching contexts.
Having test cases inline with code is massively useful:
1. For students not used to thinking about their program being split over several files. If tests must be in their own file, then understanding multi-file programs (modules, essentially), becomes a curricular dependency of writing tests. That is a _lot_ of friction early on – think about an intro student who has a fuzzy notion of a "working directory" or IDE "workspace" in general. Even if the eventual goal is to address these concepts, they shouldn't have to come first just to write a unit test.
2. When live coding in front of a class. Having tests in a separate file is a huge pain because you pay either (a) the cognitive overhead of switching buffers, or (b) precious projector real-estate on having both open. Since live-coding programs that are less than 100 lines long is a major use case for Pyret, this is a big win.
As programs scale up, for example in Pyret's compiler or in larger assignments, we end up with a blend of test cases in inline where: blocks, with the majority in separate files in standalone check: blocks. So at scale, things turn into more traditional-looking separate suites.
In my Elixir one-page code experiments, I put the unit test in the same file after the code under test. It makes sure the test only runs when the file is loaded a certain way: https://github.com/pmarreck/elixir-snippets/
For writing production software, I have the same kind of concerns. Plus, what if you want do something more than simple unit tests (property based testing, concolic testing, etc) or if someone comes up with a better unit testing framework? You'll still have to setup those tests in a separate file, and then the in-line tests are just some unused feature of the language.
However, for educational purposes, I think it's actually a compelling idea. My perspective is that while unit tests are a great idea, each unit test only provides marginal value. Thus, unit tests need to be as easy and natural to write as possible or developers won't do it.
Habits, like writing code that doesn't need much mocking, definitely makes TDD easier, but that often comes after habituating oneself to TDD. For students, I think getting them as early as possible to write unit tests would be a big win. Putting the tests right there in the code with no need to set up test infrastructure or learn a separate test framework will be helpful.
There's absolutely nothing in Pyret forcing you to write `where` blocks with immediate tests. You can also put your tests in a `check` block, which can float anywhere, and in another module.
What we find valuable is to write a _small_ number of _illustrative_ examples, which we call the “sweep”, that summarize the main behavior of the function. Think of it as essential, runnable documentation. When you come to a new codebase (including your own, six months later!), these help you quickly page back in what the function was supposed to do.
It seems like the obvious thing is to allow both tests in the source file and separate test file.
I like the idea of short functions with doctest-like tests, but written with the real language, syntax-highlighted, etc. In addition to education, it's good for quick prototyping. It's also nice to see at a glance which functions don't have tests!
But I can also see it getting a bit unwieldy for complicated stuff, so in that case you can put it in another file, perhaps with a pointer.
I'm guessing you can't stop people from writing tests in separate files, so I suppose Pyret allows that.
I guess you could write a unit test framework in Python that does this with a tiny amount of metaprogramming.
In D there is even a keyword for that. It allows to embed the tests into the documentation as examples. This has the nice side effect that your examples are actually tested. https://dlang.org/spec/unittest.html
The compiler is definitely the main application that's been written. We're not aware of any standalone Pyret applications on the order of more than a few KLOC, but those are quite different in flavor, like course infrastructure.
The next largest chunk of Pyret code out there is assignments for various courses and curricula, which are our major users and macro-scale test cases. In fact, this is a useful perspective on what we get feedback from: a broad and shallow set of use cases in education. This leads us to focus on the first-time use cases, especially on the first time seeing an error message, the first time writing a test, the English vocabulary that lines up with explaining code to a number of different audiences (especially at different age levels), and the snags and inconsistencies in syntax in even extremely short programs.
Of course, it's been _used_ by plenty of students and their teachers, who have written now thousands of their own interpreters and video games and graph algorithms and image-generation programs and type-checkers and data structures in Pyret.
Finally, the design of Pyret is intentionally conservative. Pyret is not a research experiment in some particular new and untested language feature; in fact, such features are almost by definition kept out of the language. Therefore, we believe we're building on a long tradition of crisp, clear language definitions inspired by the lambda-calculus, with little by way of weirdness or frippery.
I'm not sure I agree with the premise that a beginning programming language needs to be all that accessible or easy.
My boss (I'm a developer on a Data Science team. And my boss doesn't suck as a developer. But I do the heavy lifting for the team) just put his 6 year old girl in a google code camp for young girls. They started out with assembly, for crying out loud.
That young girl is as engaged and excited as you could ever expect.
I think that we might find over time that involvement in the nerdy things of life really comes down to environment more than it does aptitude.
Could be entirely wrong. But, you know, I'd like to see a theory that was testable before we toss that out completely.
I'm really delighted to hear your boss's daughter is having so much fun. That's a great outcome.
I don't expect everyone using Pyret to turn into a professional programmer. In fact, I don't want everyone to do so, because the world also needs electrical engineers and materials scientists and poets and painters and plumbers and everything else. So I'm not out to create a planet full of nothing but nerds (in your sense).
But I do think computing is a fundamentally different way of approaching problems. It is not universal, and even where it applies, it is not the only approach. But it's a powerful and new one. I still think nobody has put it better than Abelson and Sussman, who called this “procedural epistemology”. I would like as many people as possible to have access to this epistemology.
Bringing computing to a wide population requires designing something that works for them. I don't claim we have the perfect answer; maybe assembly would work well too, though I have serious doubts (but hey, there's an easy way to prove me wrong: I'm not standing in anyone's way…).
Over a decade ago, long before “big data” was a thing, I created a course called “Introduction to Computing for Humanities and Social Sciences” that was essentially an intro to big data. It's been a very successful course at Brown, and part of its successes, I believe, was my stubborn adherence to teaching the first month entirely in Excel. That attracted, retained, and empowered a whole group of students who by their own admission did not dream of taking computing.
So, the tool does make a difference. Pyret is a tool to tackle our goals. We as a discipline are in the midst of a huge explosion of computing education interest in many countries. A lot of these are _losing track_ of who they are for and what it takes to get there [http://cacm.acm.org/blogs/blog-cacm/200936-qualify-your-quan...]. That's a problem we're deep in the midst of trying to address, and Pyret is central to how we are going about it. Others who think they can get to “all” [modulo the above article] can and will take other bets. It's all good so long as they are actually clear about their goals and then honestly evaluate them later.
> I'm not sure I agree with the premise that a beginning programming language needs to be all that accessible or easy.
It needs to be accessible with the teaching resources (whatever combination of teachers/tutors/mentors, books, or what have you) with which it is exposed to neophytes.
Language features can make a big difference in the ease of achieving this, but the right teachers/materials can achieve it with any language.
This leads to things like desktop apps in node.js (or server apps in node.js, amirite), or high-performance apps in Python/Ruby, or mobile apps in PHP. Given this trend, a language that is very easy to learn/teach will soon take over everything, and, if Pyret is that language, brace yourselves for the oncoming "put Pyret in all the things" revolution.