I dunno... the new async stuff is cool but I'm worried it's being oversold a bit...
First of all, it is by nature a very "imperative" solution... it goes (to some degree at least) counter to the whole idea that state changes need to be limited as much as possible and isolated from the bulk of the "functional" code.
Also, it seems people are using this idea to create little localized event handlers, whereas before you'd solve this in a functional approach by having a single global event handler.
Sure, having a global event handler can be derided by saying "this global handler is spaghetti code because it doesn't isolate concerns."
But in the same way, this type of channel-driven programming can be called spaghetti code because the many different goroutines can step on each other as they manipulate the DOM.
For instance, if OP creates another channel-driven system for "editing todos" he would have to do some hard thinking to make sure the "add todos" system and the "edit todos" system don't interfere with each other and cause undefined behavior.
I should point out I haven't built a system yet that is channel-based (I plan to do this soon) so my knowledge of this approach is as of yet limited and it's possible I will like channels better once I've used them more heavily.
It is interesting. In normal Js applications there is tremendous gravity towards having mutable application state because the callbacks need a handle of some sort (normally some instance of something). So I am wondering what you mean by saying that the callback pattern is more functional?
In addition to mutable application state we have another global mutable state, one that's even harder to opt out of: The Dom. Regardless wether we use channels or callbacks this is going to be a problem whenever we have more than one actor changing it, whether that is done at the same time or sequentially.
To me the place for core.async or the newish yield operator is that blocking will allow us to wire together complex event behavior in some fairly sophisticated ways. How long does it take to get a slideshow to really work well in callback land?
Or responsive autocomplete with correct lower and upper bound timeout thresholds?
So we can handle the wiring of events and when we get to the right state we do our functional application state transition.
Then as long as we are driving the Dom off of our application state then we should be good right? * waiving my hands a bunch
> what you mean by saying that the callback pattern is more functional?
I agree that the callback approach is not functional, and that the channel approach is certainly much better than this status quo approach.
I'm comparing instead against an approach where you break down all relevant events into JSON, and then push those into a global fully-functional "event handler" function that translates events (along the "world state") into a set of DOM manipulation primitives.
With that type of approach, no channels are necessary. I agree this approach has difficulties with time-sensitive operations (such as responsive autocomplete, etc) and that channels offer advantages for this.
I'm not yet sure which approach leads to the least tradeoffs or whether there is a third approach that combines the best of both worlds that I don't quite comprehend yet.
> another global mutable state, one that's even harder to opt out of: The Dom
Well, I think the success of angular.js shows that it's possible (and probably desirable) to completely isolate DOM state changes from other program code... though the OP program doesn't do this, this is clearly just a simplified example, so I want to see what is possible once someone builds a client-side framework that is built on channels...
I'm comparing instead against an approach where you break down all relevant events into JSON, and then push those into a global fully-functional "event handler" function that translates events (along the "world state") into a set of DOM manipulation primitives
How common is this approach? I'd be very interested to hear people's experiences with it. It's what we do, except we run the global event processor on a JS timer so that it works like a game loop picking items off a queue. Our DOM event handlers (the things you call addEventListener with) are all trivial stubs that simply create JSON descriptions of events as they happen and append them to that queue. We tried many different approaches before we hit on this, and it works surprisingly well in the two hardest areas: managing complexity and keeping the UI snappy.
You can also create perceived concurrency this way: you do a little work on an item in the queue and then, if it's not done, put it back at the back of the line. That's how we do the "time-sensitive operations" you mention.
I'm not sure what to call this design. I think of it as a state machine, but that's a metaphor rather than a precise description.
Are you familiar with E? It basically works like this, plus built-in, transparent promises. http://www.erights.org/elib/concurrency/event-loop.html outlines why you might settle on this model. The messages about coroutines linked at the bottom argue against things like core.async. (This all long precedes core.async, but Concurrent ML had that kind of design.)
I've only done simple things in JS and don't understand what advantage you're getting by queueing events yourself, since JS already runs an event loop. Is it that you can inspect the queue and drop/modify events superseded by subsequent ones? I tried to do that once and it didn't help, but I've gotta admit I'm the opposite of an expert on JS UIs.
I wrote a reply that is perhaps a bit too long to put here, so I put it at http://pastebin.com/4nN04Hj3 instead.
The short answers are: (1) this event loop is different from the browser's event loop because it's app-specific. Having a single interception point where you translate the browser event stream (mousedown, keypress etc.) into higher-level app-specific terms, and then write the rest of the UI in the latter, is a strategy for reducing complexity—an application of bottom-up programming to web UIs, really; and (2) emphatically yes, making the "official" event loop trivial and running all the complex stuff off your own app-specific loop allows you to control how much processing to do and when, which can be a way out of some thorny performance problems (such as sluggish scrolling) as well as an easy path to simulated concurrency.
It's interesting that you should have said "drop/modify events superseded by subsequent ones" because that's precisely what led us to this design originally—we were desperate, in fact, for a way to do that. The bottom-up programming aspect and the concurrency aspect only became clear later.
I think I read that E's promises came out of experience with UI programming too -- that they invented http://erights.org/elib/distrib/pipeline.html for the sake of sanely programming snappy UIs for Xanadu.
Sounds like I gave up too easily on queuing and pruning events, or it was the particular platform (touch tracking on Nexus 7 and first-gen iPad).
Thanks for the experience report. :-) It might help me remember this when I get back to http://wry.me/hacking/lissajous.html to finish & polish -- I let it get all cut-and-pastey in doing drag-controls for the first time.
I've now read those pages. I knew about E for its capability model but not at all for its concurrency model, which is indeed very interesting (more interesting, to me) and close to home. There's a presentation mentioned there that I found slightly clearer than the page itself; the link is dead but it can be read here:
This model deserves to better known, especially since all this stuff is in play again nowadays. Are there other implementations of it? (Node.js might be a natural platform, given its event loop and the popularity of promises there.) I also wonder what the biggest differences are between it and Erlang.
In your pipelining example (t3 := (x <- a()) <- c(y <- b())), what happens if the messages arrive out of order? (Or does the system prevent that and if so how?) I suppose if an expression can't be evaluated because a promise hasn't resolved yet, it could just go back to the end of the queue and eventually everything will bubble up?
Also: that Xanadu? Wow, no idea.
Edit: I forgot another thing I wanted to include in this omnibus comment. The paper "On the Development of Reactive Systems" that's mentioned at your first link and which can be found at [1], starts off with a very interesting and exciting distinction between "transformational" systems (which take inputs and transform them into outputs) and "reactive" systems that "are repeatedly prompted by the outside world and [whose] role is to continuously respond to external inputs. A reactive system, in general, does not compute or perform a function, but is supposed to maintain a certain ongoing relationship, so to speak, with its environment". I think this is extraordinarily lucid and that the authors are right to talk about concurrency—and probably complexity in general—in the latter sort of system as being a different challenge than in the former. Unfortunately, the paper quickly sinks into a pit of software process goo and never returns. A little Googling gives me the impression that important technical work did come out of it, though, and if anyone knows what the high points are I would like to see them.
The VatTP protocol is meant to guarantee 'just enough' ordering as at http://www.erights.org/elib/concurrency/partial-order.html -- there was a talk about VatTP, with slides, but I haven't seen a proper write-up and never learned it.
About Erlang, besides capabilities and mutable data I think the biggest difference comes from blocking receive. This gives it more of a CSP flavor. My own experience is limited -- my exposure to Erlang was mostly helping other hackers. (I also made an Erlang-influenced Scheme dialect I was unsatisfied with for boring reasons, but it was kind of in between: immutable data, channels as capabilities, synchronous messages.)
As far as I understand CSP is not a coroutine model - though you can express it. Hoare covers this in his '78 paper. As far as delving into its utility for UI, it's worth looking at Pike's work and the Concurrent ML eXene work.
Yes, I've read the Pike and CML, they're good work. Haven't got around to Hoare 1978, and I don't see it online, but as I understand it, threads = coroutines + interleaving; adding interleaving won't reduce the problems in reasoning about code. If the distinction is that processes don't share state, I don't think that works, since you can implement mutable cells using channels and processes. (Obviously it's less of a problem than, say, pthreads, if that's the only mutable shared state.)
Well a really interesting thing is that the channels approach helped me "discover" the queue approach that you are mentioning. As I worked on the channel examples I kept seeing the equivalent Js patterns that would make a more functional approach possible.
The decoupled queue approach seems to be rare in practice and obviously superior. Wish I had thought about it long ago. I just followed the examples available to me.
I really believe the channel approach lifts this notion (values on queues) to the forefront with the addition of blocking sequential semantics.
Yeah my examples were a bit simplified, but I did try to make the DOM state a function of the application state. There will be more examples coming in the near future :)
> How long does it take to get a slideshow to really work well in callback land? Or responsive autocomplete with correct lower and upper bound timeout thresholds?
Those are not hard problems, really. Doesn't core.async solve bigger issues?
Wether they are hard or not. They seem to confound developers as the edge cases of shared mutable state stack up. Core.async is offering another way to express solutions to complex behavior and is certainly worth investigating.
I would say that is the bigger issue 'expression'. I have to say though I am not advocating the creation of huge systems this way but I am very interested in this as a different approach and I am going to see how far it takes me.
Nice to see that people sometimes asking "what for" question.
I think it is what could be called "Haskell syndrome" when people tend to write abstract things in "very fancy way" without asking that simple question.
There are tons of blog posts like "look, I could write this abstraction in Haskell this way using this kind of monads!". Well, you can, very clever, the question is - what are the chances to encounter this abstraction in real-world problems? Isn't there a straightforward but less fancy solution?
Should I really have "all this fancy stuff" to process onClick events which can be done in 3 lines of plain ugly JS?)
Should I really have "all this fancy stuff" to process onClick events which can be done in 3 lines of plain ugly JS?)
"All this fancy stuff" is just an exploration of what's possible. The verbosity can (and will) be abstracted away via Clojure's excellent tools of abstraction (higher-order functions and macros). Your "3 lines of plain ugly JS" aren't going to scale because they rely on mutation. Mutation is deceptively quick and easy when you're just starting out. However, once your program grows large it turns into a sprawling mess of complexity.
Haha, when you say that, David, I feel like Einstein just jumped into a physics forum and said "there's no hard thinking required to understand General Relativity."
I've been slowly migrating my JavaScript code to clojurescript and I haven't wanted to touch JS since. Apart from requiring a build step to generate JS, clijurescript is the browser language I always wanted. JS seems like such a hack.
Have you settled on a good workflow for ClojureScript projects? cljsbuild at least lets you amortize JVM startup time with the auto mode, but the Write-Compile-Test loop (as Joel Spolsky calls it) still feels a little more sluggish than what I'm used to in other contexts.
They need more work but I use browser REPL or Bodil Stokke's Node REPL http://github.com/bodil/cljs-noderepl if I need to do an intensive session of interactive coding without refreshing or waiting for recompiles. I contribute a bit to core.async now and it's definitely a goal to make sure that core.async works transparently whether you're coding against the browser or Node.
You get CLJS eval in Light Table :) Open a cljs file, open a browser, point it to a page that has the initial output of a leincljsbuild run, and press cmd+enter (or ctrl+enter) in your file.
> Clojurescript is the browser language I always
> wanted. JS seems like such a hack.
You're getting downvoted for reading into someone's enthusiastic personal endorsement and then ending it with the tiring "Tell me something good about <thing I refuse to read or understand on my own time but I'll partake in discussion anyways>...".
- whoa, this is cool!
- this is harder to read than the callback approach right now
Makes me want to dig deeper, fully grok and reevaluate whether or not I'd really prefer this approach to callbacks. I have similar mixed feelings to using twisted vs tornado in python.
There's no doubt channel support in clojure is awesome, just a personal matter of whether I'll end up preferring this paradigm for UI development.
I would liken it to the Game of Go (pun intended). You have a very small number of elements: chan, go, >!, <!, alts!. The learning curve is a bit steep because you have invest some time unlearning patterns you've picked up from traditional UI programming. But once you cross over the hump and you realize what is possible with just these few tools a whole vista of possibilities opens up. I'm sure the fans of go-lang would agree.
Another analogy - Alan Kay compares software engineering to architecture and how so much engineering is brute force (think pyramids). core.async (really CSP) is like an arch (think gothic cathedral). Common UI patterns no longer seem arduous - and UI patterns which once seemed far out reach can be accomplished with a reasonable amount of engineering effort.
Thanks - I like those analogies. I want to invest some time to see if the unlearning works for me. Most times when converting to a higher level way of doing something it eventually does.
Also - thanks for all the awesome core.async gists - I feel like I want to take a geek sabbatical just so I can study them :)
> this is harder to read than the callback approach right now
Why do you think this is (I'm asking honestly, not trying to be snarky). Is it because you are not familiar with reading Lisp? The notation used to access channels is confusing? Unfamiliarity with the go-block concept?
It took me a while to fully internalize the combination of syntax and power of go blocks, but now that I have reading core.async code (I have not yet had a need to actually use them, and no time to port an existing codebase) is much easier.
I question whether the statement that callbacks are easier to read and understand is merely due to being accustomed to them. Core.async represents a radical departure from existing designs, and that can be jarring. Does this mean better syntactic sugar won't be created as developers gain experience? Or course not.
Yeah, I think it's just unfamiliarity with this approach + deep familiarity with the callback approach, and I'm merely reporting my experience and not making a judgement on the approach in general. Sometimes when I see a callback it's a helpful signal that something asynchronous is happening - it's good to know. That said, with some further accustoming to the channel approach, it will probably both read naturally and be apparent where the async stuff is happening.
That's the plan - as you can see here https://github.com/clojure/core.async/blob/master/src/main/c... we pick the fastest mechanism available for scheduling tasks depending on the environment. So that's MessageChannel for modern browsers, setImmediate for Node & IE 10, and setTimeout for everyone else.
At least in the past that has been very problematic. Perhaps it will change, but I would very much like to see Clojure become a new ClojureScript target.
Because they do exist, and I think it is important to make people aware Language != Implementation, as young developers seem not to know anything about compiler implementation nowadays.
Maybe my country of origin makes me have a bit of D. Quixote, who knows.
Because you want to use a library that was written for Node and not speed time finding (or writing) something similar for the JVM? If you know Node better than the JVM, Clojurescript on Node is very likely more valuable to you when starting out than Clojure on the JVM.
Just because a library exists for the JVM or CLR doesn't mean a person familiar with Node knows that it exists or the advantages/disadvantages of the 15 libraries that will show up for any given thing they are trying to do. The libraries, profiling tools, performance tuning details. These are the things that make up a platform. Knowing the syntax of Clojure and nothing about the JVM ecosystem isn't going to help you ship any faster. Should a developer in this situation bite the bullet and learn a better supported platform? Maybe. Does everyone have time to do so? Probably not.
First of all, it is by nature a very "imperative" solution... it goes (to some degree at least) counter to the whole idea that state changes need to be limited as much as possible and isolated from the bulk of the "functional" code.
Also, it seems people are using this idea to create little localized event handlers, whereas before you'd solve this in a functional approach by having a single global event handler.
Sure, having a global event handler can be derided by saying "this global handler is spaghetti code because it doesn't isolate concerns."
But in the same way, this type of channel-driven programming can be called spaghetti code because the many different goroutines can step on each other as they manipulate the DOM.
For instance, if OP creates another channel-driven system for "editing todos" he would have to do some hard thinking to make sure the "add todos" system and the "edit todos" system don't interfere with each other and cause undefined behavior.
I should point out I haven't built a system yet that is channel-based (I plan to do this soon) so my knowledge of this approach is as of yet limited and it's possible I will like channels better once I've used them more heavily.