> There is no way for your operating system to know exactly how much stack space a thread will need so it allocates an amount on the order of around a megabyte. You only have around a bakers dozen gigabytes of RAM, so you can only make give or take 10,000 threads.
That's not true at all. The megabyte(s) of stack space is virtual, spawning a thread only requires a few kilobytes of memory initially, and is a lot cheaper than many realize.
The explanation in the video is good, here is a transcription for convenience (emphasis added):
> OpenJDK implements threads as thin wrappers around the OS thread [...] OS threads are expensive, by which I mean that we cannot have too many of them. Maybe a few thousand active ones, maybe ten thousand active ones, but that's about it. And the reason for that is that the OS doesn't know the various languages, and the most costly component of a thread is its stack, and the operating system doesn't know how the various languages it might support makes use of the stack. So it must address the worst case and allocate a very large stack. It's usually on a megabyte scale. And even though we use virtual memory and not all of it is committed to physical memory, once we touch a page it becomes committed – it can't be uncommitted again because the OS doesn't know how the stack is used. And in any event, all the operations are done at a page granularity which is 4k, it's also not that small.
for java, i wonder if you could make the stack-size smaller to increase the number of threads possible (as long as you know your application can handle this smaller sized stack - aka no recursion etc).
With Fibers you can implement user-space switching between virtual threads. For example, if a thread is going into a top-level select, switch it to a small stack putting the original large stack into a pool. On wake resume switch back to a larger stack. That’s basically what Project Loom is doing I suspect.
Stacks are allocated statically upfront. However, you can change the size the kernel allocates, you can point them to use a custom memory space instead of the default one and you can change the stack you're running on. Also, like all memory pages, stack space isn't actually allocated until you modify it so if your stack depth doesn't get crazy large/you're not storing large objects on the stack, it's possible your 16 MB stack is only using 1 MB or so of real RAM.
In Java, it defaults to 1MB on platforms other than Windows.
I've seen people OOM themselves by setting the default thread stack size massively high - they had likely gotten confused with the args (they'd used -Xss when I think they meant -Xms) and given their threads 1GiB of stack each.
I can't speak to how Java manages stack size between itself and the OS though, I'm sure there's deep magic there.
Fun fact, you can declare a stack size at compile time (something like `cl ... -stack:0x5000000,0x5000000 ...`) which let you create insane stack size so that you can fit your all your software in the stack. Useful when you don't want to had an allocator when making tiny tiny executable files :D
I think they meant that the main cost of a new thread that you'd like to create is the time it takes to spawn (i.e., the time it takes to set up the new thread and get it ready to run) and NOT the memory that gets allocated for it, nor the operating system overhead of keeping it around afterwards, etc, etc.
(I'm not agreeing or disagreeing; I'm only trying to helpfully clarify)
People sometimes respond with some puzzlement to InterruptedException, unsure what they should do. All it means is that some other thread has requested that this thread stop what it's doing. So do what you think your code should do in that situation. E.g. in the case of `say`, the most reasonable thing is to simply return.
You shouldn't normally call `Thread.currentThread().interrupt()` inside a `catch (InterruptedException e)` unless, perhaps, you don't throw anything. That would prevent any blocking operation inside cleanup code from working. As a general rule, a thread's interrupt status is set or InterruptedException (or some other exception) is thrown, but not both. That is how JDK methods also do it. If they throw — they clear the interrupt status.
While there are use cases where lightweight threads are a benefit, I think a web crawler is pretty much the ideal case for async/await, given you are almost exclusively waiting. Although I must admit I don't really understand this example, why is the fetching being parallellized, like you typically want to add sleeps, even with a single thread you've got yourself a gatling gun that can DoS a website if you aren't careful.
There's also some optimizations that could be made. Instead of having
Go does have a concurrent map in the standard library, but the documentation recommends not using it unless you have specific access patterns that it's optimized for:
You lose type safety when using it. I'm guessing with generics in 1.18 now, we'll see an updated generic version of sync.Map whenever a bunch of new collection types are added to the standard library that take advantage of generics.
FWIW while map[T]interface{} is somewhat more convenient to use, for efficiency reasons you should prefer map[T]struct{}: interface{} being a fat pointer is takes 16 bytes * the capacity of the map/set. struct{} is a zst, so takes 0.
It also signals much more clearly that it's a map to nothing, which is a set, a map[T]interface{} could be an actual map to a bunch of random things.
Not that I have seen. map[T]struct{} is more common, with if _,ok := m[x]; ok {} used for testing if a key is in the map. The space savings really add up with large sets.
At least at Google, map[T]bool is more common, by an order of magnitude. Probably because map[T]bool is recommended by our official style guide, which tends to favor readability over a negligible performance difference.
That is largely a distinction without a difference, the main gist of my comment was that using an interface{} value, while convenient in some ways, is way costlier than necessary (at least in memory) without any real advantage.
And I much appreciate your feedback :) Whenever I try to think of what was the best practice idiom, I always mix up map[T]struct{} and map[T]interface{}, due to the curly brackets at the end.
As you can tell, it's been a while since I've written Go.
In Go, while it is possible to use a concurrent hash map, it's generally not recommended. In most cases, you should use a channel.
In one of my first Go programs, I used mutexes to avoid concurrent writes in a critical part. Then I moved the writing process into a goroutine that receives its input through a channel of size 1. In my case, the later code was easier to maintain and more performant than the former, mutex-based, code.
In Go, the select branch to take is non-deterministically chosen from the set of enabled branches.
For the examples I don't think it makes a difference, but in general you cannot assume that a select attempts to choose branches from top to bottom.
I wonder if that's on purpose, to find bugs earlier. I know that the implementation of map in Go will intentionally shuffle the keys when you try to iterate over them, so that you never accidentally rely on insertion order.
It's very much on purpose, just like default maps being unordered. People should not rely on the ordering as it breaks the semantics of these features according to Go. It fits the Go philosophy well.
I've read somewhere (possibly when Go came out) that it's on purpose. For fun, Alan Cox once made a post on Google+ (RIP) showing a crappy random number generator based on Go's select.
Loom's virtual thread are much more similar to GHC's threads than to go routines. And GHC Haskell is also able to spawn millions of threads, trivially.
> In Go there is less noise, but also there no way to interrupt Go's time.Sleep.
You can do so in one of two ways using the context API (https://go.dev/blog/context) while keeping the code synchronous and idiomatic:
package main
import (
"context"
"fmt"
"time"
)
func say(ctx context.Context, s string) {
for i := 0; i < 5; i++ {
select {
case <-time.After(100 * time.Millisecond):
fmt.Println(s)
case <-ctx.Done():
return // Operation canceled or timed out.
}
}
}
func main() {
ctx := context.Background() // Use context cancellation semantics.
go say(ctx, "world")
say(ctx, "hello")
}
If the code is reasonably performance sensitive AND there's a high degree of likelihood the context is canceled before the operation completes, you can use a time.Timer similar to what follows:
for i := 0; i < 5; i++ {
func(i int) { // Wrap in closure expressly to support defer statement.
t := time.NewTimer(100 * time.Millisecond)
defer t.Stop()
select {
case <-t.C:
fmt.Println(s)
case <-ctx.Done():
return // Operation canceled or timed out.
}
}(i)
}
The examples translate Go idioms directly to Java code rather than using Java idioms; you wouldn't normally write Java like Go any more than you'd write Go like Java.
They also make use of constructs that are overkill for such simple examples. For example, the first example could be written as:
which would also replicate same concurrency bug as in the original Go program (the Java example, as written, does not have that bug, but neither would
var t = Thread.startVirtualThread(() -> say("world"));
say("hello");
t.join();
).
Go makes certain concurrency bugs easy to introduce and hard to find, while Java helps avoid them, but that is also not easy to see from these particular examples.
Is there an idiom in Loom for cancellation propagation and deadlines? I can't find anything in https://download.java.net/java/early_access/loom/docs/api/ja... about it. Do we need to run a second virtual thread that waits on the cancellation condition and interrupts, and then handle that exception gracefully in the interrupted virtual thread?
After mulling between Elixir vs Go to have a tool in the toolbelt covering the use-case of concurrent server-side apps, I picked a book on Go and went through the first chapter.
Disenchanted to hear (some) concurrency bugs easy to introduce and hard to find in Go. Could someone mention how does this compare to Elixir?
These cases work flawlessly in Go, in my experience:
* Batch 10 tasks and run them concurrently (using WaitGroups).
* Locking resources while reading and writing with a mutex like sync.RWMutex, using .Lock() and .Unlock(), often together with defer.
* Locking resources while only reading with a sync.RWMutex, using .RLock() and .RUnlock(), often together with defer.
* Running code "in the background" with goroutines, ie. go func() { <code goes here> }().
* Detecting race conditions by building with go build -race and then running the program.
However, when using channels, select, closing channels, waiting on channels in a for loop etc, things can easily become more subtle.
On a general basis, I disagree that concurrency bugs are easy to introduce and hard to find in Go. However, pure functional programming languages like Haskell, with little state, will always have the upper hand when it comes to concurrency.
I would have used .startVirtualThread more if I wasn't under the impression that I needed to do Thread.currentThread().interrupt() when catching and re-throwing as a RuntimeException.
So I was choosing between the noise of that in the examples and the noise of the executor + submit with a lambda that returned null.
Of course, you have to consider whether you SHOULD escape it.
I do believe you probably have more control over Java's threads, making them a bit more verbose. But, both Java itself - as you mention - and languages built on top of the JVM have ways to make it less verbose.
Well, that's because goroutines and channels are something, that's first class in go and has a special syntax.
While in Java, it tries to mostly keep the existing Java syntax.
I guess people that write Java are used to a lot of "lasagna" all-is-object boilerplate and ignore it. The same way go people are used to ignore the `err != nil` lines everywhere.
I developed and maintained Java software professionally for over 20 years, since 1.0, but now I work in Go. There is no way I’d willingly go back to Java, and these examples really underscore some of the reasons.
I do however wonder, now, what I’m missing out on with other languages, particularly swift and rust.
Swift has great support for enums (and ADT in general). That's about the only thing i miss when i switch from swift to go.
Generics and protocol oriented programming is nice, but the ergonomics makes it still not that enjoyable (you start banging your head against walls pretty quickly beyond the most simple examples).
Rust has a great, Haskell-inspired way to create algebraic data structures (with the enum keyword). These same structures can be very nicely destructured and pattern matched on.
Judging by the examples, when it comes to code that deals with concurrency both Go and Java are difficult to maintain. Plenty of times I have dealt with "clever code" (written by others and by my past self) that works 99% of the time.
Go solution to TreeWalk problem is wrong, or, rather, not real-world, because it has a memory leak. You need to make sure all goroutines finish, and not stuck on reading/writing to a channel.
In Kotlin coroutines you can .cancel() a coroutine, but Go doesn't allow to cancel goroutines.
time.Sleep(100 * time.Millisecond) could also be: <-time.After(100 * time.Millisecond) so that an imaginary timeout can be easily handled with a select.
Be careful with time.After, if you select on time.After and another channel c, and c breaks the select, then the timer will not be deallocated until it expires. So if the select is on a loop you can easily leak memory. Another of the Go hidden gotchas.
Good read! I just saw that in one examples you create and start a thread by invoking #startVirtualThread, and then you call start on the returned thread again.
Nearly all SQL Database access goes through it and it won't be rewritten in any practical time scale. It can't be meaningfully async without a rewrite and kotlin's coroutines can't change that.
With virtual threads that synchronous api can be used at the performance level of what a hypothetical JDBC rewite would give.
(Not that you want to hit your DB that much, but you know what I mean. You don't need to specially wrap or bundle apis to get into an async call juggle.)
There are already some efforts in that direction. You can check out r2dbc and the many vert.x sql clients. While they're not really jdbc, they're fully async like you described.
I was investigating this lately, and as you say, they are "efforts", and while laudable, I wouldn't go so far as to call them solutions. There are open issues/caveats as a result of the async model.
I do think there is something great about being able to continue to leverage existing JDBC-related tooling, given that it already exists.
And the importance of this first class support cannot be overstated. It means:
- meaningful complete stacktraces
- debugger support
- all blocking libraries get the benefits of async code automatically [1]
- all monitoring features of the platform work
- profilers get more useful again
- less mental overhead when programming, i.e. simplicity
[1]: With one caveat: Libraries which call directly into blocking native calls do not become async automatically. But they are very rare on the Java platform.
Kotlin's co-routine framework will be able to use Loom style green threads as soon as it comes available. Until then, it uses its own callback based mechanism to schedule co-routines (very similar to using Futures or Promises technically).
If you are already using co-routines, there's very little you will need to do other than updating the library when they add the support for this. Other than that, it indeed already has essentially all of the feature set that Loom adds to Java. So, do several other frameworks like Spring's flux, vert.x, rx java, etc. Kotlin's co-routine framework actually has extension functions for all of these as well as things like threadpools, worker threads, etc. in Java.
The value of Loom is mainly adding low level JVM support for this. Other frameworks will evolve to use that as well. The Java API to this is probably not that interesting; even if you do use Java. You probably should be using something a bit higher level. There are quite many frameworks for Java implementing all sorts of ways to do the same things. All Loom does is add a few useful things for these frameworks to do their thing a bit faster.
Kotlin's co-routine library is actually a multiplatform library that works in natively compiled kotlin on IOS/linux/windows/etc. (and soon wasm), in kotlin-js in a browser or node.js, and in the JVM. On each of these platforms, it uses a platform specific implementation as well as ways to seamlessly interact with platform specific things (e.g. promises in a browser, futures, threadpools, spring fluxes, etc. on the JVM). So, how co-routines are executed and scheduled is platform specific. The resulting code tends to be very portable and look very similar. I actually maintain a few multi platform kotlin things that support co-routines and they work great on both the jvm and in a browser.
When Loom releases, it will just be yet another thing that co-routines can be executed with. On the JVM, you can trivially create a co-routine dispatcher from a thread pool. A co-routine dispatcher is basically where you schedule your suspend functions to run. Loom basically just provides alternate threadpool and thread implementations that follow the same API that use the underlying jvm functionality (which is the really interesting thing it adds). So, the changes for Kotlin's co-routine library to support this are going to be pretty minimal. In fact, a lot of its current extension functions should just work against the Loom API without any changes.
There's nothing in the Loom apis that is particularly interesting if you use co-routines but the low level JVM optimizations are of course interesting from a performance point of view. Also interesting is the improved ease of dealing with stack traces; which can be a bit of a pain point with co-routines on the jvm. So, lots of good reasons for Kotlin users to look forward to this but no big reason to wait for it. You can use co-routines right now and the same code will continue to work great when Loom gets there in an LTS version in a few years. But it might run a little faster and be a bit easier to debug.
> You probably should be using something a bit higher level.
While there's nothing wrong in using something higher level, the main reason for using higher level libraries are because using threads doesn't scale past a certain point.
The problem with libraries that try to work around the scalability issue is one of compatibility. Using co-routines is great, but you need workarounds when targetting JDBC or a Http client. You have to be using coroutines, or coroutine friendly code, everywhere, or you risk reducing performance.
For this reason, we just use regular threads and threadpools in our projects. Once Loom lands we'll change the executors to spawn virtual threads and then we'll be done.
That said, coroutines shine if you can't wait for Loom to land or if you're targetting something other than the JVM (like Android).
I think you have a wrong concept of what kotlin co-routines are and are assuming limitations that it simply does not have. Anything you can do wit threads, executors, etc. You can trivially use with co-routines. Better still, mostly it uses those things directly and there is no performance overhead.
I've used co-routines with:
- spring flux
- hibernate & jdbc (not a problem, you just use your connection pool as usual and use a threadpool based coroutine context to ensure you don't block your asynchronous request handling main thread. You might prefer switching to some more modern versions of that that support asynchronous IO though. But that stuff is still relatively new and many people stick with blocking IO for this for now. Not in any way a show stopper for using co-routines.
- apache httpclient with synchronous IO and a connection pool (same thing basically).
- apache httpclient with asynchronous IO (very easy to integrate it's callbacks with suspendCoroutine { ... })
- redis (both synchronous and asynchronous)
- mongodb
- Java stream API (there are loads of Java libraries that use that)
I get it, some people don't want to switch language. So, stick with Java. But do it for the right reasons. Performance really isn't one. Anything Java does, Kotlin can do too. And generally about as fast by virtue of simply using the exact same things in the standard library. And generally without the boilerplate and a lot of syntactic sugar (i.e. it's nicer). Probably if the Kotlin version of something is slower than the Java version, it's because you are doing it wrong. I can't really think of a lot of cases where that would not be easy to fix.
I've seen my fair share of buggy Thread based Java logic over the years; trust me co-routines are way nicer than that. You can still get into trouble with them of course (and mostly for the same reasons). But it covers all the bases pretty well and generally does a better job of keeping you out of trouble.
> think you have a wrong concept of what kotlin co-routines are and are assuming limitations that it simply does not have.
What I’m referring to is this:
> you just use your connection pool as usual and use a threadpool based coroutine context to ensure you don't block your asynchronous request handling main thread.
For everything that is not coroutine compatible out of the box, you need to do something like what is described above.
When porting an existing application to coroutines, you also have to identify the places you need to introduce thread pool backed contexts and use them approprietly. You also need to setup these thread pools correctly so that you don’t end up using more threads than if you didn’t target coroutines.
For us, this extra complexity wasn’t worth it. If we were making an app from scratch and could choose coroutine friendly solutions for everything from the start, that would be different, but porting a legacy app to coroutines hasn’t been a good experience.
Fortunetly, project loom means all our current thread-based code will be just as scalable as coroutines in the not-to-distant future. That also means we can spawn more (virtual) threads and see a nice speed increase as well.
Btw, we’re already using kotlin for most code, just not coroutines.
OK, I get you don't want to do the work to migrate an existing Java application. It's a lot of work. If you don't have a lot of problems with it right now, you could just leave the code as is.
But just so you know, what you are describing is code that is currently boiler plate heavy in Java and essentially making it a lot simpler and easier to reason about with Kotlin. I think you are confusing problem and solution here.
In any case, this is how you create a CoRoutineContext backed by a thread pool.
val myContext = newFixedThreadPoolContext(100,"db-context")
That's it. There's nothing else to do but "use it". There's no need to call shutDown. Or have some thingy to intercept uncaught exceptions. Or have a lot of boiler plate around using it. a simple launch(myContext) { ... } does the job.
most of our thread pools are long lived, so we don't ever shut them down. When the pool is not long lived, we have a try-with-resources wrapper which handles the shutdown.
Most of the time, we do calls to .execute and then join the different threads at the end. The code running within the threads represent the majority of our code, so the boilerplate you're talking about might 10 out of 2000 lines.
Our only problem is the number of threads we can spawn safely, and that limit is removed with loom.
The interesting aspect is not "running Kotlin coroutines on Loom", but eliminating the need for source-level coroutine keywords and solving the problem of function coloring, no?
Sorry, but that's not what Loom is about at all. It's mostly about the jvm internals; that's the main chunk of work. The API level stuff is just designed to be consistent with the existing Thread API, with all it's technical debt, flaws, boiler plate, etc. that has inspired alternative frameworks and approaches to be written. It's consistent with that is the best you can say about it. To me it looks like it's a nod to other frameworks to make it as easy as possible for those to integrate this stuff. Good approach, very valid. It's not designed to be more usable / less boiler plate heavy. That's what other frameworks are for. Loom is not a replacement for these frameworks but something that is designed to seamlessly be integrated by them.
There's only 1 co-routine related keyword in Kotlin: 'suspend'. The rest of co-routines ships in the form of a normal kotlin multiplatform library that you just add to your project. Java has several keywords that you don't need in Kotlin (e.g. synchronized, volatile). You can mark things as synchronized with a simple kotlin function. And likewise there's a Volatile annotation. Mostly that's just useful for kotlin code that needs to be used from Java.
The function coloring bit seems to obsess a lot of people. I just don't see the issue with it. For me the boiler plate that Java makes you use instead is a much bigger issue.
Did you see the examples? Even with the blocking threaded style the Java versions gave me nightmares. And they didn't even bring up a cancelation example (not 100% sure on that). Cancelation and subsequent graceful shutdown is one of the hardest concurrency problems that (a) you are very likely to need in practice and (b) conveniently often are left out of example code. Heck it isn't even great in Go (too explicit imo) but at least they solved it.
Yep this is a utility function in sure thousands have written, including myself. It's not in std which is weird but go had been a bit inconsistent since context was a big api change.
Fun fact: that func has a hidden bug, if one should be pedantic. Can you spot it?
[Weird, I remember already replying to this comment.]
Yes that's what I was thinking of :)
I'm a bit unsure what's the value in draining the channel (which is internal to this func), given that Go should garbage collect channel (and in this case perhaps even some escape analysis).
EDIT: Ah they aren't just closing the channel, but sending a time.Time. That makes sense, the timer wouldn't have anyone to send to if it's unbuffered.
I think mechanically the plans for ScopeLocals can be a part of implementing a scheme like that. Either way - I need to read this thoroughly in the morning.
I deploy a single golang binary to well over 15k servers (many are only 100mbit) across many data centers and update it many times a day/week. Using xz, it compresses down to about 5mb.
As much as I love Java (I co-founded Apache Java), golang is a great fit for this usecase.
The majority of these machines are PXE booted, with a super minimal ubuntu based OS distribution... which would mean distributing a JVM as well.
Total boot payload is about 160megs... which I need to do some more work on... I think I can get this down to about 120megs, but it hasn't been a priority yet since this is working well enough for now. About 54megs of that boot is just some third party drivers and right now, those are .gz encoded... need to switch to xz to bring it down.
I presume it would have to be because there is no guarantee that the JVM will be installed and configured exactly correctly on all of those machines a priori.
Jlink helps solve that problem if you have a properly modularized project, but that's hard to do if your dependencies aren't also properly modularized.
Doesn't GraalVM as mentioned in the root comment solve that problem? IIRC Graal can compile a JVM + Java application to a single binary, but I could be mistaken.
I find it hard to be convinced to use GO at all. If i wanted efficiency, i'd bite and go with something close to the metal. If i wanted something that can build huge applications and has a rich framework ecosystem - i'd go with Java. GO seems like a middle solution for some exceptional cases, which i haven't deal with.
It is a fresh language and i quite enjoyed exploring it, but i just can't see the use of it. (also, the multiple return value functions are great, but if only there was a way to use a particular value directly from the function without having to assign them to variables)
The modern Java packaging also includes the runtime itself, there is not even a JRE anymore. And readability is somewhat subjective, Java is a really simple language, all the writer does is just method calls into various existing libraries. Since those didn’t get a chance to evolve yet with the upcoming virtual threads they are a bit longer at times.
So all what left is opinionated in-language support vs library support, and it is not clear cut which is the winner. Java seems to allow finer control over the primitives of concurrency, so a library or another JVM language could build up a much better abstraction over the mechanism.
I think the differences would drop off significantly if one were to write Java while keeping the Go guidelines and proverbs in mind; that is, reduce or eliminate the amount of 3rd party libraries (you don't actually need a dependency injection / wiring framework for most applications, just instantiate your objects with dependencies by hand in your main method), reduce the amount of abstraction layers, don't try to be clever, stick with regular loops instead of Streams, etc etc.
I agree, though do keep in mind that a complex framework like Spring doesn’t play in the same league as a typical go microservice in terms of what kind of application is getting written in them.
But I really hope that the new, record-based, minimal boilerplate java style gets hyped up more and more, we really should prefer compile time metaprogramming instead of reflection-based solutions. (E.g. mapstruct is really cool!)
Golang still has an advantage over Java + Loom + Graalvm, JVM as a better GC but Golang using more inline/value types puts less pressure on it and diminishes memory usage. So you need to wait also for Java project Valhalla to match Golang profile.
Hypothetically there is a version of "static Java" that could start to encroach upon Go's domain. This may or may not come out of Project Leyden, but I think for the reason the other commenter gave GraalVM isn't exactly it.
Still closer though, and GraalVM native images are pretty cool.
That's not true at all. The megabyte(s) of stack space is virtual, spawning a thread only requires a few kilobytes of memory initially, and is a lot cheaper than many realize.