Author of the go netlink library here. I've run into this issue a number of times. There has been conversation in the past about adding some kind of new runtime command like LockOSThread to prevent new threads from being spawned, but it didn't gain any momentum.
Even though I am a big fan of go, I've personally built two container runtimes in other languages do to the namespace clumsiness.
Personally, I think rust is an excellent alternative for namespace utilities.
One way to alleviate the problem at least for netlink library is to create a function which calls runtimeLockOSThread, sets into the required namespace and then opens a netlink socket using only raw syscall apis. One has to be careful in this code path to not invoke go runtime (i.e both socket and setns should be raw syscall apis) and not even trigger any allocations so that go runtime doesn't get a chance a spin a new OS thread. Once a socket is created in the required namespace you can get back to the caller namespace and return the socket fd. Now this socket fd is bound to that namespace and all netlink operations on that socket will happen in the target namespace.
Disclaimer: I am one of the original libnetwork authors and we have been aware of this issue with go for some time now.
One was a simplified runtime in c, similar to a stripped down version of systemd-nspawn. I can hopefully share the other one in a few weeks. I'm going through an open source approval process for it.
I completely agree, and I've been coming up with some ideas (based on runc) to see what sort of improvements can you have if you do a full redesign in a language that isn't as painful as Go to use.
At the moment I'm incredibly busy, but later this year I might be able to start working on that too.
I think the proper title is: Linux Process and Threads Don't Mix.
The Linux syscall interface exposes certain functionalities that are much more easy to reason about at the process level such as namespaces, capabilities, seteuid and so on. However these syscalls all operate on the thread level (since the kernel treats threads pretty similarly to processes). Therefore in order to perform these operations safely you need some sort of process wide mechanism to apply the operation on every thread (and don't forget error handling!)
This is _not_ just a golang problem or an M:N threading problem as many comments suggest. The kernel really needs to provide new syscalls for these features that operate at the process / thread-group level. The current syscalls are extremely difficult to use correctly in any multithreaded context in any language. When you consider the security implications of these features it makes the problem even worse.
Check out https://ewontfix.com/17/ for a really good analysis of the difficulty musl libc has faced making a multi-thread safe seteuid on Linux. There are also many bugs in glibc related to this as well. Linux makes userspace responsible for patching up the leaks in the kernel's process abstraction and that's really not a job that userspace is in the right position to take on.
> The kernel really needs to provide new syscalls for these features that operate at the process / thread-group level.
Or it could provide another clone flag that indicates that threads spawned that way should share privileges and similar things, then runtimes that need threads to all behave the same way can opt into that. I suspect that some tools do advanced privilege kung-fu that relies on those per-thread properties.
> The Linux syscall interface exposes certain functionalities that are much more easy to reason about at the process level such as namespaces, capabilities, seteuid and so on. However these syscalls all operate on the thread level (since the kernel treats threads pretty similarly to processes). Therefore in order to perform these operations safely you need some sort of process wide mechanism to apply the operation on every thread (and don't forget error handling!)
This is hardly a linux specific issue. Prominently for instance pthread_setugid_np exists on OS X, threads for different subsystems exist on Windows etc.
The M:N problem in Go is that you cannot control which thread runs which code. So yes, you could wish the OS exposed different APIs, but presently you can manage this situation in languages that let you manage threads.
The person you're replying to has already made that clear. It's, in fact, also possible to manage the problem in Go with some finagling. But like Go, if you have multiple threads, it's difficult
I've hit this exact problem with multithreading in C and setuid and just because it _can_ be managed in C doesn't make it easy or straightforward.
Therefore, I mirror the sentiment that there needs to be a way to operate on a process level, even if that has some interesting consequences.
(P.S.: In C, if you're using glibc, it DOES actually patch this issue up on its own using one hell of a nasty hack.)
This leads to go code being roughly as messy/clumsy as C (or whatever else) code in the sections that need concurrency and also need to change namespace. That's unfortunate, but I don't know that it really "raises a few eyebrows".
I mean, what's the better alternative to Go for this work? Maybe Rust? It is, at least more controllable at a lower level...but, not as easy to pick up for people coming from a C/Python/Perl/Ruby systems and ops background. I'm not saying one should use Go for containers/namespaces programming, but a lot of people are with some success (probably also banging into the namespaces issue now and then), I'm just saying it's not obvious to me what the better alternative would be.
No, it's a lot messier than C. In a regular C program (not using any special libraries for M:N threading) you wouldn't have to spawn an entirely separate process.
This issue is one of the downsides of Go's M:N scheduling. The OS is simply not aware of what the Go runtime is doing, and as a result you get impedance mismatches like this.
It "raises a few eyebrows" because M:N scheduling is unpopular outside of Go and Erlang. It was tried early on in the Linux world and abandoned precisely because of issues like this. Go has repopularized M:N lately, and it's proof that such a system can work for lots of apps, but the downsides of that decision are every bit as real today as they were in the early days of NPTL.
This makes sense, and I know it's why Rust abandoned green threading. But I can't help but worry that the focus on async I/O in Rust as a way of avoiding this issue is going to bring the language down a path that isn't as ergonomically pleasant as Go or Erlang's M:N threading for things like highly concurrent web services. Do you share this concern, or do you think Rust can achieve the same level of ergonomics without the impedance mismatch?
I hope the Rust team takes a look at Kotlins coroutines (and all the stuff they managed to build on top of them).
I think Kotlin is a good example because it is built ontop of a 1:1 runtime and uses very elegant primitives.
I was pretty disappointed Rust didn't ship with async/await and some form of lightweight thread but it's understandable.
async/await still has the fundamental problem of composability that all the other attempts at sugaring around an event loop have (aka the "functions have colors" problem [1]). It sucks for collaboration, which is one of the most important things for modern software development.
I don't like that article, because (1) it claims Go is doing something new, while Go is really just threads with a particularly idiosyncratic implementation; (2) it ignores the fact that you can convert async to sync by just blocking and sync to async by proxying out to a thread pool. The "what color is your function" problem really isn't a big deal.
Historically, much (I would probably argue most) C concurrency has been implemented with fork. Certainly not all, and there are many ways to handle it in C...but fork is really common, and I don't think it's considered all that big of a deal to do so. It is idiomatic (at least historically and across maybe billions of lines of C code), and not much harder to reason about than many kinds of thread implementation in C; in fact, it's easier to reason about fork than something like POSIX threads, IMHO.
But, are you saying that to switch to a namespace in Go one must fork, whereas one wouldn't need to fork in C unless you need to switch namespace and you need concurrency (because you can determine when things will happen with precision in C)? I don't know. Again, this is beyond my understanding of Go right now.
I may just not be understanding the implications of this. The code to deal with it looks reasonable enough to me; it's a smallish function, easily isolated. I think the author did a great job explaining the problem, the troubleshooting, and the solution. I just didn't see the problem as being all that damning of Go...but, that may be a reflection of my shallow understanding of the problem, or of the implications of the cost spawning a new process (to me, I always think back to the old adage "fork is cheap on Linux").
Arguably it has been implemented with clone(2) which has flags.
> in fact, it's easier to reason about fork than something like POSIX threads, IMHO.
Not if you call fork() in a multi-threaded program. That ends _exceptionally_ badly (let's just say there's a reason Go doesn't expose syscall.Fork and it has to do with horrific deadlocks).
> I just didn't see the problem as being all that damning of Go.
The problem is more subtle, and it comes down to maintainability and understandably. If you ever decide to read the runc codebase, I apologise. One of the reasons the codebase is so scattered is because of these sorts of hacks where you have to work around issues in the Go runtime (because it doesn't give you enough control). In the article, whenever you read the small function you have to keep in mind that it's actually spawning a subprocess (which then means you have to think about what namespaces had the process joined and so on). Go is an okay language, but it simply wasn't designed for stuff this low-level. We would be much better served with Rust in my opinion.
> But, are you saying that to switch to a namespace in Go one must fork, whereas one wouldn't need to fork in C unless you need to switch namespace and you need concurrency
In C you don't need to fork to switch namespaces, you just call setns(...). For the PID namespace you need to fork, but that's just a quirk of the interface.
In Go, you theoretically don't need to fork either (syscall.Setns is available). However, there is no real way to safely use it. First of all, the namespace interfaces in Linux are quite fragile when it comes to multi-threaded processes, but combine that with a runtime that will switch you between OS threads at random. And while the documentation on runtime.GOMAXPROC and runtime.LockOSThread might trick you into believing it's possible to stop the Go runtime from doing a clone(CLONE_THREAD|CLONE_PARENT), you can't.
> There is some prep for ugly contingencies(pthread_atfork) but it usually just works.
Actually, that's not true. Perhaps it's theoretically possible, in some cases, to make your threaded code work with fork with the appropriate pthread_atfork() handlers, but in general, it's a total mess. Thread A has a lock, thread B calls fork(), thread B tries to acquire the lock. The lock is held, but thread A is not running in the child. That much should be obvious.
The usual fix is to write a pthread_atfork() handler for every lock in your program. Before forking, it locks all of your mutexes. After forking, in both the child and parent, it unlocks them. You can see the holes in this. One simple hole is that fork() is async signal safe but pthread_mutex_lock() is not, but most people won't fork() in a signal anyway. The big problem is that you now need to make all of your locks recursive and figure out, globally, what order to lock them in. This means registering your atfork() handlers in exactly the right order, something which is more difficult than it sounds. Another problem is the performance implications of acquiring all the locks in your program.
Namespaces + threads are fine, the only problem here is namespaces + M:N threads + no method for pinning a green thread to an OS thread, like forkOS in Haskell.
> Namespaces + threads are fine, the only problem here is namespaces + M:N threads + no method for pinning a green thread to an OS thread, like forkOS in Haskell.
Not quite. unshare(CLONE_NEWUSER) and setns(fd, CLONE_NEWNS) require your process to be single-threaded. So precise control over threading is needed.
On paper, all code between a fork and exec (in a multithreaded process) must be async-signal safe as well. I'm not sure if this is also true for the callbacks with pthread_atfork (I've personally never used it), but it's something to keep in mind.
There’s a defect report somewhere about this exact topic—there’s no requirement that pthread_atfork handlers are async-signal-safe, but fork is marked as async-signal-safe. The question is whether the documentation should be fixed, whether atfork handlers should just not run if fork is called in a signal handler, etc. IMO if someone’s forking in a signal handler in a multithreaded program nothing good is going to happen anyway.
*...it's a total mess...theoretically it's possible....speculation on standard behavior based
on platform behavior...namespace(x)<-threads->chosen implementation detail...functional language reference...
-- Translation: Design is broken
> If a multi-threaded process calls fork() ... the child process may only execute async-signal-safe operations until such time as one of the exec functions is called.
I don't know about "design is broken". I personally think the fork()/exec() model is a clumsy way to create a new process, but that's a matter of taste.
> Rust is painful to write, painful to learn and makes you ask yourself:
It's painful to write because writing software that is memory safe is not easy. Rust makes it insanely easy in comparison to the "old way" of doing it (hacking together a C program then having to patch security holes every release, never being certain that your code is actually memory safe).
> Can I not write a well designed and correct program in C? Of course I can.
Of course you can. All it takes is a separate formal proofing process, countless hours of static analysis and many more months of fixing security bugs. Good thing that's so much simpler than Rust's static analysis which runs on every compilation and mathematically proves that its memory model was not violated.
I think if we have learned one thing in the history of C programs, it's that this sentiment that "a sufficiently smart programmer can write safe C code" is quite harmful. While technically true, I don't think I've ever met such an individual and I doubt one exists.
> Can I not write a well designed and correct program in C?
We have decades of coredumps and exploits to convince us that nobody can. How many well-known apps can you name that have never blown up randomly? I can't think of a single one. How much more failure until it's reasonable to think that C requires an inhuman level of perfection and almost any alternative would be an improvement?
I hear good things about seL4 but that wasn't written so much as translated from Haskell during a formal process that makes Rust look like finger painting.
> I hear good things about seL4 but that wasn't written so much as translated from Haskell during a formal process that makes Rust look like finger painting.
It's actually more fun than that. They have two implementations of seL4, and they use formal proofs to show that the Haskell model is identical to the C implementation. How much fun is that! /s
GNU Bash segfaults for me whenever I type ~[tab]. Thankfully this only happens on the small few systems I had to manually patch after the epic Bash vulnerability a couple of years ago - older internal systems that are still in use for historic reasons rather than regularly relied upon in production. So it's never been annoying enough to fix.
Rust and Go advocates will have you think that they have conquered security via the memory mgmt and API front but there is still the off chance they haven't..or that the SA (what is left of that maligned profession outside playbooks and the devops marketing you read here and online) has allowed you a chance for glory.
There are basically two rules to a well written C program (if I am now allowed to speak despite the public outcry).
1. Do not trust user input. This is a cardinal rule in whatever source. If the rule were followed vigorously in
every case there would be 90% less exposure. When you take user input, filter.
2. Learn the standard and stick to it.
Finally #3 (unix) Write an application to do a certain thing well.
On #1, having just stumbled across a deserializer that can be commanded to allocate a 2^63 byte buffer, I agree 110%.
On #2, the problem is that the standard says things like "walking off the end of an array is undefined behavior" and "use after free is undefined behavior" yet we don't seem to have any programmers who can be trusted to reliably avoid these problems with zero runtime checking.
I'm downvoting you for being profoundly shitty and mean, not because I disagree with you.
I do disagree with you, as it happens, but I'd downvote this sort of thing if I agreed too. There's a difference between bluntness being read as rudeness and what you're doing. Stop.
> I can't understand anyone who would compare Rust against C in a positive way without possibly profiting from it.
A year or so ago I wrote a Postgres extension in Rust. It was a real delight to have a package manager and to only have to worry about allocations within Postgres.
But whenever I want to code a big hairball without libraries or namespaces and use autoconf and all that great stuff, I still stick with C.
Please report this to the Hacker News mods, if you have evidence. It's against the rules, and they check this kind of thing. Nobody should be doing it.
> IP Holders, well this is speculative.
All of Rust's stuff is MIT/Apache2 licensed. There is a trademark on the name and logo, held by Mozilla, but other than that, there's none.
My first thing I wrote in Rust was porting a trivial assignment for a security course in University from C to Rust. It turned out I had an off-by-one buffer overflow that Rust caught.
This clearly showed me the advantage of Rust over C.
> But, are you saying that to switch to a namespace in Go one must fork
The article is stating that to control the namespace that the threads execute in, that a separate process must be spawned so that the entire process can be forced into the correct namespace. Otherwise, the runtime can spawn new threads as it sees fit and you don't have control over which namespace they are in.
It might be possible to work around this from within the same process if it were possible to force the runtime to not spawn new threads in particular cases. If you could control when the runtime was allowed to spawn new threads, then you could organize the program / threads in a way that would keep the correct code operating in the correct namespace. Unfortunately, you can't.
Disclaimer: I don't know Go, but this is my understanding from reading the article.
I mostly agree. Of course, for as cheap as a fork can be (and in Linux it's very cheap), it's not as cheap as a systemcall.
When changing namespace happens often in a hot path, forking might not be fast enough.
I disagree with Walton that the blame is on the N:M threading. It's really on the Linux kernel that has never made a clean distinction between threads and processes, both in kernel and in the APIs.
I agree. Bestowing certain permissions/properties on a thread doesn't make sense. The thread shouldn't "enter the namespace", an API should retrieve a handle to that namespace that should then be usable from any thread in the process via some appropriate calls.
On the other hand having individual threads entering namespaces means your broker process does not have to constantly switch namespaces, it can use shared memory between differently privileged threads to do the brokering.
I know nothing of the implementation here but presumably these namespaces can exist side by side in a way that doesn't require any "switching". If switching is expensive that would make context switching between a thread in one namespace and a thread in another namespace just as expensive?
If you have one thread in one namespace and another in another you now have to worry about what you can do in the context of a callback. This asymmetry just makes any multi-threaded program more complicated than it needs to be (and already is).
Switching is a system call (setns), in principle shared-memory IPC does not involve context switches, just lock-free data structures. I'm not sure how common this is in practice since shared memory also has some downsides if you're doing this for security.
But there also are non-security applications of namespaces.
And it's not like namespaces are the only per-thread thing in linux. Capabilities, uid and signal handlers come to mind.
Just saying, but Erlang do not target that type of "System Programming" and the answer to that problem in Erlang would probably work through totally different way to do it. This namespace thing would not be a problem.
This is not a problem of M:N. This is a problem of Go being badly designed. Not new.
More like reification of the very old problem from the times when we got
threading API, with thread safety of functions that alter process-wide state
(umask(), chdir(), sleep()), except now it's calls that alter some
thread-specific thing.
It's a problem with Linux not permitting inspection of interfaces through a namespace. If I have a chroot I can look in from outside easily. Apparently not so with container interfaces.
As far as I know, Solaris threads are actually M:N threads on the system level. M:N threads are extremely efficient. A "native" thread is a heavyweight construct - e.g. by the stack space allocation. So this puts a limit on how many threads you can spawn in a program. For many tasks, this would force you to roll out your own M:N mapping system yourself. Doing it on the language level usually yields the better results.
Due to the implicit M:N mapping, Go goroutines are extremely cheap. This allows you to spawn as many, as your algorithm naturally requires. The Go runtime will automatically map them to native threads - typically one per avaliable CPU. As a consequence, a Go program that heavily uses goroutines has a pretty clean code and scales without much overhead across a large variation of number of CPU cores.
Rust is much better for this (though I still feel some of the fork/exec interface has similar warts to Go). However, you're wrong in saying that it's "roughly as messy/clumsy" as C.
Let me tell you how runc works. runc is written in Go, and we take an OCI configuration file. Because we can't just fork and set up all of the namespaces in Go, we have a C function called nsexec which is specified as __attribute__((constructor)). This ensures that our code will execute before the Go runtime boots. The parent process writes (using netlink as the wire protocol) to a pipe that the child has open and is parsed in C. Then, the child will have to do a series of forks, unshare, setns, {open,read,write} and so on (and the final PID needs to be sent back to the original parent) in order to set up and join all of the necessary namespaces.
In C, this code would be _immensely_ easier to read, write and maintain. Just look at LXC. Personally I really wish people had just gone with Rust earlier on rather than implementing everything in Go. I've had nothing but pain from Go.
Thanks for explaining how it's done in runc. That does sound pretty awful. So, even though the initialization can be outsourced to a C function, you still would prefer to be working entirely in C? Are there no advantages to Go for runc? And, would it be possible for someone to write a somewhat standardized Go library for doing this grunt work?
Is it merely fear of C that keeps so much of the container infrastructure on Go? I've only spent a couple of weeks looking peripherally at Go, and I already like it better than C (which I've poked at peripherally for ~25 years), but I don't know it well enough to know its warts.
> Is it merely fear of C that keeps so much of the container infrastructure on Go?
Sort of. Go binaries are statically linked by default which is a must in situations where you are e.g. unsure about what libs are available in the current environment. You have to go through a lot of hoops to make sure your C executable is really fully statically linked.
> you still would prefer to be working entirely in C?
No, but there are more options than just Go and C. Rust is an option that I'm shilling at the moment (though I've only started learning it, so take that with a grain of salt). The main reason I would want to write it in C is because there's a lot of string parsing code you have to write in order to make container runtimes work -- and as we all know that's probably the #1 source of security vulnerabilities.
> Are there no advantages to Go for runc?
There are, mainly due to network effect (everything else is written in Go) and getting contributions from the community (Go is easy to pick up). Unfortunately there are also disadvantages, and quite a few of those disadvantages are present in Go but are not present in other memory-safe and low-level languages (Rust is a good example because to be quite honest it's the only player in this space that doesn't try to do more than necessary).
Go is a good language for it's designed for (Web servers and similar things), but from my experience it's not the best choice for low-level tasks. We've seen cases where long-running container daemons (not naming any names) will crash if you run more than 1000 containers on a single system. They don't crash because of the actual daemon code but because of issues with Go's GC (it doesn't actually free memory sanely, it uses MADV_DONTNEED which inflates RSS and causes OOM to kill your daemon).
> And, would it be possible for someone to write a somewhat standardized Go library for doing this grunt work?
Of course (and you could argue that we have done that in runc with github.com/opencontainers/runc/libcontainer/nsenter), but the thing to note is that in order to get around problems in the Go runtime you have to split out a single piece of code into separate processes and have to now redesign how a single function would work. So moving it to a separate library means that development is even more frustrating (you've created an API around the internal implementation of whatever thing you're working on).
> Is it merely fear of C that keeps so much of the container infrastructure on Go?
I think the network effect is the main reason. Most of the people I've worked with know quite a lot of C (we do kernel work sometimes) so writing a runtime in C would be frustrating but entirely doable. The problem is that you couldn't just import it into a Go project (and people don't like cgo because it makes binaries harder to build in certain cases).
> I've only spent a couple of weeks looking peripherally at Go, and I already like it better than C
Go definitely has it's uses, and I still use it for new projects. For example I recently wrote a tool for dealing with OCI container images in Go[1]. The standard library of Go is quite nice (though I found some bugs in archive/tar but let's not go there) and I'm always quite amazed just how much you can do before you have to import external libraries.
But I recently started learning Rust and I am _really_ enjoying being able to understand what my program will actually look like when compiled. If you've ever had to strace a Go program, you'll know exactly what I mean. Debugging Go programs is basically fucking impossible.
I've been tinkering with go, and the project I'm planning to use it for is container-oriented (building/using/distributing them for non-technical users, more than working with them at a very low level like runC or similar, but still, it's very useful to know). As an aside, umoci looks, at first glance, like one of the components I thought I'd need to build...so, that's cool.
I'm not gonna keep bugging you with questions; I'll go read the code. (I'm also finding rust really neat, conceptually, even if I'm not yet finding it easy to read or understand.)
I never said it was a nice hack. Personally I think this would be infinitely better in C or Rust.
Oh, and I haven't even mentioned the absolute shitfest that is the cgroup namespace and how you have to set up cgroups before you unshare it because its behaviour changes based on what cgroups you were in when you unshared it.
You can keep using Go. Its behaviour here is actually usually what you want.
To avoid the authors' issue, you can write some functions in C. CGo has a very high level of integration (you can mix languages in the same source file) and would be quite simple for the case of a setns/execve wrapper.
No, no it doesn't. CGo is slow as balls to call into and return from.
Types can't fully be shared.
It's interacted with via comments.
You can't use cgo to control the threading of the Go runtime itself, which is the real problem here.
The behavior you want is to be able to call linux syscalls that operate on threads and not be utterly fucked.
That behavior cannot be accomplished with go nor go+cgo easily.
> You can't use cgo to control the threading of the Go runtime itself, which is the real problem here.
Go is deliberately opinionated about threading of the runtime. I think it's unlikely Go will offer much more control over these internals (beyond GOMAXPROCS), given the philosophy around e.g. GC tuning.
Cgo is a beefed-up runtime.LockOSThread() that could be used to avoid having the author resort to a helper process.
> CGo is slow as balls to call into and return from.
They're slower than Go function calls, but they still take only nanoseconds. This is negligible on the authors' scale of "launching an entire container".
For their case of "it is not possible to guarantee that a new OS process ... will run in a given namespace", you're not even returning from Cgo after exec.
> It's interacted with via comments.
Do you use build tags? Or go:generate? Like it or not, it's idiomatic.
I'm with you on the types, though! `go tool cgo -godefs` helps, but it would be great to see improvements especially in the reverse case of exporting Go buildmode=c-shared for C consumption. Still, a little marshaling seems tidier than a whole helper process.
Yeah, you're correct. Though it doesn't change that it's a deceptively named variable -- you have to read the runtime docs carefully to realise that it only refers to the number of threads concurrently executing Go code.
> I mean, what's the better alternative to Go for this work?
Separate processes, like the post suggests?
I have no idea why someone would expect user-level pseudothreads to execute across system-level primitive boundaries.. seems fairly obvious to me.
I don't expect a chrooted daemon (e.g. apache, etc) to have access to parent thread contexts.. etc.
Fork & pipe IPC shouldn't be too difficult for anyone to understand, beyond that, if you don't understand these things, you probably shouldn't be writing code that complex..
In Linux, many properties are task-related and therefore the API is also task-related. Namespaces are not the only problem. See for example https://github.com/golang/go/issues/1435. The problem is Go runtime does not provide a way to exclusively lock a system thread from existing or future routines. Most other languages would work just fine (C or Python for example).
The documentation for both GOMAXPROCS and runtime.LockOSThread lie to you. Neither of them allow you to stop the runtime from creating new threads (or executing library code in the wrong thread).
Trust me, we've been trying to get Go to co-operate with containers for quite a few years. It's not as simple as just reading the standard library docs. ;)
Okay, I think I understand. I thought the problem referred to above with setuid was separate and would be fixed by locking the thread, but spawning sub-threads with the wrong UID is a problem. See also my sibling response.
No, it doesn't. Go runtime can decide to spawn a new thread from the locked one. The new thread will inherit the characteristics (namespace, uid) of the old one. See https://github.com/docker/libnetwork/issues/1113 for more details.
> Go runtime can decide to spawn a new thread from the locked one
That sounds like an implementation issue, why not assume the documentation is the intended behavior and this side effect is a bug? I'd support a CL to fix this behavior or add a block-clone-from-here runtime call (but the end result of that is you want the thread to exit when you're done with it, and not to go back into the pool... which is also new behavior). At the minimum, this behavior of new threads spawning from LockOSThread could be documented.
As a workaround, what about CGO -> pthreads -> spawn a control thread free from the Go scheduler -> call back into Golang to run a control loop function? You can do this in init() to ensure it has full control over itself. Or will Golang call clone() from unscheduled code?
> That sounds like an implementation issue, why not assume the documentation is the intended behavior and this side effect is a bug?
Because after discussion with the Go devs they've concluded it's not a bug. To be fair, it's their decision to make a language runtime hostile to user's being able to mess with the process model, it just makes programs hard to write.
> At the minimum, this behavior of new threads spawning from LockOSThread could be documented.
The thing is it is documented[1], it's just very subtle:
> LockOSThread wires the calling goroutine to its current operating system thread. Until the calling goroutine exits or calls UnlockOSThread, it will always execute in that thread, and no other goroutine can.
An implication of the emphasised part is that if you use 'go' (or a function you call uses 'go') inside a locked goroutine, you are guaranteed that goroutine will be scheduled on another thread. Which is not what you might want or expect (personally I would expect goroutines created from a locked goroutine to act as normal coroutines). The problem is made much worse because a lot of the Go standard library uses 'go' internally and there's no way for you to know what functions use it and what functions don't (and what functions might use it in future versions). Not to mention that there are even more edge cases where functions you call could end up on separate OS threads.
> As a workaround, what about CGO -> pthreads -> spawn a control thread free from the Go scheduler -> call back into Golang to run a control loop function?
At that point you're really just massively hacking around the Go runtime. I would not be confident that such hacks would be a good idea in the long run. Remember that Go doesn't have any real forking model in its standard library or language, so the language provides no guarantees that it has to "play nice" with foreign threads.
Also, calling from foreign C code into Go is quite difficult (especially if you're calling into an _already running Go program_ which might reschedule your code at any time and would require hooking into core runtime internals).
> You can do this in init() to ensure it has full control over itself.
init() runs after the Go runtime starts up, you would want to do it as an __attribute__((constructor)) in C code so it is started before the Go runtime.
> language provides no guarantees that it has to "play nice" with foreign threads
I hope it continues to play nice, as I have a project depending on this behavior (bindings for a CPU emulator which spawns a thread for the CPU main loop, and calls back into my Go code on some events).
Thanks for the info. I guess the biggest problem is making sure `go` in a locked goroutine doesn't release threads to the scheduler from the locked goroutine? That's a weird thing to reason about. I initially guessed you'd need a separate feeder thread, but you want `go` from a locked thread to keep the uid and namespace of the locked thread.
So I guess there are two potential solutions:
- goroutines spawned from locked threads are pure coroutines and will not be scheduled on another thread
- goroutines spawned from locked threads can only execute on the locked thread, or on threads spawned from the locked thread that don't predate the goroutine. Child threads can also only be used for matching child goroutines, and will be more aggressively collected. This would converge closer to an oldschool threading model than green threads, but should hopefully prevent bugs like UID/namespace hopping.
This is the kind of thing you could possibly do with a small patch to the Go runtime. Go makes it so easy to build the compiler/runtime, perhaps forking Go for this is reasonable until there's an official solution.
There's also the issue of what to do with the primary thread that has been marked unsafe when it returns. Instead of LockOSThread, you probably want "give this OS thread and its children magic sandbox scheduling behavior, which includes collecting the thread when it returns, because we're going to call stuff like setuid".
I don't know, exactly, but I know with some confidence that C++ has never had significant uptake among ops and systems people. The folks who build systems, run systems, and dip into code on occasion to make the systems run, but not as their full-time job, have never (to my knowledge) fallen in love with C++. They probably all know some C, Perl, and Python...maybe not a lick of C++ (except the "C" part).
I think C++ may just be too rich a language to be a part-time thing. I think Rust might have the same problem (though I'm finding it easier to read than C++, it doesn't seem to be afraid to require a lot of learning time from its developers). But, I'm willing to entertain other theories.
For whatever reason, Go seems to have very quickly entered that category of language that systems and ops people are comfortable with.
I agree that rust will have an adoption problem amongst part-timers. Rust code is unreadable until you spend a few days with it and writing it requires a week or two of wrestling with the borrow checker.
In contrast, one of the reasons I'm a big fan of go is it is extremely easy to read for almost anyone. People pick it up very quickly.
I can only speak about my own experience as an ops person, but C++ always just felt off to me for some reason. I tried to learn and use it for about 6 months and I always felt like I was so bogged down in minutia and standards.
C let's me focus on the underlying system and it is what Linux is written in, Python is quick to write and has great libraries, and Go is amazingly easy to pick up/read.
C++ always felt like it was more so a language designed for programmers who love code than something that lets me focus on what I love (systems).
Better C++ than Go or Rust. Antecedents and track record counts. Young people like the new thing. Go and Rust are capitalizing on that. Learn C. Take the time..it takes a
couple years and some pain to learn it and then you will be amply rewarded. These languages (go|rust) are reactive and suffice for some purposes but they really kind of suck in every other possible way.
I'm old(ish), have worked professionally with C, and I think I have to disagree with you. Certainly, learn some C; it's the language on which all of our platforms are currently built.
But, building new things in C? Nah. I don't see any reason for that. Since C was designed, we've ("we" as in our industry, not specifically me and you) learned a lot and our systems have grown massively in all dimensions. It would be optimistic to assume we don't need different tools 40+ years on.
C++ looks good on paper. It has containers and types and generics. You come to find out that it's all based on meta-programming which is an interesting idea: Basically you're writing code to write code, and that code writes code. The whole system is macros all the way down. There's no native language support for anything but macros and the macros implement everything.
The student's experience is that he can quickly solve a problem using a list of stacks of strings (vector<stack<string> > > in the parlance.) Which is fine, almost like typed-python until you make a mistake. Then, the compiler, who knows nothing about those types which were all built by expanding macros, is not your friend.
Miss a minor `*` and a single line of code will fully expand its underlying macros giving a 10-page, indecipherable error message.
Should you make it to runtime, no debugger can tell you the contents of a vector, string or stack. They're just blobs of buffers and pointers with mangled (and yeah, that's the word that they chose: mangled) names to make them extra unreadable.
This is when most people ask if they can just have C back.
In C, I could not find good libraries for basic stuff (dynamic string manipulation, variable-sized arrays, hash tables). I tried glib but it was much worse than C++'s STL.
It is entirely possible to write C-style programs in C++ (very few classes, globals all over the place) and so on, and at least for me, C-style software in C++ are much safer/easier to debug than C-style software in C .
In my experience the C mindset is to write those yourself. The lack of templates means that it's very difficult to write proper generic containers. You either end up with macro soup, void * soup or code tailored to your needs. Or a lot of the time a combination of all three.
But I'm like you, if I know that I'm going to need "advanced" data structures I'll pick C++ over C (or these days more likely Rust).
To go along with this, due to overloading and inheritance rules you cannot look at a small section of C++ code and determine what it does without reading up to the entire program.
I think that holds for any program written in any language. If the abstractions are confusing then it will be hard to read, be it layers of C macros, over-use of template programming or bad class hierarchy. BTW C++'s standard library is mostly functional (having roots in scheme), which makes it pretty easy to reason about.
> I mean, what's the better alternative to Go for this work? Maybe Rust? It is, at least more controllable at a lower level...but, not as easy to pick up for people coming from a C/Python/Perl/Ruby systems and ops background.
You know, it's interesting. I've been programming with Python for about 6 years now. I've also picked up Javascript, SQL, bash, and PHP along the way. I'm always gaining a little bit of C knowledge here and there when writing C extensions for my Python applications. I'm a fairly experienced programmer at this point. To the point:
I tried picking up Go one day because I was hearing so much about how it could replace Python as network glue code with better performance and reliable concurrency. I can't really validate or invalidate those claims. That said, I found Go to be sort of difficult. The syntax is really simple. Compiling is really simple. Concurrency is even simple. However, need to do something in a different way than Go decides is correct? Well, you can't. It won't even compile. The difficulty in Go is in learning about what the compiler thinks is OK. I don't really like that. You don't really know if your code will work until you compile. Basically, I just think Go isn't really flexible enough for modern programming. I find that Nim can do Go's job better than Go can for my use cases anyways.
> The difficulty in Go is in learning about what the compiler thinks is OK.
I think this is similar to what people think about the type system in Haskell or the borrow checker in Rust. With every higher level language comes new things to learn and obey.
But with Rust once you learn to work with the borrow checker you can work at any level of abstraction. With go you get what the Go implementors decided was best for you. (I omit Haskell because I don't know it)
I want to mix C++11/14 into that also. Now that the typesystem is used by the standard library to describe resource ownership in code whole categories of errors can be found at compile time. If you are new this it can seem like you are fighting the compiler, but once you learn that the compiler just won't let you make certain kinds of mistakes you get access to every level of abstraction with C++11/14 and Rust. You can code in terms of passing database bindings betweens threads in threadpools or you can twiddle individual bits in specific registers and everything in between.
I don't know Go well, but it seems really limited. You can't write certain classes of bugs, but you also can't write many design patterns.
I have experience with Rust and Go as well as a dozen or so other languages. Go usually has one or more easy ways to solve a problem, and in my experience, these easy paths are usually much easier than in other languages. There are certain problems for which this isn't true (like the one documented here), but it's generally true. These easy paths are often not elegant, and sometimes they trade a 1% risk of type error for a 60% easier solution. They may also trade a small amount of performance for a large boost to usability.
Whether or not I choose Go is almost always comes down to whether I want to solve a problem quickly and with decent performance/readability/etc or if I want to take a lot longer for a solution that is more aesthetically pleasing or with a smaller risk of error.
I'm sure lots of people will argue that their language is faster to develop in, but apart from toy programs or those requiring libraries unavailable in Go, this has never been my experience. In particular, even after two years with Rust, I still have trouble reasoning around memory management, lifetimes, how to pass around functions, how to do anything asynchronously, etc. Go isn't perfect for anything, but it strikes a good balance for the kind of work I tend to do.
> I don't know Go well, but it seems really limited.
That was my experience, too. Vaguely interesting in a couple of ways, but definitely a bondage and discipline language and nowhere near compelling enough to make me want to put up with that.
Even then you don't know if program is correct. There have been pieces of software running in production for 20 or 30 only to fail because of some unforeseen and planned for error condition.
My favorite was the Comair christmas failure back in 2004 or 2005. I tried googling the root cause, but I think it was an Integer overflow in the 16 bit integer they used for storing the flight number. The system was designed in the mid 80s and 65535 flights was an insane amount, but but when the software failed on Christmas it interupted more than 1,100 flights.
I am curious why people are downvoting me. It seems obvious to me to that proving software is correct is hard or impossible. It building and passing the tests help to increase confidence in correctness, but doesn't prove it.
No one disagreed with this comment and it is one of the least controversial thing I have said on HN. I think I have a downvote fairy, someone who just downvotes everything I say. If so, why bother? If I don't have one then when you downvoted me, why didn't you respond?
I disagree. For instance, if I'm writing something in C and I know that it is syntactically and semantically correct, I know that it will definitely compile.
Go hijacks control flow and makes it more difficult to reason about how your code will be compiled and executed. At least that is true in my personal experience.
Sorry, I don't understand what you mean by "Go hijacks control flow". Could you elaborate cases, where Go did not behave as you expected? And where syntactic and semantic correct Go code did not compile?
As far as my understanding is concerned, channels are a way for goroutines to pass data between one another, correct? Yet in this particular case, the channel `messages` is operating in the same manner as a generator or array. So, am I creating goroutines by passing messages to the channel or am I just using the channel as a simple buffer?
Channels are a safe way for goroutines to communicate. Safe here means, that the channel itself performs all the necessary locking needed for any number of goroutines to read or write "at the same time" to and from the channel. Other than that it does behave like a plain pipe. You write into it and can read from it "on the other side". Creating a channel does not create a goroutine. A goroutine is only created if you use the "go" command, like "go f(42)". This would run the evaluation of f(42) in a separate goroutine, which behaves like a thread, but uses less resources. So in neither of your examples, goroutines are created. You can optionally add buffering to the channel. A write to a channel without a buffer blocks, until the value you write to the channel is read. If you have buffers, you can fill them without blocking, even if the values are not consumed immediately. Without the buffer, the first example would get stuck, as the first write to "messages" would block until there is a read on that channel - which is never performed. But due to the buffering, the two writes to "messages" are performed without blocking, and the values can be read after that. A third read on the then empty channel would also block infinitely.
The flow of data through channels can be confusing until the concept becomes familiar, but the behavior is well-defined, so the behavior is as easy or difficult to predict as of any program of the given complexity.
It really just seems like you're saying "When I think my Go code is correct, it turns out it often isn't, but when I think my ${other_programming_language_i_know_better} code is correct, it turns out it usually is.
When I read about channels and various forms of IPC/inter-thread communication in most other languages, and remember the past pain of threads and mutexes in C/C++, I am so happy to be able to send a message in Erlang to another Erlang process (Pid) as easily as this:
Pid ! {self(), 42}
And in the process identified by Pid, to receive the message:
receive
{From, Data} when is_pid(From) ->
handle_data(From, Data)
end
And beyond that, the process identified by Pid can be running on a physically separate host system - it's transparent.
Too bad we can't use Erlang for systems programming, but you can't have everything!
>Am I implicitly creating goroutines by using channels or am I simply creating a buffer?
A channel is just a thread-safe queue. An unbuffered channel blocks until something else receives the sent value. A buffered channel allows n sends until it blocks. Using a channel will never create a new goroutine, its just a way send data. You can use a select statment to wait on sending or reciving from multiple channels or to send or get without being blocked by a full or empty channel.
In C you could put together a RW mutex, condition and a linked list or array and have the same thing pretty much.... If you want to create a goroutine you need to invoke "go X".
Yes but the problem with languages like Nim is lack of support and maturity. Go is more versatile and at the same time more mature than anything out there. It is a different design and it excels in what it does (considering all tradeoffs now).
Will Nim be as versatile and solid as Go in the future? Hard to predict but i would say no. You need a solid financial backing and certain amount of adoption where people actually write software that makes them money.
> Yes but the problem with languages like Nim is lack of support and maturity.
Agreed, that's why I don't use it for anything super important yet. Nim is approaching a 1.0 release soon. Go has a similar problem in that it is maintained almost entirely by Google who has a history of dropping projects without warning.
> Go is more versatile and at the same time more mature than anything out there.
This is objectively incorrect. In fact, Go aims precisely to be non-versatile for the sake of simplicity. That is why Go does not have generics for instance.
> It is a different design and it excels in what it does (considering all tradeoffs now)
I don't think it's design is all that different. It looks like a stripped down version of C and it's definitely not the first PL focused on concurrency.
> Will Nim be as versatile and solid as Go in the future?
Nim is already leagues above Go in the versatility(I assume you mean flexibility?) department. As far Nim being as "solid" as Go, I'm not entirely sure what you mean. If you're asking about stability, I believe that Nim can reach a similar level of stability as Go, yes.
> You need a solid financial backing and certain amount of adoption where people actually write software that makes them money.
I agree with this. However, it's not always a quick process. The only reason Go is as popular as it is is because of Google's size and reputation(edited). Every programmer on Earth heard of Go within a few days of it's official release. Nim is taking a slow roll approach. Look at Python. It took almost 15 years before it started getting really popular.
All that said, I didn't come into this thread to argue about Go vs Nim. I've been accused of shilling Nim in the past. I'm sorry that I like talking about PLs I enjoy using.
It's more than fine to be enthusiastic about programming languages. However, some of your points about Go are dubious.
The reasons Go doesn't (yet) have generics are practical rather than philosophical. And well-documented.
I also don't believe Google's marketing budget contributed much if anything to supporting Go. The _reputation_ of Google and the Go authors was far more important. (Personally, when I saw people like Brad Fitzpatrick try Go and rave about it making programming fun again, I decided to try it.)
You can happily accuse me of shilling Go if you like :-)
> The reasons Go doesn't (yet) have generics are practical rather than philosophical. And well-documented.
Can you give an example? I guess I fail to understand why a statically typed language would choose to forgo all of the advantages generics provide. Does it have something to do with Goroutines?
> I also don't believe Google's marketing budget contributed much if anything to supporting Go.
I didn't necessarily mean their marketing budget. I should probably edit that. I meant that anything they do is news.
> Can you give an example? I guess I fail to understand why a statically typed language would choose to forgo all of the advantages generics provide. Does it have something to do with Goroutines?
Like I said, the objections are practical, not philosophical:
Also, bradfitz said recently on the GoTimeFM podcast that doing Generics and Go2 together makes sense.
> I didn't necessarily mean their marketing budget. I should probably edit that. I meant that anything they do is news.
I feel like the comments from others about Dart give the lie to this one. Go is fantastically, dramatically, massively more popular than Dart, which is also from Google.
> Like I said, the objections are practical, not philosophical
Russ Cox and Rob Pike keep repeating this mantra, but I don't buy it. No, I don't believe they object the idea of generics itself. And yes, generics pose practical complexities and a plethora of issues that have to be resolved.
But so does every other language feature. The features you choose to add reflect your philosophical values and priorities.
Go chose to bake some hitherto very niche features that could have been as libraries into the language. Channels in particular, are a language construct only because Go doesn't allow operator overloading and generics for user types. But channels get them, because channels are demonstratively important for Rob Pike[1].
There's nothing wrong with that of course, but that's a philosophical decision. Why channels can avoid the vagaries of interface{} boxing, but not sets, linked lists or queues?
The reality is that every modern statically typed language except Go has generics, and they all implemented them very well. The ML languages and Ada were already doing it in the 80s, and OCaml and Eiffel managed to combine generics and polymorphism back in the 90s.
It seems to me that when originally designing Go, Pike, Griesemer and Thompson just didn't think generics are worthwhile enough for the effort it takes to properly research them.
When Go was started, its authors mostly looked to languages which implemented generics later in their lifecycle (namely Java and C++) and their implementations suffered from problems due to other, rather obvious, design defects:
https://research.swtch.com/generic
I'm happy to see that this attitude is changing, and other languages are looked at.
> The reality is that every modern statically typed language except Go has generics, and they all implemented them very well. The ML languages and Ada were already doing it in the 80s, and OCaml and Eiffel managed to combine generics and polymorphism back in the 90s.
> It seems to me that when originally designing Go, Pike, Griesemer and Thompson just didn't think generics are worthwhile enough for the effort it takes to properly research them.
Are you arguing that it's easy, and they just haven't done it? Or are you arguing that it wasn't important enough at first, and so the implementation developed in directions that preclude straightforward implementations now? I can believe the latter, but after reading the multiple, detailed proposals from Ian Lance Taylor, I don't believe the former.
> The only reason Go is as popular as it is is because of Google's marketing budget. Every programmer on Earth heard of Go within a few days of it's official release.
I think Google put lot of marketing budget for Dart. But I don't see it ever comes in discussion regarding popularity or lack of it.
It is fine a lot of people do not like Go but claiming its popular just because of Google seems baseless.
Well, Go is obviously not a bad language. That said, I do think that it's adoption was primarily fueled by Google's popularity. Imagine go being released by a single person or small group of people. How would people have heard about it?
I think this isn't quite true. From my perspective, Go did get a massive boost but not from Google. The language was originally designed by Robert Griesemer, Rob Pike, and Ken Thompson.
It is fair to say that without those names attached I would have likely passed it by. Google is boring, but those three names, for me at least, I had to take a look.
If Google could make a language popular, where does Dart fit into the story? People seem to love Go, and with very little encouragement from Google. I don't see a lot of love for Dart...though everyone who's played with it seems to like it well enough.
Someone said this and it feels correct: it is not a bad language, but it is also not a good language.
> Google & adoption
I think it helped get around the initial cycle of drawing in the curious. An argument counter to claims regarding the importance of the people involved would point out Plan9 etc. that are hardly widely known, much less adopted.
In sum, I think the role of Google in one becoming a Go programmer depends entirely on when it happened. Today, of course Go has its own brand name. 8 years ago, it was Google that eased the debutante phase of the language.
> Go is more versatile and at the same time more mature than anything out there.
This doesn't seem like a very credible statement on the face of it. Is Go more mature and more versatile than Python? Than Java? Than...Scala? Go does seem to share a lot of use cases (and limitations) with Java; and it would be hard to call it more mature.
> You know, it's interesting. I've been programming with Python for about 6 years now. I've also picked up Javascript, SQL, bash, and PHP along the way. ... However, need to do something in a different way than Go decides is correct? Well, you can't.
...
Pardon me and no offense, but it sounds like you are hitting the 'statically typed language' boundary.. all of the others you mention are fairly loose and dynamic. Go & C, not so much. It sounds like your use of C has been library code, which presumably is more 'data processing' oriented and so doesn't require much structure or control of process/runtime/etc.. which is where you will run into this stuff on the c side..
This is why I moved from c/c++ into dynamic languages to start with.. that said, as I grow more sophisticated and can 'deal' with the typing/lower 'machine' level control, the more I can understand other tools.. even C++! ..
i mention this because each layer of abstraction is there for a reason.. best to view with a fresh pair of eyes imho
I usually do this, but sometimes I'm writing a really long function or something and I want to save part way through so I don't lose my progress from some unlikely, yet catastrophic, failure.
It's only reliable if none of your dependencies are spawning goroutines during initialization. If this is the case, some goroutines (yours' or the dependencies') can end up with increased privileges.
The workaround functions by opening the privileged socket and re-executing the binary as an unprivileged user with access to the filehandle. Any background goroutines would exit with the privileged parent.
Also: please don't spawn goroutines during init(). There's generally a better time and place, and in the event that you justifiably need a package-level background routine you can spawn it on demand with a sync.Once.
But you also want drop that privilege after using it to get the same effect as the "daemonize dance" GP referred to. And that's also per-thread, just like namespaces.
And that's unfortunately only an issue because Linux doesn't implement POSIX's model for thread privileges. In particular, glibc has to implement the nptl in userspace to make Linux threads look like POSIX threads.
I get the feeling that the Go runtime doesn't do the same.
I've ran into this problem before. IMO this issue has nothing to do with go and the solution is straightforward.
Simply create a sub-process whenever entering a new namespace because the operation isn't concurrency safe within a process.
Note that you'd run into this bug within any multithreaded process, whether the code was written in go, Java, c, or whatever.
This is discouraging considering go was initially designed as a systems programming language. I wonder if there is another way for go to handle blocking syscall such that this use case would become reliable.
It's a common practice in C to use thread pools for concurrency and queue tasks for those thread pools (e.g. libuv). I work on a C++ code base that mainly uses Goroutine like green threads. If you use a scheme like that you're kind of in the same position (at least with respect to anything running on that pool). It's true that it's easy to "escape" and manage your threads explicitly but unless you know exactly what you're doing you could run into similar issues.
I really don't think this is a language design issue but clearly if you want absolute control you can't get that with the Go run-time.
Because this isn't really a language issue. It's an OS issue. Go assumes that all threads are equal and therefore any Goroutine can run on any thread and threads are all equal. Linux got late to the thread party and its threads are kind of like processes that haven't decided if they want to be processes or threads.
It's true the Go runtime could give users more control over goroutine and thread scheduling but that would kind of defeat the purpose of not needing to know about it and having Goroutines as the only flexible unit of concurrency.
I think the kludge here is on the Linux side. Having some magic properties bestowed upon threads doesn't make sense. The property should be accessible via a handle that can be shared amongst all threads.
What's an example of it not being true in Windows?
I think it's mostly true in Linux and the situations where it isn't true are subtle and require expert knowledge in some specific areas. It's not like there's a big sticker on Linux that says threads are generally not symmetric.
I get the bit of half the process being in one space and half in another. I just find it odd. Can half the process run as root and half the process run as another user? Maybe yes? Can a file be open in half the process and not open in the other?
At any rate, I think it's not black and white. Green threads do make sense in general and introducing different thread types and more granular control makes things more complicated.
I think you're missing my point about APIs and handles:
if handle can be used across threads then this sequence will run correctly regardless of what thread is executing it, it doesn't rely on anything bestowed on a thread.
(EDIT: thread local storage I guess is an example of asymmetry between threads but it's intentional/clear asymmetry designed for specific purposes and something you don't need to access in Go e.g. because you don't use threads directly).
Maybe your assumption of symmetry is just backwards? If you look at the clone syscall then threads are more akin to processes that just happen to share virtual memory and some other things such as IO priorities, file desciptor tables, cgroups, .... Many of those things can be shared individually. In other words they are considered orthogonal features that, when taken together, happen to function as threads.
I'd be interested to know how to do that. Go's Chdir() calls SYS_CHDIR on Darwin, and Posix requires that chdir affects the whole process.
In Linux you can unset CLONE_FS when creating a thread, but the Go runtime does not do this.
I note the Windows docs[1] say "Multithreaded applications and shared library code should not use the SetCurrentDirectory function" (which Go's Chdir() calls)
It's possible they didn't know different Linux threads can be in different namespaces, it's not exactly something everyone knows. Namespaces in Linux aren't that new (15 years or so). It's possible they knew but decided this wasn't an important enough use case to warrant language features.
I've never needed to have half my process in one namespace and half in another. It's a niche application. Green threads/goroutines is something I use extensively and I wouldn't give it up for the ability to have half my process in another namespace. There's probably some middle ground there in giving Go users more control over which thread pools run which goroutines...
Someone should file a Go bug and see what the response is...
> Someone should file a Go bug and see what the response is...
From memory, several people working on Docker have done so over the years. It's still a problem because the only "real" solutions are:
1. Do what glibc does and implement nptl(7) (effectively a way to make Linux threads look like POSIX threads by synchronising certain operations on all threads). This would require making first-class library APIs for Linux features (outside of the wild-west that is syscall).
2. Give programs far more control over threading, which would require making runtime.LockOSThread and GOMAXPROCS actually do what their documentation says. However, that would restrict their ability to be opinionated about threading (and would almost certainly cause deadlocks in some programs) so I understand why they don't want to do this either.
Even though I am a big fan of go, I've personally built two container runtimes in other languages do to the namespace clumsiness.
Personally, I think rust is an excellent alternative for namespace utilities.
EDIT: there is more information and links in the issue in the netns library: https://github.com/vishvananda/netns/issues/17