Pledge works well if the software developers implement it on their own application.
It also works well if the software developers document what syscalls they rely on and what permissions they need.
When it comes to retrofitting something like pledge (or seccomp) into an existing application when you've not developed it and/or can't easily tell what syscalls are being called then it's always a nightmare.
It doesn't really matter if it's pledge or seccomp at that point (although undoubtedly seccomp is far harder to make use of), if you're doing this kind of security by retroactive whitelist, you're going to have trouble making it work. It's going to take time and effort to implement.
> When it comes to retrofitting something like pledge (or seccomp) into an existing application when you've not developed it and/or can't easily tell what syscalls are being called then it's always a nightmare.
Quite the contrary. If the software in question has been written in a remotely sane way, adding some basic pledge restrictions is a matter of adding one line: pledge("stdio rpath whatever you need", NULL) - it usually goes somewhere in main, after setup() but before while(!quit).
You can usually figure out the permission set within a few attempts, even without a very good understanding of the internals, as most (sane) programs will do only a couple of things: an httpd needs to accept connections, read static files, write logs, etc; a window manager needs to talk to X11, open font files, etc; of course there are also complex beasts like Chrome but that one has been done as well.
The *real* challenge is breaking up a complex program (e.g. a streaming music player) into separate processes that are concerned with just one or two things, e.g. separate process to make requests over the network, a separate one to decode media, another to maintain an on-disk cache, and so on. Placing restrictions on these subprocesses is the easy part; figuring out where to draw these lines is what's hard.
I think realistically most of the things people seriously care about pledging are complex application and server software so yes the point is you run into problems trying to pledge things. Obviously you can come up with the minimum set of things you can pledge pretty quickly to get things working, but that doesn't guarantee they will keep working. And you can also be very broad, but there's a point at which that doesn't gain you much in terms of security (or as much as you may have set out to gain).
This reminds me of a situation where I tried to use firejail to isolate this proprietary piece of software, I ran firejail in the "auto-generate-something-sensible" mode and then tried running it in that profile. It would just randomly break at that point. I never quite figured this out due to lack of time. I was expecting to be able to roll with an auto-generated profile at first and tighten it later, the actual end result was there was no profile at all.
The other issue you run into is getting things which work sometimes and then stop working randomly. Especially when it's a large graphical application. It will do something strange when you click a specific button and crash. Now you are annoyed, probably not in the mood to debug this, so maybe you make a note for later. Now you have to recreate the issue under strace, figure out what you need to pledge now, and repeat.
Yes, if you're trying to pledge ls, it's pretty easy. If you're trying to pledge anything non-trivial (i.e. anything which would _really_ benefit from these security restrictions) you end up iterating a lot.
> This reminds me of a situation where I tried to use firejail to isolate this proprietary piece of software [...].
This is exactly the point where the experience between pledge and e.g. firejail drastically diverges. The entire reason why pledge is so nice, is because it assumes source access. You can use execpromises to "jail" something you don't control, but the set of promises is always going to be unnecessarily broad, as even the sanest software out there often needs a tiny backflip before it can enter the main loop. Source access also means you have the means to investigate what exactly went wrong, or to actually fix the stupidity (rather than broadening the privileges).
The amount of things you can do to a proprietary blob to contain it is pretty limited - by definition! I think using a container/VM to completely isolate it would be a better call.
> It will do something strange when you click a specific button and crash. Now you are annoyed, probably not in the mood to debug this, so maybe you make a note for later.
When you do pledge("... error", ...), the app will get errors from disallowed syscalls, rather than a SIGABRT, which is useful when you're not sure. Mis-handling an error can still blow things up, but that's a sign that maybe the overall code quality is not great. In any case, yes you do basically need to be running the application under ktrace (/strace) for as long as you're not certain.
Actually what I think would be great is a "tracing mode" for pledge, where the kernel reports violations (PID + syscall with parms + suggested promise), maybe even takes a core snapshot at each violation, but otherwise doesn't hurt the application.
> If you're trying to pledge anything non-trivial (i.e. anything which would _really_ benefit from these security restrictions) you end up iterating a lot.
Indeed, but that's a curse of complex software, not a shortcoming of pledge itself. OpenBSD introduced pledge and contained almost the entire base system within a single release cycle.
I personally think that as an industry, software is going through a crisis of explosive complexity. I see efforts like pledge as a mirror, through which that complexity stares back at us, in all of its ugliness. Blaming the mirror does not address the issue.
I was never trying to imply that pledge had shortcomings.
I don't think we're disagreeing on anything here anyway.
The point I was making is that if you've got a large and complicated piece of software, which you didn't write yourself, which wasn't written with the intention of someone implementing a syscall filter for it, you will have a bad time. It's not quite as bad as if you have the code but it's always going to be pretty bad regardless.
I think pledge is great, and the rollout was really good (I use OpenBSD for my home router and for some other infrastructure). The OpenBSD developers were in the beneficial position that they are already familiar with the source code for their base userland, they already regularly audit and maintain security improvements for it. Also noteworthy is the fact that most of the OpenBSD base is (intentionally) not formed of extremely complex software.
There's no tracing tool to build policy with pledge? Seems like an obvious area to add functionality if it doesn't exist.
Commercial tools have had it for a long time.. even automatic profiling. Either explicitly profile during a test stage, which is best, or profile-on-first-observation.
In the full automatic mode, which is not optimal but is least effort, any operation performed in the first XX minutes/hours/days are considered 'allowed behavior' and anything after that is denied. Then it will either enforce or 'wait-to-enforce' where enforcement mode only turns on if there are no policy violations in the next XX configurable units of time.
1. You really need to understand the application more than that. Does ls need network sockets? Sure does, if you have yp enabled. But this won't appear in your trace unless you trace in such an environment. (Although pledge on openbsd transparently handles this case for you.)
2. Just because a program makes a system call doesn't mean it should. Or should at that moment. A lot of late initialization can be done earlier for tighter policies. Auto traced policies tend to be extremely broad, permitting too much stuff.
And it’s not just your application, too: it’s what you depend on. Surely your command line application that works with a handful of files is fine with read/write? Your libc might be using something else like iovec or press/pwrite under the hood.
It also works well if the software developers document what syscalls they rely on and what permissions they need.
When it comes to retrofitting something like pledge (or seccomp) into an existing application when you've not developed it and/or can't easily tell what syscalls are being called then it's always a nightmare.
It doesn't really matter if it's pledge or seccomp at that point (although undoubtedly seccomp is far harder to make use of), if you're doing this kind of security by retroactive whitelist, you're going to have trouble making it work. It's going to take time and effort to implement.