Maxime Villard has been doing great work finding lots of bugs, many with a code scanner he wrote called Brainy. As an OpenBSD user I've been watching with interest from the sidelines. More info on Brainy results:
How does one go about conducting such audits? Is it just a case of looking at the code long with an experienced eye to spot potential flaws? Or are there standard techniques and tools that people follow?
The simplest and probably the most common is to simply read the code. With experience, patterns of vulnerabilities pop out at you; also, you get good at quickly eliminating code that isn't likely to be in the critical path of a vulnerability, allowing you to focus on the code that is.
Some people start from patterns and work backwards through code looking for reachability. 10 years ago, you might have done that by sweeping for memcpy's and working back to inputs. Nowadays you'd probably be looking for pointer/length math, or allocations and frees.
A primitive automation technique is get the target hooked up to a debugger and then run a blind fuzzer at it looking for faults. More advanced fuzzers will track basic block coverage and tune inputs to reach new paths.
If you're really committed to a particular project (like, someone's paying you to do it, and you have a fair bit of time), you might hoist the code out of its natural setting and build a test harness for it, so you can fuzz it "in the abstract" without having to deal with any of its surrounding code. I've done that for OS work before (you'd be surprised how much of a kernel you can "port" back to userland, if all you need to do is exercise logic).
Peers of mine have also instrumented virtual machines like QEMU to do this kind of testing.
Finally, the most modern way to do this kind of work is to generate models for the code and then use provers to verify things about them. SMT solvers were all the rage for this a few years ago; my sense is that code proving tools have gotten more interesting than that since then.
Right, I'm not being very precise; you don't necessarily want the whole kernel (because then you have to model inputs for all the interactions with all the kernel subsystems), but rather to isolate the one component under test.
1.Invalid inputs potential:
Inputs->outputs. If you can stuff invalid input and the code can't detect it or crashes, its a vulnerability.
Fuzzers automate this step by creating lots of hostile/invalid input state.
2.Off-by-one,length/stack/buffer-overflows: check the length of input/buffers, does the code handle all cases?
C memory leak detectors/memory sanitizers can detect this.
3.Logic/Design flaws: Does code path assumes something that can be changed before or after execution? Does the code assumes a fast path that requires all circumstances to be correct?
Does the code ignores something that influences later execution(e.g. generic approach where special handling is required, special handling deferred to later step, losing global state information(e.g.with function calls))?
Static analysis tools will find most of this.
4. Exceptional/Corner cases: is the code exhaustive?
Can it handle all states of program? Usually the error is programmer assuming X can't happen and optimize it out, but due code evolution X suddenly becomes possible or was possible with exotic configuration/state.
This often can't be automated even with static analysis tools because is program-specific knowledge.
Rewriting it in Ada/rust/ocaml might be an alternative: they allow to specify exhaustive compliance with requirements and check all state combinations at compile time. Functional programming can also handle this with additional overhead and restricting the program global state. Security-centric stuff should not be manual:
C requires to do it manually or introduces run-time asserts.
The reason world doesn't run everything in Ada or equivalent, is that flexibility and speed are often as critical as security:
network stack that is 100% proven secure but twice slower will be replaced by network stack that is 99% secure and is twice faster. Same applies to most latency requirements:
if you program is secure but introduces X latency, it will be replaced by faster stuff that works reasonably well.
I guess a lot of desktop users would be very fine if they had 100% proven secure hostile-environment-exposing network stacks as long its able to handle some dozens of mbits. However I don't think that a proven secure network stack will be so slow, so that's interrupting normal network experience.
Most users don't see security, they see speed and latency.
Example for gaming: why are most games in C++?
C++ provides fastest speed/latency from the hardware without low-level effort.
In all areas where real-time control is critical, speed/latency are most important.
Imagine a Java GC stopping a program for a few milliseconds to recover memory, while it controls a fighter jet.
A server that is 30% slower because its written in Ada.
A program that requires 50% more memory, exceeding physical RAM and starting to swap.
Even C sometimes not fit to the task of extracting maximum performance, especially on embedded/mobile devices. Assembler(which is far less secure than C) is still used for speed-critical loops, audio/video decoders, cryptography and compression.
Users can sometimes see security failures (e. g. ransomware, data loss) as well as performance failures.
If it's only about speed you could argue games could have been written in C, I think C++ is used because it give the best trade-off between performance and productivity/software maintenance.
Imagine that people perfectly lived with lower computing resources few years ago and a small amount of performance gains could be traded in favor for security leading to a world with less security nightmares.
I think Javas misplacement in safety-critical or real-time-critical environments is not primarily because of its low performance, but because of its non-determinism. The most important thing in real-time applications is not primarily performance but rather timeliness.
Safety-critical or mission critical environments (aviation, military) often prefer Ada or its safer subset SPARK over error-prone languages such as C.
In theory it should be possible to conduct such audits ("assessments" probably fits better) in a scientific and thorough manner, so that the process and its result are replicable. I assume that in most cases software security assessments are not done in this way probably resulting in the finding of some bugs but no true assurance statements of the code itself.
If somebody here can give some references on the state of the art of software security assessments I would be very thankful for that.
Standards are sort of a double edged sword in infosec, because:
* if I can just run a scripted audit and go home, then any dipshit can do that and security professionals won't have jobs. Or fewer ones will. Which means less infosec software gets produced.
* if an attacker has access to that script, it's more or less a matter of time until they develop an exploit to circumvent it, slightly dependent on their skill. Eg, industrial software might be in less danger here because the exploit surface is limited to in-facility, if their network stack is good.
So, patterns in general are a smell worth investigating. If something works, I try and find out if my PM will gift me some time to automate and smoke test what I come up with. If something never works, I learn, but I also automate that too, and generally just try it anyways.
Tooling is more or less standard, kali linux, unix tools, and then whatever they've found to work.
Maybe eighty percent of what I use is basic kali toolbox stuff w/ whatever extensions I've collected, and the rest is tools I wrote for mapping, measuring, and muscling my way to success. Lots of old python, some new python, tons of C, I've got perl scripts laying around, I've got afl installed somewhere, I use lua for some things.
Memory errors are common security issues. There's tools like valgrind to help find those.
I don't really know how data race stressing is done, but I'm sure that's a common route in finding network stack bugs.
Afl-fuzz finds a lot of data processing bugs, off by ones, pointer/deref issues, etc. They've found some pretty big ones, too.
When I am handed code to audit, I'm looking for areas where I can tell the author was rushed, or where he was copying code from somewhere. The more the better, I want to know what he thinks like, how he writes, and that helps me find something that would be difficult for him to implement.
Sometimes you get a big obfuscated mess and you gotta try and make a rough 50,000ft map of it and start profiling, even stepping through with qemu if that's what it takes to get a clue.
Infosec is one of those devilish details kind of fields. Once you get the feel for how things break, then it's mostly rote work. Sometimes you just gotta trust that if you read enough code you'll find a piece that sucks enough for you to break something.
Sometimes you don't find anything. But I'm not a researcher, so I'd then move on to other pastures, whereas the research auditor presumably has to write a report or talk to his peers?
>I stumbled across a PR from 2010, which was briefly saying that PF’s TCP-SYN entry point could crash the kernel if a special packet was received. Looking at the PF code, it was clear, after two minutes, where the problem was.
I wonder how often problems like this languish in bug trackers (due to a shortage of developer availability? flood of low-quality bugs? other problems?)
Maybe such things could be solved by having a bug tracker linked to a sort of CI system where you could upload code/tests/etc to _demonstrate_ the bug in question
That helps, but it still requires someone knowledgeable to look at the demonstration code and confirm it is correct (trivial for crashes the kernel issues with demonstration code running in userland, because all of those are real, however ugly and nasty the code demoing it is, but less easy or even very hard for anything else. For example, leaks memory can be hard to discriminate from uses lots of resources)
If there are many such demos, many of them red herrings (users misunderstanding APIs, for example), developers may choose to only investigate reports from people they trust to report real issues.
Since he is paid to work on NetBSD, I wonder if his work on OpenBSD and FreeBSD will earn him anything. But really glad they are shared among BSD lands.
Off Topic Questions, is NetBSD still being used in Appliance? Apple AirPort used to use it as OS, but now it has been discontinued.....
This is great news. Since they took a beating in the last security findings in code that was not even enabled by default. Leading to it seeming much more serious than it was.
If we want anybody to buy into it, this needs to not crater performance. Now, in some cases kernel bypass is actually used to increase performance, so I have hope; we just need a fast communication method in user space so other programs can use our new stack. And for adoption, we should ideally be able to intercept existing network calls (library override?).
...huh, that actually sounds plausible. Have I missed anything?
https://www.m00nbsd.net/474b193ce624a19d39a2286d73f06826.htm...
Take a look! There is likely software that you use in the list - for instance, one of the bugs is a memleak in OpenSSH.
I don't use NetBSD but I just donated $20 to the NetBSD Foundation anyway. Thanks for sharing code/patches!