The one-line summary: "Basically, no one seems to grasp that when stuff that's fundamental is broken, what you get is a combinatorial explosion of bullshit."
Exactly. A few examples.
- C's "the language has no idea how big an array is" problem. Result: decades of buffer overflows, and a whole industry finding them, patching them, and exploiting them.
- Delayed ACKs in TCP. OK idea, but the fixed timer was based on human typing speed, copied from X.25 accumulation timers. Result: elaborate workarounds, network stalls.
- C's "#include" textual approach to definition inclusion. Result: huge, slow builds, hacks like "precompiled headers".
- HTML float/clear as a layout mechanism. Result: Javascript libraries for layout on top of the browser's layout engine. Absolute positioning errors. Text on top of text or offscreen.
- The UNIX protection model, where the finest-grain entity is the user. Any program can do anything the user can do, and hostile programs do. Result: the virus and anti-virus industries.
- Makefiles. Using dependency relationships was a good idea. Having them be independent of the actual dependencies in the program wasn't. Result: "make depend",
"./configure", and other painful hacks.
- Peripheral-side control of DMA. Before PCs, IBM mainframes had "channels", which effectively had an MMU between device and memory, so that devices couldn't blither all over memory. Channels also provided a uniform interface for all devices. IBM PCs, like many minicomputers, originally had memory and devices on the same bus. This reduced transistor count back when it mattered. But it meant that devices could write all over memory, and devices and drivers had be trusted. Three decades later, when transistor counts don't matter, we still have that basic architecture. Result: drivers still crashing operating systems, many drivers still in kernel, devices able to inject malware into systems.
- Poor interprocess communication in operating systems. What's usually needed is a subroutine call. What the OS usually gives you is an I/O operation. QNX gets this right. IPC was a retrofit in the UNIX/Linux world, and still isn't very good. Fast IPC requires very tight coordination between scheduling and IPC, or each IPC operation puts somebody at the end of the line for CPU time. Result: elaborate, slow IPC systems built on top of sockets, pipes, channels, shared memory, etc. Programs too monolithic.
- Related to this, poor support for "big objects". A "big object" is something which can be called and provides various functions, but has an arms-length relationship with the caller and needs some protection from it. Examples are databases, network stacks, and other services. We don't even have a standard term for this. Special-purpose approaches include talking to the big object over a socket (databases), putting the big object in the kernel (network stacks), and trying to use DLL/shared object mechanisms in the same or partially shared address space. General purpose approaches include CORBA, OLE, Google protocol buffers and, perhaps, REST/JSON. Result: ad-hoc hacks for each shared object.
The anti-virus industry is a result of DOS and Windows not having a unix permission model. Running as the user with highest privileges was the norm.
Now that a unix permission model is the norm, viruses are comparatively gone and replaced by malware that simply tricks the user into installing it. No permission model will help you against this. As a partial result, now we see things like iOS where we remove control from the user, or OS X where we try to make it inconvenient to be duped into giving access.
There are certainly still exploits that don't require duping the user, but the anti-virus industry certainly wasn't established based on these.
That's not really true. No permission model will be 100% effective, but a more fine-grained permission model might lead to more users saying "Um, no, mysteriously executable pornography, I don't want to give you my bank records and the ability to email my friends."
Let's imagine that I have privilege grouping sub-users, something like name.banking, name.work, etc. Now my work files can't see my banking unless a window pops up going "Would you like Experimental Thing for Work to have access to name.banking?"
I think being able to explain to the computer how my data is grouped, and access patterns in it, is more natural for users than most of the security models we have today.
It's also much easier to have two copies of the browser load, depending on if I'm invoking it through name.banking or name.general. And much easier to explain to grandma you do banking when you use name.banking and you look at cat photos in general.
Grandma isn't stupid, she doesn't understand how technology work. Making permissions based around how she categorizes her information and how she divvies up tasks is more natural for her than insisting security only work if she understand how computers work.
I said it's not going to reach 100%, probably whatever we do. Probably there are ways to improve on the Android and iOS permission models - there was talk in another thread about "deny this permission but fake it" options, there are probably ways things can be presented better, there might be ways permissions can be divided better, &c... Manifestly, there exist plenty of users that don't pay enough attention to what permissions they're granting. I wouldn't be surprised to learn that it's an improvement over user behavior patterns on user-account-only permission systems, though.
The UNIX world was hardly a model of security until somewhere around 2000. Both HP-UX and Irix of that era could be hilariously insecure, with it being utterly trivial to break through the permissions model.
Thanks to UNIX boxes being the bulk of the always on systems attached to the internet at that time they presented most of the attack surface, and consequently an industry of people to attempt to protect them.
No they aren't. There's tons of them. You don't need to be admin for a virus to be a problem. All the data a user cares about is owned by that user anyways. There's plenty of "haha I encrypted your files, pay me if you want to access them ever again" extortion viruses.
All the instances of these that I've seen rely on social engineering to do their thing though (we had a teacher at my school fall victim to one recently, which is moderately entertaining [when you have up to date backups] when you have a bunch of read/write network shares), as opposed to regular files/executables 'infected with a virus' which is how I generally look at viruses in the traditional sense.
Lots of viruses used social engineering since the start. The only difference now is that once run, it doesn't have admin privileges, so it is harder for it to make itself a permanent fixture on the system.
Not for the vast vast majority of users. An OS or programs are by far the easiest things to obtain; modern windows or mac also ship with rescue images. What I'll miss from my drive are the documents I've created, work I've done, pictures, moves, etc. Per-user protection -- such as using root to install -- protects the OS or programs, but helps not at all with anything that's painful if it gets lost.
./configure is not for dependencies, but for platform configuration. Consider that ./configure is used before you do a full build. A full build doesn't need dependencies; an incremental one does.
The configure script is hack to solve another "worse": no direct way to get pertinent platform information from the C environment, like what functions are available, how big is some type and so forth.
You can get the size of a type, the problem is you can't get it at preprocessor time. Which is the result of another problem: the C language operates in two phases: preprocessing and compiling (three, if you count linking). Preprocessing, compiling, and linking phases are walled off from each other, resulting in problems. (Preprocessing shouldn't even be a thing that you do to most source code.)
My understanding is that other languages that don't have this separation of compiling objects and linking them together, can't do incremental builds, and are generally quite slow to build.
C++ is infamously slow to build. I've seen the results of profiling Clang (known as a particularly fast C++ compiler) and the preprocessor takes a big chunk of time. C compiles a fair bit faster.
Try out the Mono compiler for C# some time. It is so fast that you might as well recompile your entire project every time you change one line of code. I'd pay serious money to get that kind of performance from a C++ compiler. Tons of other compilers are really fast. JIT is fast all modern browsers. Go compiles in a snap. Python starts running immediately.
The only other language I use with a compile time comparable to C++ is Haskell.
> Poor interprocess communication in operating systems. What's usually needed is a subroutine call.
Across languages? Even if you stipulate that all languages you care about have the same notion of "subroutine call", how do you portably handle data marshaling?
Marshaling is an important, and neglected, subject in language design. Compilers really should understand marshaling as a compilable operation. In many cases, marshaling can be compiled down to moves and adds. Done interpretively, or through "reflection", there's a huge overhead. If you're doing marshaling, you're probably doing it on a lot of data, so efficiency matters.
For Google protocol buffers, there are pre-compilers which generate efficient C, Go, Java, or Python. That works. They're not integrated into the language, so it's kind of clunky. Perhaps compilers should accept plug-ins for marshaling.
Most other cross-language systems are more interpretive. CORBA and SOAP libraries tend to do a lot of work for each call. This discourages their use for local calls.
Incidentally, there's a fear of the cost of copying in message passing systems. This is overrated. Most modern CPUs copy very fast and in reasonably wide chunks. If you're copying data that was just created and will immediately be used, everything will be in the fastest cache.
Fortunately, we can generally assume today that integers are 32 or 64 bit twos complement, floats are IEEE 754, and strings are Unicode. We don't have to worry about 36-bit machines, Cray, Univac, or Burroughs floats, or EBCDIC. (It's really time to insist that the only web encoding be UTF-8, by the way.) So marshaling need involve little conversion. Endian, at worst, but that's all moves.
> Marshaling is an important, and neglected, subject in language design. Compilers really should understand marshaling as a compilable operation.
I generally agree with what you're saying.
This is one of the things COBOL, of all languages, generally got right: You had a Data Definition language, which has been carried over to SQL, and the compiler could look at the code written in that language to create parsers automatically. Of course, COBOL having been COBOL, this was oriented to 80-column 9-edge-first fixed-format records with all the types an Eisenhower-era Data Processing Professional thought would be important.
The concept might could use some updating, is what I'm saying.
> Most modern CPUs copy very fast and in reasonably wide chunks.
And most modern OSes can finagle bits in the page table to remove the need for copying.
> strings are Unicode
By which you mean UTF-32BE, naturally. ;)
> It's really time to insist that the only web encoding be UTF-8, by the way.
This might actually be doable, if only because of all the smilies that rely on Unicode to work and the fact UTF-8 is the only encoding that handles English efficiently.
And most modern OSes can finagle bits in the page table to remove the need for copying.
That tends to be more trouble than it's worth. It usually means flushing caches, locking lots of things, and having to interrupt every CPU. Unless it's a really big data move (megabytes) it's probably a lose. Mach did that, and it didn't work out well.
What's usually needed is a subroutine call does not imply it can only be satisfied with a subroutine call, but that an abstraction that looks and feels like a subroutine call is preferable to one that looks like a stream of bytes.
And the point is exactly to address data marshalling, which is a hard enough problem that reducing the number of applications that have to independently solve it would be a great benefit.
That most developers don't even tend to get reading from/writing to a socket efficiently right (based on a deeply unscientific set of samples I've seen through my career) implies to me we really shouldn't trust much developers to get data marshalling right.
Worst case? your app falls back on using said interface to exchange blocks of raw bytes if the provided model doesn't work for you.
I agree with you regarding all of the above, except for the DMA bit. That has to do with crappy drivers causing crashes. Without direct DMA you'd have horrendous performance issues. It is not a transistor count issue.
These days pluggable devices (SATA, USB) don't get DMA access. Only physical cards do (PCIe, etc.) -- again because of performance issues.
Some machines have had an MMU or equivalent device between peripheral and memory, to provide memory protection. IBM's channels did that. Some early UNIX workstations (Apollo) did that. But it has sort of disappeared.
Both FireWire and PCIe over cable expose memory via a pluggable interface. In the FireWire case, it's not really DMA; it's a message, but the ability to patch memory is there. FireWire hardware usually offers bounds registers limiting that access. By default, Linux allowed access to the first 4GB of memory (32 bits), even on 64-bit machines. (I once proposed disabling that, but someone was using it for a debugger.)
I don't know about IBM channels. PCIe root has the ability to restrict transfer to a certain range and depending on system configuration there are remapping registers that translate between PCIe address and host memory address -- which you can fudge with to remap things how you like.
Firewire, was basically external PCIe (before there was PCIe) and you would be able to do DMA and there was a proof of concept of someone using an early iPod to read/write host memory.
You can't with things like eSATA or USB. There is no DMA capability for the external device to exploit. The host controller (EHCI and alike) are the ones doing the DMAing. You can't write directly to memory with those. Of course USB is exploited by doing things like descriptor buffer overflows.
I feel like LINQ is attempting to do something for the "big objects" problem. It's currently really for database access, but I've seen a bit of talk about extending it to other resources, e.g. hetrogenous parallel processors. I could see an interesting future with tools like LINQ providing simple, backend agnostic accessors to computing resources, e.g. network access, database access, file access etc...
Exactly. A few examples.
- C's "the language has no idea how big an array is" problem. Result: decades of buffer overflows, and a whole industry finding them, patching them, and exploiting them.
- Delayed ACKs in TCP. OK idea, but the fixed timer was based on human typing speed, copied from X.25 accumulation timers. Result: elaborate workarounds, network stalls.
- C's "#include" textual approach to definition inclusion. Result: huge, slow builds, hacks like "precompiled headers".
- HTML float/clear as a layout mechanism. Result: Javascript libraries for layout on top of the browser's layout engine. Absolute positioning errors. Text on top of text or offscreen.
- The UNIX protection model, where the finest-grain entity is the user. Any program can do anything the user can do, and hostile programs do. Result: the virus and anti-virus industries.
- Makefiles. Using dependency relationships was a good idea. Having them be independent of the actual dependencies in the program wasn't. Result: "make depend", "./configure", and other painful hacks.
- Peripheral-side control of DMA. Before PCs, IBM mainframes had "channels", which effectively had an MMU between device and memory, so that devices couldn't blither all over memory. Channels also provided a uniform interface for all devices. IBM PCs, like many minicomputers, originally had memory and devices on the same bus. This reduced transistor count back when it mattered. But it meant that devices could write all over memory, and devices and drivers had be trusted. Three decades later, when transistor counts don't matter, we still have that basic architecture. Result: drivers still crashing operating systems, many drivers still in kernel, devices able to inject malware into systems.
- Poor interprocess communication in operating systems. What's usually needed is a subroutine call. What the OS usually gives you is an I/O operation. QNX gets this right. IPC was a retrofit in the UNIX/Linux world, and still isn't very good. Fast IPC requires very tight coordination between scheduling and IPC, or each IPC operation puts somebody at the end of the line for CPU time. Result: elaborate, slow IPC systems built on top of sockets, pipes, channels, shared memory, etc. Programs too monolithic.
- Related to this, poor support for "big objects". A "big object" is something which can be called and provides various functions, but has an arms-length relationship with the caller and needs some protection from it. Examples are databases, network stacks, and other services. We don't even have a standard term for this. Special-purpose approaches include talking to the big object over a socket (databases), putting the big object in the kernel (network stacks), and trying to use DLL/shared object mechanisms in the same or partially shared address space. General purpose approaches include CORBA, OLE, Google protocol buffers and, perhaps, REST/JSON. Result: ad-hoc hacks for each shared object.