Hacker News new | past | comments | ask | show | jobs | submit login
A long two months: Reflecting on mitigating Meltdown (cooperi.net)
114 points by bcantrill on Feb 24, 2018 | hide | past | favorite | 15 comments



segmentation is the closest feature really, and it was thrown out for 64-bit AMD64

That was one of the "big things" with the 286 and protected mode, and enhanced even further with the 386, so it's a bit ironic and sad that it can't be done in 64-bit mode --- and I'm sure it could be done in a backwards-compatible manner if AMD chose to (double-wide descriptors seem to be a reasonably popular thought), but it feels like a lot of the decisions made with AMD64 were more just to spite Intel than benefit anything or anyone else.

Related: http://www.pagetable.com/?p=25


It's not so insidious as you suggest (from your link):

VMware chose not to support 64 bit virtualization until AMD reintroduced (optional) segment limits on later models of their Opteron and Athlon 64 CPUs. Intel never implemented 64 bit segment limits on their EM64T/Intel64 CPUs, because their 64 bit processors soon implemented VT/Vanderpool, which also worked around the problem. So this is why VMware requires a certain model and stepping of the AMD CPU line or a VT-enabled Intel CPU in order to support 64 bit virtualization.

Intel was the one that dropped segment limits, AMD added them in later...

It sounds more like advances in hardware virtualization yielded better solutions than a bunch of additional segment registers and slow context switches. It's only now that we realize without those you can trick a processor into fetching memory it shouldn't and that the only viable software solution has been to effectively implement separate page tables and enforce copying between their stack contexts at the user/kernel boundary anyway (so we're back where we started with slow context switches).

In theory, you could mitigate specter by running every process in its own hardware virtualized address space, right? It's just much more complicated to do that because you'd need to move the kernel to the hypervisor ring or rewrite everything to communicate entirely using a shared physical resource like a network interface. If VT-x allowed shared memory regions with ownership restrictions (does it?) you wouldn't even need to do that.


> Intel was the one that dropped segment limits, AMD added them in later...

I was there (at VMware). That isn't quite what happened.

We got an early engineering prototype of the AMD64 chip. Ran at like 400Mhz or something, and was in an impressively massive case.

It caused two big problems for us: 1) as you note, it disabled segment offsets on all segment registers except %FS and %GS, which are commonly used for thread local storage. And much worse for us, it disabled segment limit checking entirely, which we used to protect the virtual machine monitor (VMM) in the guest's virtual address space. 2) it disabled LAHF and SAHF instructions, which cause a pretty noticeable performance regression in the binary translator (BT).

AMD was pretty helpful when we notified them, and Rev D and later of their chip had a special "VMware bit" that re-enabled segment limit checking and LAHF/SAHF. Intel was less helpful, and told us to just wait for Vanderpool (later renamed VT-X). Early versions of Vanderpool were actually slower than the traditional BT VMM. And annoyingly, for years VT-X didn't support real-mode or big-real-mode, so we had to keep emulation/BT around for that, which is really only used for the first half-second of bootup. AMD's Pacifica got this right from the start.

Ole and some of the other early monitor hackers wrote a paper that covers some of this history if you want the gory details: https://www.ece.cmu.edu/~ece845/docs/vmware-evolution.pdf


"t sounds more like advances in hardware virtualization yielded better solutions than a bunch of additional segment registers and slow context switches."

The most secure OS's of the past used both paging and segments but for different things. Later on, the numbers I saw on Atom put segments at twice as fast as paging which matters given too much overhead makes people use mechanisms less. Finally, both Native Client and Code Pointer Integrity used segments for app-level protection which could run in VM's isolated by other mechanisms.

So, they still have value given simplicity, efficiency, and fine granularity. I didnt like seeing them go given new developments were happening on segment-based designs.


Every process already does run in its own virtualized address space, without VT-x. That's what virtual memory is and does. Spectre can cross process boundaries at points where processes have the same memory mapped into their private address space (to wit: shared libs). If you enable VT-x VMs to share chunks of memory, you'll have the same damn problem. (This may be what "Spectre type 2" is, I'm not sure).


I thought the issue was that under the same kernel all pages mapped into memory can be read because the processor does not perform the virtual memory checks for the speculatively executing code...which is why the fix involves unmapping pages on context switches. VT-x gives you "physical" isolation not virtual, I thought. Like two VT-x spaces don't share the same page table mappings, do they? I could be completely wrong.


> I thought the issue was that under the same kernel all pages mapped into memory can be read because the processor does not perform the virtual memory checks for the speculatively executing code...which is why the fix involves unmapping pages on context switches.

That's Meltdown. In Meltdown's case, the processor does not perform the check to see whether a mapped page is privileged, kernel memory (ring 0) or unprivileged, user space memory before speculatively executing it, and that's where the need to unmap kernel pages before leaving a syscall (and remap them before entering one) comes from. Spectre is confined to a single process's address space, so assuming no processes share code, an attacker would only be able to leak information from within the same process via Spectre. Which is enough a vulnerablility to warrant concern, as with precise enough timers you could exfiltrate information (passwords, etc.) from anywhere in a browser process via JavaScript. The problem is, processes share code all the time, via shared libs, which makes Spectre attacks able to cross process boundaries.

VT-x is a hardware virtualization layer. It doesn't provide some sort of physical barrier to different contexts running on the same CPU wired to the same RAM leaking memory.


> The problem is, processes share code all the time, via shared libs, which makes Spectre attacks able to cross process boundaries.

I don't think that is how it works. Just because multiple processes map the same code (read-only) doesn't mean that they also have mappings to each others' data. If two processes shared mappings to the same data (e.g. after a fork() with no exec()) then what you say may be true.

Just because I can write a program that uses libc doesn't mean that I can use that as a means to extract ssh keys from an ssh-agent that also is using libc.


Actually, it does. By warming up the branch-target buffer to call a particular function (called a "gadget") inside libc as the attacker, you can induce the victim process to speculatively execute that function, and thereby leak data. It's a bit more complicated and involved than Meltdown, or Spectre within the same process, but it is possible and depending on the gadget chosen, any data within the victim process could be recovered.


I think you are now confirming what I was saying. The attacker uses a gadget in libc (or any other code) to cause side effects that are effective for reading memory within the process's address space.

Since each process has its own address space, this does not allow process A to read memory from process B's address space. For this to happen, process A (the attacker) would need to interact with process B to cause the speculation to happen within process B. Or, as stated in the spectre paper:

> Drawing from return- oriented programming (ROP) [33], in this method the attacker chooses a gadget from the address space of the victim and influences the victim to execute the gadget speculatively.

Speculation performed within process A (my web browser) doesn't give you access to the data within process B (ssh-agent).

Maybe I misunderstood the intent of your sentence that I initially quoted.

[edit to fix formatting]


Process A uses repeated calls to the gadget to trick the CPU into thinking "hey, that's a frequently jumped to place, I'd better remember that for speculative execution." Then when Process B runs, when it comes up on a branch instruction, it calls the gadget speculatively, bringing with it information about process B across the call. This may require some interaction with Process B, but it's minimal (opening a socket that Process B is listening on might be enough).

Whatever computation resulted from the speculative call is discarded BUT its effect on the cache is NOT. Process A can thus use timing information to determine what's in the cache and thus, exfiltrate data from process B.

So no, you can't read memory across process boundaries directly. The CPU's virtual memory mechanism provides at least that much. The significance of Meltdown and Spectre is, that isn't enough. Using speculative execution, cache trickery, and timing-based attacks, information CAN be exfiltrated from a process even with virtual memory protection in place.

If this gets you paranoid about your ssh-agent leaking data, good. One (relatively) easy way to mitigate this is to use the retpoline compilation technique, which replaces function calls with a RET to the function location. Because the target of a RET, being on the stack, is unknown up until the moment it happens, current CPUs do not speculate across a RET. (Though were it not for Meltdown and Spectre, maybe future CPUs might have made some guesses?) So Process B could not be induced to speculatively call the gadget if it were compiled with retpoline.


Using Meltdown can you read memory across VT-x boundaries, though?


Unlikely, because the VM probably doesn't have fine-grained control over which system calls are made when, which is necessary for a Meltdown attack.

VMs are, however, suscpetible to some forms of Spectre attack, allowing code within a VM to exfiltrate memory contents from other VMs, or from the host machine.


So I guess if the cash itself is not flushed between switches in execution across VT-x boundaries, then data could leak in the same way?


Basically, yep.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: