Yikes. This looks like exactly the kind of patchwork hack (in the "kludge" sense of the word) that keeps later on biting people in the ass over and over and over again.
So let's say you have a job for which FreeBSD + EC2 is the right answer (or you just really like FreeBSD + EC2), so you make use of this, and then it works for a while and works well so you begin to build some kind of infrastructure on it, and even though you're paying extra for it, you don't mind because it works.
The FreeBSD project in the meanwhile gets a little pressure off of their Xen paravirtualization efforts, which bothers people that would like to use FreeBSD under Xen not-on-EC2, but maybe that's not a big deal.
Eventually Amazon changes something, since this is exploiting a not-officially-supported "feature", and the whole mess stops working. People who were relying on FreeBSD-on-EC2 are hosed, and they're probably not a big enough market for Amazon to care very much about (although Amazon does have better support than most companies their size, so maybe they'd do something about it), FreeBSD still has poor Xen support, and everybody's standing around with their pants down.
Not very good long-term thinking.
Nice hack though. But I think you'd have to be a little wet-behind-the-ears to build much on it.
The FreeBSD project in the meanwhile gets a little pressure off of their Xen paravirtualization efforts
That's not how it works. It's quite the opposite, actually -- the more people are using a system, the more money is available to fund further work on it.
Eventually Amazon changes something, since this is exploiting a not-officially-supported "feature", and the whole mess stops working.
Amazon is the most incredibly paranoid about not breaking existing systems of any company I've ever seen. (I sometimes wish they were less so, actually -- I've had to work around "this was fixed in Xen years ago but EC2 is running an old version for compatibility reasons" bugs.)
I don't think FreeBSD is big enough to get any sort of special treatment from Amazon, but I'm pretty sure Amazon wouldn't want to set a precedent for "Amazon broke my website".
Two reasons: First, this is a lot of work, and in really scary parts of the kernel. Second, 99% of places using Xen are now running HVM, so improved PV Xen support isn't as useful as it sounds.
Not really. The only time it would break is if Amazon moved Windows to hypervisor virtualization. When they're doing hardware virtualization, practically any OS should work and without much "kuldginess"
Colin's not suggesting some crazy hack, it's exactly the same as buying a Dell, formatting it to remove Windows, and then install BSD instead.
This is never going to happen though. Not because Windows PV is not possible. Windows was ported to Xen PV in the distant past because Cambridge University had a Windows source license.
The reason it's not going to happen is because with the latest chips and 64 bit, hardware full virtualization is faster than paravirt. Paravirt on x86-64 can't use the i386 segmentation trick, because x86-64 (long mode) doesn't have full support for segment registers. So it has to use some other trick which is slower.
I like the name of "defenestration" for this technique. Particularly with the OED reference to back up the technically non-standard but still precedented usage of the term.
This is a great news, thanks for your efforts on this!
However, it's really sad to have to pay the Microsoft toll over FreeBSD (afaik Windows EC2 Instances are a bit more expensive because the cost of the OS licence is factored into the hourly cost).
I hope Amazon will do something about this, like providing HVM-based Unix instances.
why doesn't amazon just have a HVM flag on instance creation?
Probably because they didn't need it. Amazon is the most customer-centric company I know, but this cuts both ways -- I've never seen them add a feature unless they have customers asking for it.
Something to keep in mind: HVM is generally slower(even with VT-x) than paravirtualization...It's not by a huge amount, but you'll definitely be seeing higher IO latency. While this is a neat hack, you're paying more and getting poorer performance.
HVM is generally slower(even with VT-x) than paravirtualization
Not in this case. The problem you're talking about is with emulated hardware; EC2 uses paravirtual devices with HVM, so the performance there is identical to completely paravirtual instances.
So let's say you have a job for which FreeBSD + EC2 is the right answer (or you just really like FreeBSD + EC2), so you make use of this, and then it works for a while and works well so you begin to build some kind of infrastructure on it, and even though you're paying extra for it, you don't mind because it works.
The FreeBSD project in the meanwhile gets a little pressure off of their Xen paravirtualization efforts, which bothers people that would like to use FreeBSD under Xen not-on-EC2, but maybe that's not a big deal.
Eventually Amazon changes something, since this is exploiting a not-officially-supported "feature", and the whole mess stops working. People who were relying on FreeBSD-on-EC2 are hosed, and they're probably not a big enough market for Amazon to care very much about (although Amazon does have better support than most companies their size, so maybe they'd do something about it), FreeBSD still has poor Xen support, and everybody's standing around with their pants down.
Not very good long-term thinking.
Nice hack though. But I think you'd have to be a little wet-behind-the-ears to build much on it.
(Much love to FreeBSD, not trolling, etc. etc.)