Many thanks to shykes, mdaniel, backjlack, and the many others that contributed to this terrific release. So much has been accomplished in just a few months, which augurs well for what Docker will look like by year's end. Bravo, everyone.
Feels like there is some wheel-reinventing going on here. Maybe someone could help me out on why the build scripts here would be preferable over something like Chef to manage the repeatable builds within a new container? Or maybe I've totally missed the point.
If someone is able to accomplish what they want using the new build script functionality, why would they want to incur the learning curve, overhead, and complexity of Chef? Don't get me wrong — Chef is a powerful tool and has its place. But for many use cases, it's totally overkill. I think Docker's new build scripts may fit the bill nicely for a sizeable number of folks without needing to figure out how to integrate with an external configuration management system.
Edit: As shykes mentions, the two are really more complementary than overlapping. I'm planning to use Docker's new build scripts to construct pre-built Docker images, which I will subsequently deploy via Ansible (a configuration management tool akin to Chef). Up until today I was building Docker images via Ansible, but now I can handle that within Docker itself, freeing up Ansible to handle tasks better suited to its bailiwick.
Can you post your sample before/after configs? It'd greatly help me understand how Docker can help me, as I can't run it on any of my machines currently (although I just retried it and the "unexpected EOF" error seems to have gone away).
Dockerfiles are not a replacement for chef or puppet scripts. They're a very lightweight way to assemble a container by running unix commands, and then freezing that container into an immutable, portable unit of deployment. Some of thos unix commands may be calls to Chef.
If you're a chef user, you can use docker to "just-in-time" compile your chef recipes into a container almost as portable as a static binary. You can then send that container to me, and I'll be able to run it regardless of what configuration management I like to use on my machine.
It seems to me that you should be able to roll back an built via a Dockerfile back to any step in the build, and then rebuild from that point. A deployment via another CM tool would probably lose that ability.
Perhaps it's my inexperience, but I find chef scripts to be very fragile because of the large amount of external dependencies they generally contain. All of these dependencies are very stable: nobody takes down tarballs and packages randomly, but a large number of low probability events can add up to a high probability.
That makes image based deployments appealing, but you still need to have a reliable, repeatable capture of the steps that went into building the image...
I've used Chef extensively and found this to be exactly the case.
In my experience, going back and running deploy scripts I wrote a year ago have about 10% probability of still working. Aside from chef's breaking changes (and versioning mess), there's a ton of things outside of my control that take things down. Debugging has been a nightmare.
I love the concepts and community around Chef, but I've found for my use case, it was an all around bad choice.
> then freezing that container into an immutable, portable unit of deployment.
Doesn't doing that make a git repo of all the files currently in the container? Wouldn't it take ages to upload this to your machine, versus just uploading a dockerfile and having it rebuild the image from scratch?
I get that Docker containers are sort-of VMs, but I don't really understand the freeezing functionality at the moment... It seems to me that you'd have to freeze the entire binary image, whereas an ansible script/Docker build file would be much more portable and result in the same thing.
Ah, errnoh on IRC also explained some things to me, it seems that Docker reuses "parent" images, so if you launch 10 Ubuntu containers they will only take up 10x20mb plus the parent image of 600 mb, rather than 10x600, hence the freezing.
That's amazing, it means that I can replace my small VMs with Docker containers on a big box.
I think the confusion is the difference between a box and a container: I grok that I would use a dockerfile to configure a container, and I would use chef/puppet to configure a box. But, _why_ can't I use chef/puppet to configure a container? Why is a dockerfile (in concept or syntax) _better_ than chef/puppet?
I think the fundamental difference is that a container is built from the _outside_ (the way you would build an executable), whereas a box is configured from the _inside_ (the way you configure a physical computer). Docker is designed for the former, and Chef is designed for the latter.
What causes the confusion, I think, is Docker's premise that the best executable format for software is a virtual computer encapsulating all of the program's dependencies. Because of this, the software you're running starts looking a lot like the computer it's running on - because they're both computers! It's just computers running on computers running on computers, or as Alan Kay liked to say it, "Real Computers All The Way Down" [1].
I believe things are bound to get more confusing for a while, but the result I believe will be one of computing's most exciting development.
There surely is an overlap. Chef is meant to manage an heterogeneous environment of machines. Things like per-operating-system templates and attributes for example don't really make sense if you're targeting a single container.
It all depends on the use case. You may have the luxury of building all your containers from a single OS distro - but others might use containers to automate the testing of the same build across a large matrix of distros and application versions. In that case using Chef would allow them to keep the variation between Dockerfiles minimal.
Another example is using Chef to maintain your Docker deployment and the distributed system supporting it, then using Docker to deploy your application payload.
One of the reasons why I really enjoy while reading the notes is the fact that Docker does not shut itself into Linux world and promises working with alternative technologies, like BSD Jails. That's a great news and I am looking forward to see it happening!
On my list of things to do: compare docker to zerovm to libvirt to ??? and figure out which one works will work best with rdma (infiniband) networks. Combined with nixos (maybe), this could work wonders for hpc data provenance.
What about OpenVZ? It isn't as light as Docker (you get a whole OS) and solves a different problem, but it is solid at providing a OS virtualized container.
OpenVZ is almost identical to LXC, which is what Docker uses under the hood. You could port Docker to OpenVZ very easily. In fact, there is a plugin planned to do exactly that.
Fun fact: before running on lxc, dotCloud (where Docker originated from) was initially based on OpenVZ (which was great). As the mainline kernel's support for containers improved (in large part, I believe, thanks to the pioneering work of OpenVZ), eventually we moved over to that.
Thanks for the clarification. I'm working on a similar type of product, but involving heavy compute and data analysis. Since we aren't hosting web apps, we don't have the web routing complexity that you have, but we do share a lot of the backend pieces (OpenVZ and unionFS in our case).
I'm not sure I'm asking this question correctly, but what kind of performance hit do I take by using a Docker container within an EC2 instance instead of just running straight on the EC2 instance's userspace?
I'm not looking for any kind of concrete number, but let's say I've got a Postgres running on an m1.large. Any idea how running Postgres under Docker would change my performance characteristics?
That's correct, you can approximate the overhead of LXC (and any other similar containerization such as OpenVZ, Solaris Zones, BSD jails etc.) as zero. In terms of performance they are closer to chroot than to a virtual machine.
It is possible to use docker/lxc as a Vagrant provider. This means when you run 'vagrant up', your 'machine' could be a lxc container instead of a VirtualBox VM.
Hey guys, for an honest and transparent account of what Docker can and cannot do, and whether it makes sense to use and improve it vs. build your own, come say hi on #docker@freenode.
If we think it makes more sense to build your own, we'll tell you, and will share our experience to save you time while we're at it. The more containers out there, the better for everyone :)
It's poorly documented (so you'll need to dig into the source to figure out how to use it) and at least when I was using it did not play well when scripted (like sending commands to the container). It also wasn't terribly reliable (a command might work one but not the next time) and it seemed like it was in a state of transition. (moving towards a RESTful API?) I just think if you use it now you'll end up writing stuff (at least) twice.
Not that raw LXC is much better... It all seems very immature.
Can you give us some examples on where it is poorly documented so that we can fix those issues?
Also, if you can let us know what you were doing when the command didn't work the same way every time? Have you filed any issues for those yet? If not, can you, or tell me here and I can do it for you. If we don't know about your issues, it is hard for us to fix them.
The main question I have is since it uses LXC, which shares the kernel, I can't have Windows running in Docker or even a different Linux distro (presumably unless they happen to have the same Linux kernel)?
You are correct. You must run linux, but since most distros aren't too picky about what kernel version you run, you should be able to run any distro you want.
Please, please prioritize shared filesystems/storage with the host. I know it was hotly debated when talking about sharing data between containers, but sharing data with the host is very useful.
Version 0.4 introduces a remote HTTP api, a new Build functionality, and an experimental Openstack integration.