* learning curve of new technology: you need a new set of tools, you can't just use free/ps/top/etc. (well, you can use them, but you don't get as much insight as you get by using the container tools directly); you have to learn yourself, you have to train colleagues and newcomers, etc.
* increased complexity if you have a simple setup; for example if your deployment now consists of copying just 1 file or 1 folder and starting up an app/service, then using containers will include additional moving parts in your setup
* if you already have legacy tech, migrating won't be straighforward, especially for stateful applications (think JSF, ASP.NET Forms, etc.)
* as jeremyjh pointed out below, unless you have a specific strategy for this, you will have duplicated dependencies in containers, which need to be checked for vulnerabilities; you can't just apt update && apt upgrade on the host system from just the security repositories and call it a day (more or less)
It isn't an unsolvable problem but I predict that in many organizations, containers will mean more instances of critical hot-fixes (to library components outside the kernel - e.g. openssl) not being applied across the board because the development teams manage the container image and once the product is in maintenance no one is going to be updating them. This is mitigated by a smaller attack surface but its still a concern.
When I learned Docker for the first time I was so surprised that so few people mention the problem. As one who is used to automatic secutity updates provided by Linux distributions, relying on manual `docker build`s gives me anxiety. I use Docker because it solves many of my problems, but I have not been able to feel easy from potential security concerns from the beginning.
What I meant by "automatic" was Ubuntu/Debian's unattended upgrade feature. I've used it for years without any problem, and it has relieved much of my maintenance burden.
Maybe I need to enable it in containers too? I have no idea how to use it on (containerized) Alpine Linux though.
Even without unattended upgrades, finding the list of `Dockerfile`s and `docker-compose.xml`s that might be affected by a new vulnerability sounds more complicated to me. Until now it hasn't been that difficult but I'm still a bit nervous if I somehow missed some vulnerable images.
you can configure unattended-updates with apt-get. I guess you can do the same with docker if you use some sort of CI service + auto deploy script, but it's way more work + way more opportunities for failure
There is a lot of misleading marketing hype, misunderstanding, and plain misinformation about how containers/Docker make updates, and in particular security updates, easier, when the opposite is the reality. A lot of people buy into the CoreOS marketing hype, as though it was the only Linux distribution to offer automatic security updates. If you use containers as recommended, in fact the only thing CoreOS gives you are kernel updates, which is less than what other distributions provide. To get userland updates with Docker, you have to update your base image, then rebuild and re-deploy your application image, for every single application that you have. Or use custom kludge layers for updates: https://github.com/SUSE/zypper-dockerhttps://github.com/projectatomic/rpm-ostree
Making maintenance and updates harder is definitely a container specific problem.
Many organizations have operations teams that are responsible for ensuring servers get patched on a regular schedule, and there is an entire industry of software to confirm whether or not they are doing it properly. But I know for a fact that at some of these same organizations, the deployable unit is a Docker image that the development team pushes to the repository when it suits them. Operations doesn't have the ability to patch the software in the container, and the auditing tools haven't caught up either.
Bad habits. There are people who with a straight face claim configuration management is unnecessary if you have a single shell script, aka Dockerfile.
Over-simplicity becoming complexity. You used to be able to write an app in C, pre-build it for a dozen platforms, then run it on a dozen platforms. Now you have to run everything on one platform (a virtualized containerized Linux environment) because anything else is literally impossible. You can still run the app anywhere - it just has five or six layers of virtualization, abstraction and tooling now.
More complexity. Containers are supposed to just be processes, but you can't run them without preparing the host OS first, and you can't really use them en-masse without a complex scheduler and tooling.
Nobody mentions it, but be careful your disks don't fill up just from deploying regularly. People think containers "cost almost nothing", and then all of their over-use of containers results in resource drain from the 15 million things necessary to configure and build and deploy and manage the things across distributed clusters. It's more resource-intense than people advertise.
The container is supposed to be this read-only isolated self-sufficient jail, so good luck getting anything out of it without a bunch of hoops. You seriously need to specify 50 different config values just to do simple things like networking, writing to a persistent disk, picking what to run or stop, interacting with other apps.
People say "a container is just another process" - except each one may be using an entirely different Linux distribution worth of software dependencies. And you still have to track what containers you're running, because you will depend on X release of Y container.
Functionally speaking, containers are almost like a VM inside a VM. It gets that obtuse.
I don't know that devs are actually saving time using containers vs a single dev platform. I think the problem all along was nobody knows how to maintain platforms, so they literally create and throw away whole platforms every time they build an app.
Most of these comments are about Docker, but not necessarily specific to it in practice.
> You used to be able to write an app in C, pre-build it for a dozen platforms, then run it on a dozen platforms. Now you have to run everything on one platform (a virtualized containerized Linux environment) because anything else is literally impossible. You can still run the app anywhere - it just has five or six layers of virtualization, abstraction and tooling now.
I see more and more "here is a Docker image!" things pop up for tiny shell script utilities on github. It's like autotools with 100x the complexity and "portable" to exactly one platform. People have no clue what they are doing, they just want any excuse to use Docker.
> The container is supposed to be this read-only isolated self-sufficient jail, so good luck getting anything out of it without a bunch of hoops. You seriously need to specify 50 different config values just to do simple things like networking, writing to a persistent disk, picking what to run or stop, interacting with other apps.
This is my biggest pain point with containers, and where I feel Docker marketing is lying the most. Docker is pitched as making development easier, when in fact containers significantly impede debugging. You even need special tools (like https://github.com/emacs-pe/docker-tramp.el) to do the most basic tasks like look at files. The marketing hype is that all you need to solve production problems is a log server and a few line graphs of metrics (better hope you instrument your app code for the right metrics...). Add in microservices and now you need distributed tracing too. This extra infrastructure is a ton of work to set up for small apps.
If it's immutable containers we're talking about, then patching needs to be handled by your image build system and containers need to be restarted in order for the patches to be applied. Yes, you'll need an image build system and at least one staging environment.
Applications need to be designed with immutable containers in mind. This makes them a pretty niche technology.
LXD, which offers mutable containers, doesn't suffer from those last two points. Of course you don't get the benefits that the immutable design brings.
>so it's perfectly possible to run windows workloads as well
afaik windows containers only support a subset of the full win32 api, so it's exactly the same as running actual windows. that said, simple asp.net sites should work fine
Admittedly, it's been a while since I played with Docker on Windows, but the last time I used it the containers were huge and temperamental. Has the experience improved in the last year?
I don't run a lot of windows container stuff day to day, but from what I can see they are improving, definitely image size is coming down quite a bit.
Microsoft are obviously investing quite a lot of effort in making the windows container ecosystem more robust, as well as improving the experience of running Linux containers on Windows.
I think this is eventually going to do for virtualization what virtualization did to servers. Less oses deployed maybe less virtualization and baremetal container os
The essential downside to using containers is that you are adding layers of abstraction to your infrastructure. Adding layers of abstraction usually increases complexity.
However, layers of abstraction should also bring benefits wherever they're used. Docker and Kubernetes go hand-in-hand with distributed software or "microservices." If you don't see any benefit to that architecture, you're likely going to introduce unnecessary complexity into your infrastructure with containers.
The more deployments of individual software applications you make, the more you can gain from containers. And the inverse is also true - if you rarely deploy new versions or updates to your software, using containers will require tackling complexity that doesn't benefit your use-case.
None, really. Well, it depends what you're asking precisely. In order to explain, let me share an analogy that I've been thinking about, as a way to describe containers:
When you launch processes on Linux, you can choose for all the processes to share memory with each other (in which case we call them threads), or the processes can have separate memory. We call this capability "virtual memory", and it allows processes to pretend that they and the kernel are the only software running on the physical machine. (For example, every process might think that it can write to the memory address 0x08048000, but in fact that address is virtualized and mapped by the kernel to some different physical memory address.)
Though we take it for granted, this memory isolation between processes was the first form of kernel namespace. Modern Linux now has many of these namespaces, which can provide a virtualized view of the file system, network, IPC, uids, and so on. "Containers", then, are processes that employ many of these namespaces to provide isolation from others on the system. (To extend the analogy: just like regular processes think they have their own memory, containerized processes can have their own file system with a distinct /tmp, distinct process id namespace, or namespace of users that exist on the system. Every process can run as PID=1 if it wants, within its own PID namespace.)
These namespace primitives, from which containers are made, are present on modern Linux systems whether one chooses to employ them or not. The primary downsides to consider are efficiency of the system, and the complexity of managing it. If your system (CPU & OS kernel) implements virtual memory efficiently, then there isn’t much downside to using it. The same is probably true for most kernel name spaces, though file system and networking virtualization can have a performance hit if not configured with care.
Kernel namespaces are a core feature of Linux and can be taken advantage of without using technologies that are specifically "container-oriented". For example, consider systemd: it provides a comprehensive set of parameters for employing namespace features in processes that it launches. Parameters like PrivateTmp=yes or PrivateDevices= [1] will virtualize just one part of the system (/tmp or /dev and such, respectively). PrivateUsers= and PrivateNetwork= allow you to launch with a new user or network namespace. It's really that easy to employ namespaces: add a line to your application's unit file.
When you look at container technology as a set of system primitives, then it makes sense to consider employing them individually. They don't have to be used in an all-or-nothing way. You might give your daemon access to the regular network
with a private file system, or you might use the regular file system too while virtualizing just /tmp and /dev. You can isolate processes in whatever way makes sense, and share whatever resources make sense. If a process doesn't need access to the network, and doesn't need a shared /tmp with the rest of the system, then there's little downside to giving the process its own namespace. Indeed, I expect it will increasingly be considered a best practice for services to isolate themselves according to the principle of least privilege. This reduces the chance that components will interact with each other in unexpected and undesired ways, and may have some security benefits.
(If the question is about using Docker or some other specific technology as part of a development and/or deployment process, then that's a different topic. Specific technology suites have tradeoffs beyond those of kernel namespaces.)
> The primary downsides to consider are efficiency of the system, and the complexity of managing it
This shouldn't be underestimated, though!
I agree that efficiency is a no-brainer for almost all purposes, especially compared to other forms of virtualization like KVM.
But: Managing the complexity can be tough, it sometimes feels like creating all low-level firewall rules by hand for a large network. Well, for firewalls you have stuff like firewall generators, in the context of containers these are projects like Docker or LXD. But they share the same downsides: You may have trouble to see what happens behind the scenes.
The largest risk here, however, is not the working time spent to learn and apply that stuff. It is a false sense of security where your complex system might, in the end, to have holes which you assumed should not exist.
For example, not many people are aware that if the user can start docker instances, they are effectively root.
(One might argue that this is more a failure of docker than of containers. But then, why do people use complex monsters like docker in the first place? Because managing plain containers is a pain and other complex building around containers have their downsides, too.)
Good arguments. Though there is one difference I can see between virtualizing memory and virtualizing other resources, like the file system or the network. (note, I'm pretty much a container newbie).
Process memory has traditionally been an "implementation detail" of the process. It typically doesn't need to be shared, the addresses do not need to be fixed, etc. (Shared memory is an exception of course).
Resource like the file system or network, on the other hand are mostly shared. It does matter to the outside world through which port the program is reachable, it matters where it puts its files, etc. Again, there are exceptions of course, like temporary files.
Now the question is how can we pull out the configuration, like "what port must the program listen on?", and decide these tasks centrally. The container solution is (if I understand correctly) that the program will always listen on port 80, but the container configuration diverts that port to be accessible to the outside world as 8080, etc.
I predict that this is going to be a failure in the same sense that OOP has been a failure (if you agree with me on that point). It becomes a tangled mess of layers and indirections, and incredibly hard to debug.
Instead I think it is much cleaner to make the configurations like, "What port should I listen on?", "Where can I put my temporary files", easily accessible to the program. The program should not be required to define its own configuration for all the problems which containers want to deal with today. Instead it should import this configuration from a central location through a well-known interface.
There must be a simple convention where these decisions can be configured. The configuration must be easily accessible to A) the system, B) the program, C) the administrator. It's been customary to put configurations in text files for a long time, and that's mostly good. However what's been missing is that most programs have had their own configuration file written in a custom syntax and stored in a location that are not easily known by the rest of the system. The system is not really aware that the program is configured e.g. to listen on port 80. And the administrator needs to learn new configuration files for every program.
I'm not proposing a central registry where all the cruft of the system should accumulate. Instead, just for the concepts that are shared by many programs -- the resources which containers virtualize today -- should be configured through a system-wide API.
Containers itself are a solved problem, but multi-node cluster management is still in infancy state. There are loads of tools for all kinds of situations but few that really solve hard problems. So often one is left alone with figuring out why the network is not working today, or which machine's disk has been filled with garbage etc. Openshift is one of the better packages in that regard. Kubernetes simply says "here's the api, deal with it yourself".
