Flapping, you know, when a person who cannot swim is drowning and just flailing and flapping their body, most notably arms, all over the place in an attempt to stay above water so that they do not drown.
but the technologies in your list do not all address the same problem space.
I don't know about you, but for me to get a fully configured, ready-to-serve system, I need both provisioning and configuration management. With that being the case, provisioning and configuration management fall under the same problem domain.
Vagrant allows you to create reproducible development environments using shell scripts (by default), so it should be right up your ally. Docker is used to containerize applications. runC is the container runtime used by Docker. Kubernetes orchestrates containerized applications across a fleet of CoreOS (a minimal "Docker host" Linux) machines. OpenStack orchestrates your data center. They are discrete solutions because they do different things. Each piece of the puzzle does one thing and (hopefully) does it well. That philosophy should sound familiar to you!
I'm familiar with all of those technologies, and yes it does indeed sound familiar to me!
But, what is unclear to me is, why I would use a discrete provisioning solution like Kubernetes instead of using, say, imgadm(1M) and vmadm(1M) through parallel SSH?
Why would I use a discrete, proprietary solution like Vagrant when I wrote my own, simple variable expansion function which uses eval(1), and that's all that's really needed for configuration management? I'm far, far better off with a simple shell function then using Salt or Ansible. Simplicity, the core of the UNIX principle. eval(1) is much simpler and easier to understand than an entire provisioning and/or configuration management engine! Or do you disagree?
Other than templated configuration excerpts and self-assembly, both easily implemented with a few simple lines of shell code in a small shell library which OS packages can source, what more do I need to configure systems? Why make things unnecessarily complex? In fact, that exact same approach is what Chef uses, conceptually, when one specifies the configuration management policy, except that with my solution, the system administrator need not learn a proprietary Ruby dialect and need not employ a client-server architecture; they can just dive in and leverage their existing shell programming knowledge, with a zero learning curve. And because it's all just shell and AWK, the system administrator has infinite possibilities to configure the systems and do anything they need to do, without being limited by the framework of whichever technology they are using (like YAML or Chef recipes).
So why bother??? Why flap?
Why would I need runC and Docker to provision systems, if I can just install OS packages which configure the system, and then make a ZFS image?
Or, in terms of Linux: why do I need say Docker and Kubernetes, if I have everything in OS packages, and I just use KickStart to PXE boot, automatically install and configure the target node with OS packages?
have you tried a configuration management tool?
You bet your ass I did! But that's not the point, the point is, why should I learn a discrete, proprietary systems, when I can just write a simple shell library which leverages the eval(1) command to do variable expansion, and BAM!!! I instantenously have a template expansion engine so all my configuration, even code, can be parametrized using the same varaible evaluation and expansion technology which the shell uses, and which the configuration packages can call?
With that:
- I don't have to use proprietary, discrete systems;
- I can leverage technology already built into the operating system;
- I don't need a client-server architecture like Chef or Puppet, ever (ssh-sshd is just fine, thank you!)
Your strategy of calling everything a provisioning system and then decrying the surfeit and complexity of provisioning systems is deeply flawed. Kubernetes is not a provisioning system. Docker and runC are not used to provision systems. Vagrant is not "an entire provisioning and/or configuration management engine," and its functionality cannot be trivially implemented with a "simple variable expansion function." You say you are familiar with these technologies, but your familiarity is insufficient. Your comparison of Docker and Kubernetes to OS packages, Kickstart, and PXE is outright laughable. You clearly do not understand the various purposes of these tools.
You point to imgadm and vmadm as superior solutions, but these appear to be "proprietary" (to use your own word) to SmartOS. Why not add SmartOS to your list of unnecessarily technologies and apply the same critical thinking? Triton appears to occupy the same space as OpenStack (Triton DataCenter), Kubernetes (Triton ContainerPilot), CoreOS (Triton SmartOS), and possibly other components, and it appears to use the same kernel-level containerization facilities as Docker and runC. You're using a competitor's vertically-integrated solution, so no, you don't need another stack. If you weren't using Triton you would be researching these alternatives. Your argument is akin to saying you don't need a car because you have a Buick, people who drive cars are using an unnecessary, proprietary mode of transport, and they should all just drive Buicks instead.
That brings us back to configuration management tools. You keep calling them proprietary, but the cores of all these products are open source! And while I agree with you that the client/server model used by Puppet, Chef, and SaltStack is unnecessary—although it does have benefits at scale—it's worth noting that Chef Zero and Ansible just use ssh-sshd. These tools are making the same system calls as your shell scripts, they're just doing them from Ruby or Python instead. So just to recap:
> I don't have to use proprietary, discrete systems;
They're not proprietary, they're open source, and you only need one of the four.
> I can leverage technology already built into the operating system;
They do: Ruby or Python and system calls.
> I don't need a client-server architecture like Chef or Puppet, ever (ssh-sshd is just fine, thank you!)
Okay, so use Chef Zero or Ansible.
Finally, your other main argument, against abstraction, does not hold water. Everything we do is an abstraction. Shell scripts are an abstraction. Why aren't you writing your configuration management in C? (Wait, why are you writing it in C instead of assembler?) On one hand, you have a portable, open source tool with a domain-specific language. On the other hand, you have each company creating their own proprietary (this is the correct use of "proprietary," by the way) tooling, and an engineer has to learn it from scratch every time they change jobs. You may think any engineer can walk in off the street and instantly grok your homegrown shell- and AWK-based big ball of mud, but you are probably wrong. If your solution is as good as you think it is—if it's portable, extensible, and easy to maintain—you should release it and we can add it to the list of available CM tools. If it's not, why are you reinventing the wheel?
Kubernetes (commonly referred to as "k8s") is an open source container cluster manager originally designed by Google and donated to the Cloud Native Computing Foundation. It aims to provide a "platform for automating deployment, scaling, and operations of application containers across clusters of hosts"[1]
Automating deployment? I guess deployment has nothing to do with provisioning a resource... really, if you are going to argue for these tools, you should understand the concepts which they are trying to solve. Or are you one of the developers of one or some of these tools, and this is just rubbing you the wrong way, because it means that your tool isn't required? If so, just state that upfront, so everybody knows where you stand, and we don't have to discuss this any further.
your comparison of Docker and Kubernetes to OS packages, Kickstart, and PXE is outright laughable. You clearly do not understand the various purposes of these tools.
Okay, then you explain to me why I need Docker to do what I can do with PXE, KickStart, and OS packaging? This ought to be really good...
Why not add SmartOS to your list of unnecessarily technologies and apply the same critical thinking?
Because SmartOS uses zones, which give one fully isolated, true UNIX servers running at the speed of bare metal, and because it uses ZFS, fault management architecture (fmadm(1M), svcs(1), and svcadm(1M)), performance and post mortem analysis tools, and because it has everything required for massive datacenter and cloud deployments, built right into the operating system from the beginning, and not as an afterthought, knee-jerk "oh shit, we have to scale Linux to massive deployments now" reaction. By professional system and kernel engineers with formal education in those problem domains. imgadm(1M) and vmadm(1M) in SmartOS are the functional equivalents of JumpStart (or KickStart, in redhat's jargon, and AutoYaST in SLES one). That's why.
They're not proprietary, they're open source, and you only need one of the four.
Proprietary means that someone owns it, or that it is specific to something or someone, not that it is closed source.
A product can be both proprietary and open source, and freeware all at the same time if the proprietor is providing the source code and giving the product away for free.
They do: Ruby or Python and system calls.
Neither of those programming languages are part of the operating system consolidation on a proper UNIX, and on Linux, they are provided in various versions by various people who are volunteers. And anything which is not part of the OS/Networking consolidation on a real UNIX is a third party and unbundled application. That's the only distinction we have ever needed, and the only distinction we have ever made.
This mistake was already made once on UNIX, when Perl was hip just like Ruby and Python are now: we are still coping with the fallout of ripping out, and replacing all the Perl garbage which has since been integrated into the OS/Networking consolidation. You haven't yet been busted by that on Linux, but you will be, because Linux keeps repeating the same mistakes which UNIX made. Except we learned from that, and on Linux it just causes more flapping.
Between Ruby, Python and shell and AWK, I will always pick shell and AWK (and not even bash, but traditional Bourne shell), because they are stable, the interfaces won't change, the syntax will stay the same forever, they are well understood, easy to learn, and they are small and fast, very fast.
I have regularly beaten Python with AWK both in code density and speed (latest: 487 lines of Python code in 183 lines of AWK, and no kernel hooks required like with the Python code). So if you're implying that Ruby and Python are ideal for massive deployments and configuration management just because they're hip right now, that's insane, and I'm not buying it.
Okay, so use Chef Zero or Ansible.
Why? What does the Ruby dialect of Chef offer me, that I couldn't do in %pre, %post, %preun and %postun in a RPM package? Policy? Policy is best solved with a throughly thought through change management process. Capability Maturity Model and all that, you know?
Why aren't you writing your configuration management in C?
And just what do you think svcadm(1M) is written in?
(Wait, why are you writing it in C instead of assembler?)
Who says I'm not? Use the right tool for the job. The problem here is, none of these Chefs, Puppets, Ansibles, Salts, Dockers or Kubernetes are the right tools for the job. Even Linux as the substrate for all of this is not the right tool for the job! But then again, judging by the other comments here, people are starting to realize that, and that's a good thing.
Flapping, you know, when a person who cannot swim is drowning and just flailing and flapping their body, most notably arms, all over the place in an attempt to stay above water so that they do not drown.
but the technologies in your list do not all address the same problem space.
I don't know about you, but for me to get a fully configured, ready-to-serve system, I need both provisioning and configuration management. With that being the case, provisioning and configuration management fall under the same problem domain.
Vagrant allows you to create reproducible development environments using shell scripts (by default), so it should be right up your ally. Docker is used to containerize applications. runC is the container runtime used by Docker. Kubernetes orchestrates containerized applications across a fleet of CoreOS (a minimal "Docker host" Linux) machines. OpenStack orchestrates your data center. They are discrete solutions because they do different things. Each piece of the puzzle does one thing and (hopefully) does it well. That philosophy should sound familiar to you!
I'm familiar with all of those technologies, and yes it does indeed sound familiar to me!
But, what is unclear to me is, why I would use a discrete provisioning solution like Kubernetes instead of using, say, imgadm(1M) and vmadm(1M) through parallel SSH?
Why would I use a discrete, proprietary solution like Vagrant when I wrote my own, simple variable expansion function which uses eval(1), and that's all that's really needed for configuration management? I'm far, far better off with a simple shell function then using Salt or Ansible. Simplicity, the core of the UNIX principle. eval(1) is much simpler and easier to understand than an entire provisioning and/or configuration management engine! Or do you disagree?
Other than templated configuration excerpts and self-assembly, both easily implemented with a few simple lines of shell code in a small shell library which OS packages can source, what more do I need to configure systems? Why make things unnecessarily complex? In fact, that exact same approach is what Chef uses, conceptually, when one specifies the configuration management policy, except that with my solution, the system administrator need not learn a proprietary Ruby dialect and need not employ a client-server architecture; they can just dive in and leverage their existing shell programming knowledge, with a zero learning curve. And because it's all just shell and AWK, the system administrator has infinite possibilities to configure the systems and do anything they need to do, without being limited by the framework of whichever technology they are using (like YAML or Chef recipes).
So why bother??? Why flap?
Why would I need runC and Docker to provision systems, if I can just install OS packages which configure the system, and then make a ZFS image?
Or, in terms of Linux: why do I need say Docker and Kubernetes, if I have everything in OS packages, and I just use KickStart to PXE boot, automatically install and configure the target node with OS packages?
have you tried a configuration management tool?
You bet your ass I did! But that's not the point, the point is, why should I learn a discrete, proprietary systems, when I can just write a simple shell library which leverages the eval(1) command to do variable expansion, and BAM!!! I instantenously have a template expansion engine so all my configuration, even code, can be parametrized using the same varaible evaluation and expansion technology which the shell uses, and which the configuration packages can call?
With that:
- I don't have to use proprietary, discrete systems;
- I can leverage technology already built into the operating system;
- I don't need a client-server architecture like Chef or Puppet, ever (ssh-sshd is just fine, thank you!)
Why would anyone waste their time?