Is anyone here using this in production (besides Hashicorp of course)?
I'd love to hear about how it works with Ansible, Chef, etc. for real deployments. From the docs:
> Terraform is not a configuration management tool, and it allows existing tooling to focus on their strengths: bootstrapping and initializing resources
But configuration management tools can be used for provisioning infrastructure too - is it worth splitting your configuration across multiple tools?
Most of the provisioning capabilities in tools like Chef, Puppet, etc. are pretty rudimentary from the perspective of thinking about infrastructure as composable services.
So, for example, a "knife ec2 server create" or a "puppet node_aws" will kick off an aws machine, but it's up to you to manually describe the metadata around that server - what AMI, what VPC, attached subnets, security groups, etc.
But really, that's not what you want, you don't want a server, you want a composed service. You want to describe the entire stack, and you want to be able to represent that stack as code, so it can be tested, version controlled, etc. just like all of your other infrastructure.
The chef guys have clearly recognized this, and have their chef-metal project, which frankly seems like a bit of an ugly hack to get around the limitation that most config management tools have where all of the execution is on a target node (whereas with composable infrastructure you probably want some central node coordinating and orchestrating).
The thing that's more interesting about terraform, as well as Amazon CloudFormation, is that it can not only represent the traditional assets of servers and VMs, but also can describe SaaS services, which is slick.
So it's not so much splitting configuration across multiple tools, moreso of segmenting out the levels of description - I have one tool for service definition and catalog management, I have another tool for effecting change and configuration within a given service. Macro vs. micro.
We're currently using "knife ec2 server create" to create servers, and it is indeed a pain. That's part of what's driven my interest in Terraform.
In practice, though, we're hampered by Terraform's only partial support of the detailed EC2 instance properties.[1] Compared to the full EC2 API[2], there's quite a bit missing (e.g. declare an EBS volume as encrypted).
Is there any way around this by passing arbitrary key-value pairs somehow, or are we limited to Terraform's documented API?
Thanks for your reply. I actually do have a couple related "practice" questions for the theory guy. Do you have any recommendations on good tools for:
a) visually modeling your servers (something more than just lists of instances)
b) tool for monitoring all active jobs to make sure they're all running (I know, lots of ways to solve this one, Deadman's Snitch, Jenkins, etc., but curious what you think)
c) basic tool for dashboard of servers (I guess Nagios is the most common, and there's also commercial tools like Librato).
I realize those are loaded questions. Feel free to answer partials if you're short on time. Thx for your input!
Sorry for not responding sooner, I'm actually at my employer's user conference, so I was lured away by the prospect of food and booze.
a) No! Though this is actually a conversation I've had here with a couple of customers - people want a good service modeling and visualization option. I'm thinking about how that might work.
b) So, I think this is a space where the commercial tools (my employer makes a really good one, control-M) do better than OSS, I think because the nuance of job management and scheduling is so boring and detail-oriented. For OSS, Rundeck wants to do this work.
c) Nagios, Monit, Zenoss - all can do what you want, depending on the nuances of the use cases.
I think separation of node setup/configuration from orchestration is good. Terraform fits in well here, as a complement to Chef et. al. Yes, it's an extra tool but I'd say it's worth it.
There is a meta-issue on AWS coverage[1]. Many parts are already included. Myself; I'm waiting for EBS-snapshots.
The major problem for me so far was/is, at the moment, it doesn't seem like there is any real transition plan; we have a bunch of existing instances, ASGs, ELBs, etc and it isn't really tenable to tear it all down and recreate from scratch just so we have this new way to manage it. (its possible, but not clearly worth it)
Initially looking at the docs I thought the refresh command (http://www.terraform.io/docs/commands/refresh.html) would provide this capability, but I wrote up a manifest matching a subset of our infrastructure and tried to refresh it to no avail.
It is possible that the changing of the state file to JSON in this version provides a clearer path, but it still seems too hard to start using it going forward if you aren't starting from scratch
This is a planned feature in the future, we're internally calling it "recovery mode" (but not sure what we'll call it when it releases).
The basic idea is that you can give it provider credentials and it'll discover resources, or you can tell it a specific resource, and it'll interactively ask you just enough about it to recreate the configuration.
Terraform 0.3 actually laid a lot of the foundation for this, but there is quite a bit more to go. We'll get there sooner rather than later, though.
I would love to know this too. I'm sure they probably have enterprise customers that are using some of this stuff. I do love the idea of being able to store some of these things in source control.
One of the things that I am curious about is that when we promote a database slave to master, we just spin up a brand new slave and just discard the old master. From a quick glance of this, I had a hard time telling if that is something that this tool could handle. Maybe it's not really the use case for it right now. I definitely think it's really cool to just spin up a cluster in one command though.
We were at my work, but unfortunately Terraform was lacking a number of critical features for AWS at the time. It's an amazing tool, but it needs a bit more time in the oven for our use cases :)
We ended up writing a "clone" of Terraform that just did the things terraform was missing using Boto, except our state files were readable JSON instead of a binary format.
Tagging and a lot of VPC features, as well as managing EBS volumes properly. A lot of little stuff, basically, that meant that we couldn't fully automate our infrastructure using this. A lot of that has been fixed in 0.2 and now 0.3, so I have very high hopes!
Yeah, which I'm super happy about. Fingers crossed we'll be able to go and replace our home-brew tool with Terraform now that the gaps are starting to fill up.
We got pretty far with using Terraform to set up reasonably complex AWS architecture (VPC, multiple subnets etc.) There were several blocking bugs which @hashicorp and friends fixed fairly quickly, but not quickly enough for us to use it instead of AWS CloudFormation.
I can see us using Terraform when it is a little more mature - basically more AWS providers. Give it 2 to 3 months :)
I'd love to hear about how it works with Ansible, Chef, etc. for real deployments. From the docs:
> Terraform is not a configuration management tool, and it allows existing tooling to focus on their strengths: bootstrapping and initializing resources
But configuration management tools can be used for provisioning infrastructure too - is it worth splitting your configuration across multiple tools?