Devops or Mars-vs-Venus? Docker, OpenStack and Configuration as Code
The genuine popularity of Docker, and the considerable hype around it, reveals a key motivation that developers bring to cloud: configuration sucks. Assigning network permissions to an app, OS compatibility, library and service dependencies, data persistence rules, registry entries and all kinds of other parameters critical to making it work right -- the first time and every time -- is a headache. Sounds like a win for OpenStack and DevOps.
True, OpenStack provides essential automation, so that you have a transparent, well-specified set of ReST APIs that can securely expose the configuration to just about any kind of management tooling or development environment. DevOps harmony is achieved. Get a DevOps book on Amazon, get the Devs to read it, and ta-da! Devs create, Ops manages.
If only. Let’s take a close look at some of the ways the myth of DevOps can easily lead you astray, including:
- Unleashing devs on all the software defined things
- Docker vs DevOps
- What’s wrong with orchestration
- How OpenStack Heat and Murano help approach the problem
How we got into this mess
All those great ReST APIs for underlying cloud services would seem to be the secret to making OpenStack a killer app for devs. Select your VM, your network configuration, your storage (change it on the fly!), select live migration, and presto! the whole datacenter is your toy. Devs pick and choose resources through the software defined datacenter to fit their apps, and ensure business gets value from IT in the software economy.
The sad truth is this: the application-centric approach is exactly what caused the mess of modern enterprise IT, with applications dictating what hardware and operating systems make up the hosting environment, and to each its own. Welcome to the jungle. Putting devs in charge of competing in the software economy by writing to datacenter APIs seems destined to end in tears.
Why is this so hard? Configuring a single Linux server once to run an application seems easy enough; the second, third and fourth time, any dev worth his/her salt wants an admin to take it over. Configuring a broad cloud fabric of interdependent VMs (each running its own service), or one that’s changing frequently?
Docker vs. DevOps
How much programming does a software defined datacenter need? If you ask the devs, they’d prefer none at all, thank you. Writing docker pull oraclelinux is better than having to specify three dozen boot parameters. However, it’s probably a safe bet that anyone who has to run Oracle may not have the luxury of starting over with a 100% docker-based cloud. Can’t OpenStack make it as simple as a single string?
Dockerization of apps in containers is many things. One of its signal accomplishments is to tame the configuration beast via the dockerfile. Caging the runtime configuration in this relatively well-specified construct provides great pain relief -- at least, for a perfectly containerized environment.
But here’s where the discussions of DevOps are largely headed in the wrong direction. The presumption is that there is a magical place in which the interests of “Devs” and “Ops” are aligned, a ‘culture’ in which each ‘respects’ the work of the other, yadda yadda. But such discussions always sound to me like saying “Ops are from Mars, Devs are from Venus”.
The differences are not easily reconciled or surmountable. Start with Devs being code-oriented, and Ops being script-oriented. In fact, many of the flaws OpenStack has overcome in its 4.5 year lifecycle are in places Devs presumed that they can program away any operations problem. Even PaaS environments like Cloud Foundry have to do some un-natural acts in order to tame the configuration beast, and insulate developers from messing configuration stuff up. And no, you can’t just rewrite it all as 12 factor cloud native applications. IT remains an impenetrable jungle.
But where Ops can learn a lesson from Devs is treatment of configurations. Docker has lit a fire, in a good way.
Don’t cue the orchestration. Let there be Heat
For many organizations, the challenge of finding and deploying workloads that can run on OpenStack has been a clear barrier to adoption of the technology. The fact remains that much enterprise value in existing systems (and the people and processes they support) won’t easily reconstitute themselves into a cloud, or even a software-defined datacenter.
Of all the terms borrowed from devs for OpenStack, ‘orchestration’ does the greatest disservice here. It evokes a complex sequence of activities with arcane serialized dependencies. Heat is not about sequence; it’s about state.
Let’s similarly dispense with the semantic ambiguity of the term “configuration”. It’s not an activity; it’s a state. In cloud infrastructure, it’s the only way to deal with configuration, because the initial state of the configuration is the critical point of inception for any application. This represents an important change: there’s a lot of imperative in the world of deployment, configuration and operations: deployment tooling like Chef, Puppet, Ansible, SaltStack, etc. is premised on codifying complex operational sequences, so as to make them repeatable.
But the most reliable way to ensure repeatability is to eliminate the moving parts. The OpenStack Orchestration project, Heat, is designed to address this problem precisely. A Heat Orchestration Template (HOT), put in computational terms, is declarative, not imperative.
A Heat Orchestration Template (HOT) does not contain sequence information; rather, it encodes a fixed state at a fixed point in time. Here’s where the ‘orchestration’ works in concert -- in one way, like a musical score, it assigns a role and a value to each service and its parameter in the OpenStack cloud; but in another way, it’s totally not like a musical score, unless the entire orchestra is playing just one note.
As designed, the Heat template and the dockerfile serve comparable roles, with one critical exception: the dockerfile’s scope is a single machine image, with its dependencies, and bootable ‘hostlet’. Heat’s scope is a full-fledged IaaS environment, complete with rich networking choices, storage, distributed processing and more. No less important, Heat templates are suitable for CRUD operations -- so that you can roll back or roll forward to a simple configuration, managed through versioning.
In other words, this is the anchor for “configuration as code”, or, to be more buzzword compliant, configuration-as-a-service. Just like a dockerfile, Heat provides a fixed definition of the configuration required to run a service. This is the most valuable lesson that Docker can teach the world of DevOps: for every docker container, a fixed, transparent dockerfile. For every OpenStack workload, a fixed, transparent Heat template. In other words, bind the application deployment to its configuration.
Unlike docker, however, Heat addresses multiple underlying services, rather than just declaring the requirements for a single bootable VM plus any dependencies. Need a configuration set for a bunch of Docker containers? As we’ve explained elsewhere, that’s what a PaaS does. Kubernetes can do this. Mesos can do this. But across the board, OpenStack provides the most generalized platform on which to run any of these services.In part 2, we'll answer the question, "What’s the OpenStack equivalent of docker pull?"
