Yes, containers need OpenStack

With this year's OpenStack Silicon Valley focusing on the intersection between containers and OpenStack, it's got me thinking about how they fit together. I remember when Docker burst on the scene, seemingly out of nowhere. The OpenStack community was used to being the darling of the vanguard set; what did this "new" paradigm mean? Even now, a year or so later, the conversations are still going on, as developers and architects try to decide on the "right" way to construct these applications -- and whether and where OpenStack fits into the picture. Now that we've had time to look at the landscape, let's get a reality check.

Containers, in case you're not familiar, are these "self contained applications" that you can pack up and move here, there, and everywhere. Those of us old enough to hear "Write once, run anywhere" and think of Java without sneering will find this a familiar refrain. After all, being able to write a containerized application and move it around between machines easily is a powerful incentive; we developers want to write software, not engineer entire environments.

Google's introduction of the Kubernetes container management orchestration system seemed, on the face of it, to make the situation even more muddled; after all, here was a way that you could manage all of those containers easily, moving them around, scaling them up and down, and so on.

"Tell us," the skeptics said, "why do we need OpenStack again?"

Because you still needed computing resources on which to run those containers, that's why.

"But can't we just install Kubernetes or Docker Swarm or one of those other container management systems on our server and handle it that way?" the skeptics countered.

Well, you could have, but then not only would you have to worry about scaling -- something OpenStack does well, but containers have yet to solidify -- you would have to worry about the fact that every single container on that host has access to every other container on that host — whether it should or not.

You see, a container isn't quite as "self-contained" as you might think.  For example, when you create a Docker volume, that's an actual directory on the host that you can look at from outside the container. So far the container community hasn't settled on how to manage this kind of security between different containers. Containers have the same issue when it comes to ports; any container can access any other container's open ports.  Did you really want to share that environment between different apps and different developers, or even different users or entities?

Of course you didn’t.

That's why so many people who are using containers are doing it in the context of virtual machines. A VM provides an opportunity to create a completely isolated environment for your container-based application, making it possible to provide the security and multi-tenancy you need in production applications. But how do you easily manage those VMs on today's self-service-oriented data center environment?

With OpenStack, that's how.

What's more, it's not just VMs that we're talking about.  Any significantly advanced container-based application is going to need resources, such as databases, networking, and drive space. By keeping your applications in an OpenStack environment, you get the advantage of the Infrastructure as a Service capabilities it brings with it, such as being able to create storage volumes or networks on demand.

And we're not just talking about resources directly consumed by the containers themselves, either. Containers are ideally suited to today's microservices-based applications, which means that they will ideally be communicating with other resources, both container and non-container-based. That means a hybrid environment with a mix of different technologies.  In other words, the kind of environment where you'll find OpenStack.

Plus, it's a good thing that OpenStack is around, because containers are at their most useful when they're combined, either with other containers or with other applications. And that means an orchestrator such as Kubernetes for purely containerized apps, or OpenStack Application Catalog (Murano) for compositing mixed applications. In fact, Murano makes it possible to deploy Kubernetes and add containerized applications with just a few clicks, bypassing all the pain of trying to manage it on your own — including provisioning resources.

For those who don't want to use Murano, the OpenStack Magnum project is working to make container orchestration engines such as Kubernetes and Docker available as first class resources in OpenStack, deploying container-based resources on hypervisors, or even on bare metal resources, in much the same way it provisions VMs. Or you can run CoreOS, a stripped down Linux optimized for containers, as a VM. And you can do that today.

It's been proposed that we should just skip all that OpenStack stuff and start over with a containers-only management system. "Sure," these people said, "it doesn't exist now, but we can build one!"  But this would skip over five years of development on a system that actually does what they want, in favor of starting over again and solving the problems that have already been solved. (And with a great deal of pain, I might add. There's a reason that Joel Spolsky calls throwing out the code and starting again "the single worst strategic mistake that any software company can make".)

It's natural for a new technology to want to sweep everything away and start fresh.  Even OpenStack started that way, thinking it could replace virtualization giants such as VMware and public cloud behemoths such as Amazon Web Services. But eventually, when the giddiness wore off, we realized two things: first, that there was value in those approaches, and second, that enterprises were not going to suddenly get rid of every application they had in use and start over with OpenStack. So OpenStack adapted and found its place in the ecosystem as an integrator, taking advantage of what had come before.

Recently, the container folks have begun to realize that enterprises aren't going to suddenly go all-container either; they need to look for a place in the existing ecosystem that will serve their needs — and OpenStack is where they're going to find it.

Welcome, container folks, we’re glad to have you with us. It’s going to be a terrific ride.

Starting to take the intersection between containers and OpenStack seriously?  Head out to OpenStack Silicon Valley on August 26-27 to see luminaries in both fields, such as Google's Craig McLuckie, CoreOS's Alex Polvi, Battery Ventures' Adrian Cockroft, and OpenStack's Jonathan Bryce, talk about how things are coming together.