OpenStack in the gap between PaaS and IaaS
Early in my career, well before OpenStack, I emailed a joke to the dev team. The punchline: no one with a three digit IQ would debate whether Emacs was better than VI, or vice versa. What surprised me was the response I got; something like:
“Hey, that was really offensive, and in poor taste. I can’t believe you would send that over email.”
If you’ve lived the DevOps divide, you know that developers can be more defensive about their choice of development tools than a cranky frog with sunburn. Whether that stems from Maturity Deficit Disorder or arrogance, it can be one of the most significant barriers to cloud adoption, and OpenStack adoption even moreso.
There’s room for debate whether cultural differences and organizational models are a cause or an effect of IT balkanization. (Some of that debate has touched down here in our blog). My purpose, however, is to look at how OpenStack is tackling this divide head on, so that a developer's choice of PaaS can be more of a functional decision and less of a 99-year commitment.
Can’t ReST our way into the minds of developers
The great advances in *aaS-ification we’ve seen in the last ten years have left a meaningful gap between application developers and the infrastructure their apps run on. However, there’s a temptation in the OpenStack space to think we’re done when we express infrastructure constructs through a ReST API.
But that has the problem precisely backwards. Service-based interfaces to cloud components do not constitute application-ready environments, let alone environments that can reliably host and run a constantly changing mix of apps. Just publishing ReST APIs doesn’t persuade developers that “Cattle > Pets”. How exactly do we expect pet owners to turn cats and dogs into 4-legged steak-and-dairy factories? Pet servers run pet apps.
Left to its own devices, a development teams will dictate the constraints of the infrastructure for its own collection of applications, and operations teams will face the same inconsistencies and fragmentation that slow them down today. OpenStack must face up to the fact that it’s application developers who need to understand the part they need to play in this changeover to a cloud-based environment. We need to help developers see why they can’t just continue on as they have been, and think of infrastructure merely as something you blame for crappy performance and outages.
State of the State between PaaS and IaaS?
To unpack the problem, let’s work in the other direction: what do developers need to solve to make their apps work and thrive in the IaaS environment? Bridging that gap means that OpenStack needs to offer abstractions that solve real developer problems in a way that makes sense to a developer. To that end, OpenStack has several projects that can provide an opportunity for PaaS developers to help create an environment in which they can flourish.
The current thinking in the OpenStack ecosystem is to apply the same ‘aggressive modularization’ at the heart of the OpenStack project to the problem of melding PaaS successfully with IaaS. There are 3 major dimensions addressed by this modularization:
Turning source code into something that can run in the cloud
Find applications to use or consume, in the service of reusability
Generalizing deployment topology, including HW resources and configurations and SW components for execution
If you’re a PaaS partisan, you might say, “Wait a minute, PaaS does that. The End.” That’s not a ridiculous answer, nor would it be sufficient evidence to question a 3-digit IQ. But suspend your disbelief for a moment.
Solum: dispatching your source to the cloud
When I started programming, not long after the discovery of fire and the invention of the wheel, source was something I compiled (or hoped would compile; VB was big). Today, the target is not the compiler, it’s the cloud, and the output is not binary, it’s packaging.
That’s a simplistic view, of course. Build tools like Jenkins that run from source trees and produce deployable code are essential. In more advanced cloud-development efforts, Jenkins helps turn the gears of continuous integration and continuous deployment, aka CI/CD. (A detailed exploration of the mechanisms of CI/CD are beyond the scope of this post.) Deployment fails if the packaging into libraries and components is wrong; standardization of packaging is critical to keeping applications current.
The OpenStack project that’s charged with tackling this problem space is Solum. While some have positioned Solum as a substitute for incumbent PaaS technologies, it’s not an either-or. Rather than being a simple open-source PaaS, Solum is meant to provide PaaS-enabling services within OpenStack.
In fact, if Solum comes together as intended, it will provide tooling to let any given application -- including another PaaS -- generate an artifact that can be deployed on an OpenStack cloud. The idea here is that Solum will know what that “something deployable” piece is for a given cloud. In other words, Solum could provide an antidote to PaaS lock-in.
Murano: Do I have to or did someone already do that?
A level or so up the stack, reusable code is a fantasy almost as old as I am. Cloud has helped. But re-usable ready-to-deploy applications need to be easier to find and consume. Why bust your head trying to figure out how to deploy a Tomcat server when you can just push a button and get one?
The ‘service catalog’ is a well-known construct in the world of ITIL, though most application developers think ITIL is someone else’s problem, about as fascinating as, say, SNMP. A catalog that organizes ready-to-use cloud applications not only saves having to reinvent the wheels for your cloud application; it also provides a mechanism for distributing known good application components. App stores are catalogs that have made mobile what it is today. (For an interesting look at the problem of finding apps, have a look at Quixey.)
Of course, on a smartphone, you don't often need to combine applications; no such luck on the cloud, where it's a common occurrence. When that happens, it sure is nice not to have to tie yourself in knots deploying a json database from scratch, or to figure out how to fit a Microsoft SQL Server database on a Windows guest running on KVM in OpenStack. Having a uniform nomenclature for descriptors around requirements, capabilities, permissions and so on, and establishing conventions for all of those aspects of an application is as critical to discovery as it is to management and maintenance. The Murano project is OpenStack’s effort to deliver this catalog.
Deployment: Defining the resources apps need on the cloud.
A classic definition of packaging is that it defines resources and relationships. In the context of cloud, the resources and relationships are a distributed infrastructure. A simplistic view of IaaS would define a VM with some storage and a network address, where you would ‘install’ an app as though it were running on pair of Mac Minis and a router behind your desk.
Of course, this doesn’t work so well in a distributed environment with multiple infrastructure resources. In the cloud, you need to be able to tell the infrastructure that your application wants, let’s say, two networks:
Network A has Apache Servers, with IP addresses assigned by DHCP
Network B has a database, with listeners on ports X, Y, and Z
A router between them, with access policies defined as …
If you’ve ever tried to deploy an application across multiple physical resources, you might recognize these sorts of dependencies. Specifying them correctly (and persisting that specification) and debugging them when there are problems is not optional. Or trivial.
As defined, the OpenStack Orchestration project, or Heat, patterned after Cloud Formation on AWS, addresses this problem. It’s a declarative language meant to answer the question, “What does your application need?” Heat creates the equivalent of an actionable checklist, which the Heat engine translates into the necessary dependency graphs, the sequence of IaaS calls, and so on, causing the underlying infrastructure primitives to organize themselves into a well-configured execution environment. Why Heat? One advantage, of course, is that you’re not married to AWS if you deploy to OpenStack. But the other advantage is that as your infrastructure changes, you can take advantage of new components.
Can PaaS go back to the future?
So with this “gap analysis” in hand, would it not be simpler to just deploy to a PaaS and be done with it? For organizations that have a very solid grasp on the future of their software landscape, it could be just fine. For those who can look back at years of technical debt – accumulated as a result of past certainties about the future of their software roadmap – this might give pause to PaaS.
The convenience of PaaS solutions is that they provide a single, integrated framework. You can start with source and let PaaS take care of packaging, cataloging and deployment; try to move what you’ve built in PaaS from one IaaS to another, and, it may not be so simple. Each PaaS has its own way of containerizing an application, sometimes nothing more than configuring a standalone VM image on which to run the app, and letting it loose on your infrastructure.
Moving an application from one PaaS to another is also not so simple. It compares to taking cartridges from one printer and trying to use them in another -- even two printers from the same manufacturer.
It would be an exaggeration to suggest that the work in OpenStack is as functionally complete as a typical PaaS (let alone a hosted PaaS like Heroku; the experience for a developer on Heroku is pretty sweet).
But it’s the non-functional qualities, such as performance, scalability, security, and more, that should make OpenStack a more palatable long-term alternative. And if you imagine a cloud where lots of different applications can run together, in some degree of operational harmony, PaaS should give you pause (ok, that’s the last time I’ll use that pun). Cloud chaos can readily ensue when each application is in charge of its storage, compute, and networking consumption.
Looking ahead, OpenStack is trying to find ways to simplify interaction and interoperability between the natural diversity of PaaS tools so that you can combine applications from different PaaSes on a consistent OpenStack infrastructure. Over the course of the Juno release and its successor, the intended course is as follows:
Starting with the build process, Solum will provide tooling for each PaaS. Solum will expose an API that will take source code from the PaaS and generate an image that will run on an OpenStack compatible packaging/run-time. Solum will accept build instructions from the PaaS, and use those to direct whether PaaS output should go to a container, a guest, a VM, an image, and so on.
Each PaaS has an internal, walled-garden catalog of services. It specifies important details regarding how an application consumes each representative service, using basic parameters such as as a connection string, for example. By contrast, the OpenStack approach expressed in Murano, through a generalizable catalog structure, can combine applications and services independent of the development environment in which they originated. You get much broader selection in the catalog by combining services from various PaaSes or originating services from elsewhere – and avoid lock-in. (In fact, the Murano strategy also targets a range of application packaging standards such as TOSCA, Parallels APS, or even Heat Templates for inclusion in the catalog.
The Murano catalog can invoke Heat template generation at deployment time to allocate OpenStack IaaS resources to run the application set. As the infrastructure changes and new capabilities are added to the OpenStack IaaS, the decoupling of the infrastructure from the application means the infrastructure can adapt to changing requirements, and changing platform components, more rapidly.
Summary
As befits its infrastructure-focused origins, OpenStack has more to offer today on the deployment side of the PaaS-to-IaaS gap than on the development side. But accepting this as a desirable state of affairs is akin to dismissing leaks on the port side of your boat because you’re on the starboard deck. And there are encouraging signs that the PaaS world is embracing taking the opportunity seriously, with the recent discussions around Cloud Foundry for OpenStack.
Independent PaaS vendors, of course, can always just wait for IaaS vendors to adapt to their assumptions. But that’s a heady bet; the assumption that one PaaS approach solves all cloud application problems past and future is probably as safe as locking into a certain programming language from the 1990's, or some other interesting technology platform from Seattle. The better bet for PaaS vendors is to roll up their sleeves and contribute to the success of the Solum, Murano and Heat layers of OpenStack.
In fact, if the persistent debates such as Emacs vs. VI are any indication, supporting better integration with OpenStack may even be a sound strategy for PaaS suppliers to ensure they don’t go the way of Visual Basic.
