OpenStack Needs More Than One Hat

It seems that it's becoming fashionable to complain about OpenStack.  It's natural; every technology undergoes growing pains, when it's advanced enough for people to want to use it, but not advanced enough that it satisfies 100% of everyone's expectations.

It seems that OpenStack is now at that point.  Gartner is complaining that OpenStack lacks focus.  A former contributor complains that Ceilometer is ... less than it should be.  And now Matt Asay of ReadWrite has jumped on the bandwagon, complaining that OpenStack must have leadership, and nominating Red Hat for that position.

Matt's position is that OpenStack can't grow appropriately without a strong hand, and that Red Hat is in the best position to be that strong hand, guiding (or even controlling) where OpenStack goes.

Now, I'm not saying that Red Hat isn't important in the OpenStack space.  They are, after all, the number one contributor to the codebase; nobody's disputing that.  But I would definitely take issue with a number of his arguments.

"No" isn't always the right answer

Matt argues that the current process by which OpenStack is developed has significant problems.

He says that someone needs to step forward and take charge, sometimes putting the kibosh on new development:

For one thing, OpenStack is punished by undisciplined product governance. The best open-source projects, like Linux, have a strong, small leadership team that knows the value of saying "No." But OpenStack tends to say "Yes" to every new feature, not ensuring consistency in the product and not ensuring that the hard and boring problems get solved. This may stem from a lack of coherent vision as to what OpenStack is, and isn't, according to Gartner.

Part of the problem with this statement may stem from some confusion over not what OpenStack is and isn't, but rather what is and isn't OpenStack.  Just because a project is started and worked on doesn't meant that it's accepted into OpenStack.

Besides, wasn't it just a few months ago that the same pundits who are complaining that OpenStack does too much were complaining that we aren't innovating fast enough to compete?  Even Matt himself complains that OpenStack isn't being innovative enough.  Innovation comes from the ability to pursue a project because you feel that it's important.  You can't have a truly innovative environment where a single vendor can squash a project unilaterally.

OpenStack developers aren't hobbyists

Part of the problem, Matt suggests, is that developers work on what interests them, as opposed to tackling those "hard and boring problems".  But he's making an assumption that's not, in general, true for OpenStack.

But part of this may also derive from the possibility that OpenStack is driven by developers who do it for the fun of development and not with the ultimate cloud user/operator in mind. This is a problem inherent to many open-source projects, which privilege contributors over users. That sort of a mindset isn't productive or sustainable.

I'm not going to suggest that OpenStack can't do a better job of catering to users and operators; that's a different battle to fight.  But to portray OpenStack developers as doing it "for the fun of development" is just plain wrong.  True, virtually all of them are passionate about what they're doing.  And virtually all of them are getting paid to do it.

OpenStack isn't like many other open source projects.  The vast majority of contributors do so on behalf of companies, and those companies pay their salaries not out of the goodness of their corporate hearts, but because they have a business problem to solve.  They expect OpenStack to perform a specific function for real-world clients, and when it doesn't, they pay these developers to create that functionality.  That means that OpenStack developers are, while undoubtedly enjoying themselves not to find other jobs, working on real-world problems for real world users and operators.  They're not doing it for the fun of development.

Winner take most

Matt also claims that any project that starts out like OpenStack will eventually wind up in a "winner take all" situation.

Open source tends to be winner-take-all. In enterprise Linux, the winner (Red Hat) took all. In Android, the winner (Samsung) took all. Foundation-based open-source projects with multiple downstream product vendors almost always end up as winner-take-all scenarios. No, this hasn't yet happened in Hadoop, but that's the exception, not the rule.

You know what they say in the stock market, "past performance is not an indication of future results".  OpenStack is a foundation-based open-source project, that's true.  But there are several reasons why it's different from Linux or Android.

For one thing, there are simply too many major players whose futures depend, in some non-trivial way, on OpenStack.  IBM, Dell, Cisco, HP, and 23 other companies (including, of course, Red Hat) have pledged not only financial support for OpenStack, but also "strategic alignment to the OpenStack mission".   HP and IBM are both running public clouds at least partially on OpenStack.  They are not going to roll over and let another vendor dictate to them the direction of the project.

There's also the number of individual contributors to consider.  Despite anecdotal estimates that the number of people who have worked on the Linux kernel is in the many thousands, the truth is that after 20 years, the Linux 3.10 CREDITS file lists less than 600 names.  After only 3 years, OpenStack had almost twice that many contributors in a single development cycle.

And Android was never a grass-roots project to begin with, starting as it did within Google.

All of that brings me to my final point about this, the comparison with Hadoop.  Matt cites Hadoop as the exception, but the fact is that Hadoop may just be the canary in the coal mine, showing that the commoditization we've seen in the tech industry applies not only to hardware, but also these large projects.  The days where one company can swoop in and take it all may just be gone forever.

Doing what's best

I've done a lot of talking about whether Red Hat can take over as the "leader" of OpenStack, but only time is really going to answer that question.  Instead, the central question here is whether that would be a good thing.

Clearly it would be a good thing for Red Hat.  Matt's right when he points out that OpenStack's best chance (at least right now) is in private cloud, where it's not competing with Amazon, and Red Hat is certainly the vendor that would have the most to gain in that market.  And certainly it would be good for OpenStack to have such a well-placed evangelist bringing it to the enterprise party.

But that's different from controlling the direction the project takes.  Is OpenStack development chaotic?  Certainly.  Is every idea a good idea?  Of course not.  But great ideas generally don't' come from the orderly process of a pre-planned trip.  If anything, they come out of that chaos, because that chaos is fueled by the needs of real customers, who need real solutions, right now.  If one vendor were to try and exert influence to keep development to what they thought was the right direction, we might get a project that's pretty good at one thing -- the thing that one vendor wanted -- but anybody else's use case could easily suffer neglect, and even death.

One of the interesting things about OpenStack is just how many corporate entities that would normally be competitors cooperate to make OpenStack as good as it can be.  How many of them would leave if they had to bend to Red Hat's will to make a product that was tailored to Red Hat's needs?

The last time the "leadership" question came up, our CEO, Adrian Ionel, pointed out that while leadership is important, it's not as important as momentum.  Matt points out that projects with momentum can fail, using OS/2 and OpenOffice as examples.  Well, OS/2 wasn't an open source project; it had two companies (Microsoft and IBM) behind it, one of which was also a competitor.  And OpenOffice?  Well, it's true that after buying Sun, Oracle (also a single vendor) didn't want to be saddled with it anymore.   So they gave it to the Apache Foundation, which announced in October that the foundation's version of the software had reached over 75 million downloads in less than 18 months.

It's amazing what momentum can do.

Countering the complaints

There's no mistaking that Red Hat is an established leader in OpenStack.  But we said it before, and we'll say it again: OpenStack doesn't need a single dictator, benevolent or otherwise.

So yes, it's cool to complain about OpenStack, and say that "something must be done" to change it.  But the fact is that OpenStack will grow past this "awkward teenage" period, where it's gawky and the old people think that "somebody should take that boy in hand".  Can the current process be improved?  Sure.  But the community can do that on its own, without having to have a father (or mother) figure telling it to clean up its room.