7 Reasons to Switch to Managed On-Prem Kubernetes
Managing Kubernetes on premises is hard. Self-managed, deep-stack solutions claim to make life easier for Dev and Ops, but they come with complexity, vendor lock-in, and other costs. And better alternatives exist if you start thinking outside the box.
Folks who need an on-premises Kubernetes cluster are typically pretty smart about technology. Over the past five or so years, and for good reasons, some have chosen to take the ‘DIY’ approach (many fail). Others have chosen (and not unwisely) to invest in deep-stack, opinionated, single-vendor solutions. Think Tanzu from VMware.
This is not dumb. Opinionated stacks are good. When you buy into a deep, opinionated stack, you know someone has thought hard about:
How to manage underlying infrastructure(s).
Kubernetes version viability and core component hardening.
Container runtime selection, Kubernetes networking, storage, and other required features of production clusters.
Cluster integration with infrastructure networking, plus integrations with service mesh and other production adjuncts.
Security and access control.
Basic application services that can be invoked by users (such as application catalogs) and shared by apps running on the platform.
Platform operationalization and ops automation such as observability, plus webUIs and CLIs that help you run the whole kit-and-kaboodle. This is usually aimed at both system operators (generalists keeping the platform healthy) and teams (project members, those doing application-layer DevOps, and so on).
Scalability and high availability.
Cost analytics (for example, chargeback, billing) and FinOps.
This is a ton of stuff that you don’t need to think about in terms of what to choose and how to TinkerToy it all together. Yay! Furthermore, in delivering all this integrated stuff, you know these folks have thought long and hard (along with their customers) about what’s required to make a Kubernetes cluster into a technically-coherent, coder-and-operator-friendly platform. This puts you literally miles ahead of the DIY crowd, who are still looking at the CNCF cloud-native landscape diagram and scratching their heads.
Better providers of these solutions (or so the reasoning often goes), have done a lot of deep thinking around what DevOps and developers need from Kubernetes, which is more than the flexible-but-hypercomplex kit-of-parts that Kubernetes itself provides. In VMware’s case, this means:
Some kind of (usually very opinionated) build system for curating components and creating/configuring container workloads.
A highly opinionated, relatively “thick” platform-as-a-service (PaaS); an abstraction layer that makes it easier to write applications in the language(s) and environment(s) you like, containerize them, help their components interoperate, and run them – safely, resiliently, scalably, updateably – in production.
It all sounds great!
At least until you start looking at long-term budgets, limitations on your freedom to evolve technologically, and actual impacts on operations workloads and developer productivity. Here are a few reasons why – even though these things sound great – deep-stack solutions like VMware Tanzu have the wrong approach to on-premises Kubernetes today:
- Little lock-in factors everywhere.
VMware’s Tanzu is engineered as the pinnacle of a VMware vSphere infrastructure stack that limits choice all the way down to the VMware-certified metal. You can’t deploy Tanzu on commodity bare metal, though some public clouds are supported. On your vSphere cloud, complex per-CPU, version-specific licensing rules govern how you set up every Kubernetes cluster you deploy, and ensure that you keep spending defensively if you don’t want to paint yourself into a corner (for example, by deploying a production cluster and then discovering you don’t have enough licenses to keep it running past the 60-day evaluation period).
Particularly if you’re thinking about eventually moving off VMware as an on-premises IaaS (as many currently are), it makes huge sense to look beyond this model.
- Constrained choice is good - but this constrained?
Take an example anywhere in the stack, such as: what if you don’t like Tanzu Service Mesh, which is a VMware NSX fork, and a very nice service mesh? But it's also not the only service mesh: think Istio, LinkerD, Cilium, Consul, and so on. Their answer: sure, it’s possible. And getting it means figuring it out with your VMware consultant, who’s not familiar with it, or getting help from the vendor, and then maintaining it in place because your normal support downline won’t support this thing. The result? Endless multi-vendor finger-pointing when problems arise.
The take-away is that opinions are super-helpful, but the Kubernetes ecosystem moves fast and nobody’s requirements stay the same forever. And opinionated commercial platforms that aren’t backed by a specific, proactive service obligation that emphasizes your freedom to choose will a) end up feeling limiting, and b) throw you back into the realms of multi-vendor DIY chaos you sought to escape by buying “opinionated” in the first place.
- Complexity-hiding: a pretty webUI looks nice, but it doesn’t make everything easy.
A nice webUI (for example, for cluster creation, app catalog, and so on) is a given these days. It speeds up absolutely routine operations and facilitates (absolutely routine) developer self-service. But to do anything big and real with Kubernetes, Tanzu still throws you into the tangle of APIs and CLIs and YAML.
The takeaway: real Day 2-and-beyond experience with Kubernetes is still complicated, requiring many skills and lots of potentially-error-prone, fundamentally manual work (later, hopefully, automated). And any full-stack solution provides no more than a fig-leaf covering this complexity. So the question always, in the end, becomes who will hold your hand? And the sad answer is often “not the vendor – you’re pretty much on your own.” This means big money spent, long-term, on highly-skilled heads – many of whom are only looking at your platform (which has turned out not to be a commodity even though common sense says it should be) and not at what will help your business win.
- Meet the new PaaS (same as the old PaaS).
There’s nothing wrong with the idea of abstracting away some Kubernetes complexity and presenting developers with a “platform” that lets them use existing skills to deliver efficient workloads to your Kubernetes for production. But how you do this really matters.
In Tanzu’s case, their Tanzu Application Service layer is basically Pivotal Cloud Foundry, a heavy, complex, holistic abstraction that looks to operators and developers like a whole new container orchestration layer. Unsurprising, since Cloud Foundry predates Kubernetes (earlier versions run on IaaS directly) and has largely been superseded by Kubernetes at organizations that recognize K8s as providing more flexibility and agility, at scarcely greater cost of complexity.
Redefining the challenges of on-prem Kubernetes
Fortunately, there are alternative ways of looking at the on-premises Kubernetes challenges that solutions like Tanzu claim to solve (but really don’t). And they’re not hard to access, if you think a little bit outside the box.
- Opinionated stacks are great - and you can have them without lock-in, inflexibility, and roadmap roadblocks.
There is a new breed of vendors with deep roots in cloud native open source who can now provide you with extended Kubernetes platforms that are fully-built-out, pre-integrated, and hardened-and-secured for production. And these platforms can run on whatever you need them to: commodity bare metal, vSphere/vCenter IaaS, hyper-converged OpenStack, public clouds. Their management planes run on a range of enterprise Linux distros, helping you avoid lock-in to OS vendors. They accommodate Windows workers for your Windows container workloads. They do all the latest cool things (like leverage Kubernetes ClusterAPI to manage infra) and provide all the centralized deployment automation, built-in observability, and pretty webUIs that enable efficient ops and developer self-service. And they come with simple licensing terms that acknowledge the dynamic, scalable nature of Kubernetes and don’t paint you into corners so you pay, pay, pay.
- You can have (reasonable) freedom of choice, too!
Innovative new vendors that specialize in an opinionated-open-k8s-stack acknowledge there are more than one service mesh or build system in the world, and can flex to help you implement and then support what you need. And this is true up and down the stack: infrastructure integration, platform services, build toolchains, CI/CD, and whatever abstraction framework or PaaS (or multiples of these) you select.
- You’ve got a friend. You’ll never walk alone.
Kubernetes will never be simple, but can always be made simple-er. And that means most of your TCO for Kubernetes – even when you go with an ‘opinionated stack’ – goes directly into managing the stack: skilled engineers, lots of time, 24/7 on-call rotations. Most of this is really for “keeping the lights on” and has nothing to do with helping your business win.
Innovative new vendors that specialize in opinionated-stack Kubernetes know this. So they structure their ways of partnering with you to solve the interconnected challenges of irreducible complexity, skills shortage, and highly-compensated expert labor. They build up and cross-train and pay the experts who can help you implement your cloud, adapt it to your needs, maintain its many moving parts, and help you evolve it over time. Ultimately, they let you use Kubernetes “as a service.” They enable you to develop and run apps on Kubernetes “as a service.” And the best do this at low, predictable costs far less than you could afford to do on your own by assembling and managing staff with equivalent expertise.
FINANCIAL SERVICES
Cloud expertise for financial services.
Run business-critical applications on a cloud designed for financial services—backed by cloud experts with over a decade of experience.
LEARN MORE
