Kustomize vs Helm - Grudge Match, or Match Made in Heaven?

Kubernetes is complicated, so it’s no wonder that a host of tools have emerged to help operators make use of it. As the container orchestrator has skyrocketed into widespread adoption, two open source tools — the configuration manager Kustomize and the “package manager for Kubernetes” Helm — have emerged as essential ingredients for many organizations’ Kubernetes implementations.

Both Helm and Kustomize provide answers to a common problem for Kubernetes users: the YAML piles up. Much of Kubernetes’ power derives from its use of declarative markup manifests for the various elements of a deployment. But when you start dealing with the enterprise architectures that Kubernetes is really tailored for — especially across multiple environments — the manifests can quickly become unwieldy. The question becomes: How can we organize and manage configuration simply in a large-scale deployment that may encompass dozens of microservices?

The open source community got to work (as is its way) and built two tools that provide potential answers. Helm and Kustomize take two very different approaches to the question of organization, and commentators (as is their way) began to debate the relative merits of both.

Whether you’re learning Kubernetes or plotting your organization’s Kubernetes gameplan, you may be wondering which tool you should use. In order to determine an answer, we’ll need to dig a bit into both.

What is Kustomize?

In short, Kustomize is a command-line tool that provides a way to manage your deployment configuration. It maintains the broader declarative philosophy of Kubernetes, so that configurations can remain consolidated and easily tracked in version control, without forking, even across multiple environments with different requirements.

So in the common scenario where you might need to deploy a given app to both a dev and production environment, with differing configurations for each, Kustomize has you covered. It accomplishes all of this by using a “patching” system — when you need to specify differing configurations, you simply do so through a kustomize.yaml file that, once applied, automatically updates YAML declarations in an environment-specific way.

For example, if you wanted to specify different patches for your development and production environments, you would simply specify those changes in YAML files within their own directories, organized in an “overlays” directory that sits beside a common “base” in your project tree.

~/app-project
├── base
│   ├── deployment.yaml
│   └── kustomization.yaml
└── overlays
    ├── development
    │   ├── kustomization.yaml
    │   └── important_config.yaml
    └── production
        ├── kustomization.yaml
        └── important_config.yaml

The customizations are organized and easy to keep in source control. And with your project arranged as above, you can simply enter the following command to apply them selectively to your running cluster:

kustomize build ~/app-project/overlays/development | kubectl apply -f -

If you’d like a much more in-depth look at Kustomize, see our Kustomize tutorial, which walks you through the process of patching a WordPress application. For now, let’s review the benefits:

• Simplified multi-environment configuration
• Declarative approach for consistency and ease of version control
• Seamless integration with kubectl

Like Kubernetes itself, Kustomize developed as an open source project out of Google — so it may not come as a surprise that as of Kubernetes 1.14, Kustomize is integrated directly into k8s as part of kubectl, the default Kubernetes command-line tool. That means Kustomize brings a lot of utility to Kubernetes right out of the box.

So what does Helm have to offer?

What is Helm?

Helm bills itself as “the package manager for Kubernetes.” Supported as a Graduated project of the Cloud Native Computing Foundation (like Kubernetes itself), Helm aims to bring the simplicity of a package manager like apt, snap, or homebrew to the Kubernetes cluster. Once you have a Helm set up with your Kubernetes cluster, it’s possible to deploy an app like WordPress with only a couple of commands.

With a simple...

helm repo add mirantis https://charts.mirantis.com

...you can add a repository to draw on for Helm “charts,” Helm’s nomenclature for packages that install Kubernetes resources. In the example above, I’m adding the repository maintained by Mirantis. And now I can deploy, say, nginx (with a randomly generated name for the release) in a line:

helm install mirantis/nginx --generate-name

What's more, Lens users will know that an IDE with Helm integration can reduce a deployment to little more than a click.

Helm brings all the complexity reduction of a package manager to the notoriously complex Kubernetes environment. Like any good package manager, it simplifies updates and even rollbacks. And once you start building charts, they’re easy to place in version control and share.

That’s a lot of benefits, especially if you put a premium on reducing developer friction. So how do we weigh Kustomize and Helm against one another?

Weighing Your Options: Kustomize vs Helm

For those doing their cluster management on the command line, Helm will take some setup. Unlike Kustomize, Helm isn’t part of a baseline Kubernetes installation, so you’ll need to install it using the package manager of your choice or build it from binary. The actual installation is perfectly simple, but there’s a question of complexity management here: do you want to introduce an additional moving part to your architecture?

For many use-cases, the benefits will vastly outweigh the addition of this particular layer. But it’s worth noting, too, that adding another piece to your puzzle will always have security implications. Famously, V2 of Helm (it’s on 3.7.0 as of this writing) temporarily introduced major security headaches by incorporating a module that could expose Secrets in plaintext. Those vulnerabilities are long-addressed, but teams may reasonably ask how many variables they want to monitor.

In contrast with Helm, Kustomize adds another cognitive layer to operations. Even if it’s included with Kubernetes, it’s an additional tool to learn, and a team will want to consider whether it solves a problem for them. Are they staring down an intimidating YAML sprawl across dev and staging and production environments? If so, Kustomize might make sense for them. But if they’re really worried about ease of development or sharing their Kubernetes applications with others, Helm might be better-suited to their needs.

The real question, perhaps, is whether Kustomize and Helm are in contest at all.

Using Helm and Kustomize together

Most readers will have noticed by this point that while the tools can address similar problems, they’re quite different — one is essentially a templating engine that is particularly good at making multi-environment deployments manageable, and the other is a package manager that makes Kubernetes application development — and sharing — easier. Small teams particularly concerned with building quickly (or making their projects easy for others to deploy) may be drawn to Helm. Teams focused on mitigating the complexity of distributed applications in multiple environments may gravitate toward Kustomize. But the fact is, these two tools are actually accomplishing different jobs and, in many situations, may be happily used in conjunction.

Helm makes installation and updates a breeze, but it’s not so keen on customization. Kustomize...well, it’s right there in the name. Kustomize helps configure our deployments. Using Helm and Kustomize together is a reasonable option for teams who may wish to, say, install certain applications via Helm and then, ahem, kustomize them for their various environments — for example, configuring an app differently for dev, QA, and production.

That’s a common set of needs. So while there is no definitive answer for every context — some teams may only need Helm, while Kustomize will be a life-saver for others — it’s important for any team to reframe the question, so that they’ve adequately examined the possibilities. This isn’t an either/or situation...and often, it’s both/and.