How to deploy Spinnaker on Kubernetes: a quicker and dirtier guide
- August 15, 2018
Earlier this year we gave you a quick and dirty guide to deploying Spinnaker on Kubernetes; today we're going to give you an even easier way. Two easier ways, in fact. One is incredibly easy but a little harder to control, the other is almost as easy but gives you all the control you could possibly need.
To customize this Spinnaker install, you have two choices:
In this article, I'm going to show you how to deploy Spinnaker using Helm, including how to use an external values.yaml file to customize the parameters used for deployment. Simply put, there are three steps:
As part of this process, the Helm chart gives Halyard a default configuration, and once it's up, calls
Fortunately, you have the option of overriding those default values so that Spinnaker gets deployed the way you want it. You can also use the chart to add supplemental files such as a kubeconfig to the container as it gets spawned.
https://github.com/Mirantis/k8s-apps/tree/master/charts/halyard
If you decide you want to change any of those values, create a new file called values.yaml and add the appropriate YAML to it.
For example, by default the chart assumes that your Kubernetes cluster has RBAC enabled , and that it supports the LoadBalancer service type. You can disable RBAC and change the type for the Spinnaker Gate and Deck services by overriding these chart values, like so:
You can add the contents of the halconfig file you'd like to change. Here’s an example of halyard configuration with Kubernetes v2 provider enabled:
No matter what Kubernetes you are planning to use, I strongly suggest you set a separate namespace for the Spinnaker deployment if you are using the Kubernetes v1 provider, so you can delete Spinnaker without deleting everything else along with it.
Just list the accounts and they will be created after Spinnaker installation:
Of course, you don't always have a prepared halconfig. You might want to run a halyard daemon, enter several “hal” commands, and then manually start the Spinnaker deployment. In this situation you'll find the following configuration useful:
Again, remember that if you are using Kubernetes v1 provider to deploy a Spinnaker instance, hal deploy clean will delete the entire Spinnaker namespace, even if you have other things in it. If you didn't create a namespace just for Spinnaker, you can prevent this by running
Once you've got it deployed, there's obviously a ton of things you can do with it, though some of them aren't always obvious. To learn more, please go ahead and sign up for our new Spinnaker Fundamentals course.
The "point and shoot" method: Spinnaker quick install
The easiest, quickest, dirtiest way to deploy Spinnaker to Kubernetes is to use the "quick install" Kubernetes manifest maintained by the Spinnaker community. For example:kubectl apply -f https://spinnaker.io/downloads/kubernetes/quick-install.ymlThe advantages of using this method include the fact that it gives you a serviceable install without having to do ... well, much of anything except having a working Kubernetes cluster. The disadvantage is that you have to work a bit harder if you want to do any customization, such as changing the storage or adding repositories or additional cloud providers.To customize this Spinnaker install, you have two choices:
- Download and edit your own copy of the manifest, or
- Find and log into the deployed halyard pod and perform your edits from there, then redeploy Spinnaker.
The "professional camera" method: deployment by Spinnaker Helm chart
If you want to have any significant control over your Spinnaker deployment, you're better off using a method that gives you easy access to the parameters that control it, such as a Kubernetes Helm chart.In this article, I'm going to show you how to deploy Spinnaker using Helm, including how to use an external values.yaml file to customize the parameters used for deployment. Simply put, there are three steps:
- Install and initialize Helm.
helm repo add mirantisworkloads https://mirantisworkloads.storage.googleapis.comhelm install mirantisworkloads/halyard
What's actually going on
Helm is a native Kubernetes software packager; in this case, we're using it to deploy the basic infrastructure you need to effectively use Spinnaker, such as Minio for storage and Jenkins for continuous integration. Helm will also install Halyard, which will then go on to install Spinnaker.As part of this process, the Helm chart gives Halyard a default configuration, and once it's up, calls
hal deploy applyto actually deploy Spinnaker for you.Fortunately, you have the option of overriding those default values so that Spinnaker gets deployed the way you want it. You can also use the chart to add supplemental files such as a kubeconfig to the container as it gets spawned.
Halyard chart configuration and overrides
If you want to look at the default values used by Helm (and subsequently by Halyard) you can find the Halyard chart source files on Github, here:https://github.com/Mirantis/k8s-apps/tree/master/charts/halyard
If you decide you want to change any of those values, create a new file called values.yaml and add the appropriate YAML to it.
For example, by default the chart assumes that your Kubernetes cluster has RBAC enabled , and that it supports the LoadBalancer service type. You can disable RBAC and change the type for the Spinnaker Gate and Deck services by overriding these chart values, like so:
rbac:Minio and Jenkins deployment is also optional. Both are the subcharts of the Halyard chart, and you can disable them by setting:
enabled: false
service: ui:
type: NodePort
api:
type: NodePort
minio:Of course, if you disable Minio, you’ll have to configure some storage provider instead. You can do that by configuring Halyard itself.
deployChart: false
jenkins:
deployChart: false
Adding the halconfig
Now we've arrived at the halyard configuration itself. You can provide a halconfig and any number of arbitrary files by specifying their contents in the values.yaml file.You can add the contents of the halconfig file you'd like to change. Here’s an example of halyard configuration with Kubernetes v2 provider enabled:
halconfig: |You can also add files directly to the container by adding their path and content to values.yaml:
currentDeployment: default
deploymentConfigurations:
- name: default
version: 1.8.5
providers:
kubernetes:
enabled: true
accounts:
- name: local
requiredGroupMembership: []
providerVersion: V2
serviceAccount: true
configureImagePullSecrets: true
namespaces: []
omitNamespaces: []
oauthScopes: []
oAuthScopes: []
primaryAccount: local
deploymentEnvironment:
size: SMALL
type: Distributed
accountName: local
updateVersions: true
persistentStorage:
s3:
bucket: spin-bucket
features:
auth: false
fiat: false
chaos: false
entityTags: false
jobs: true
pipelineTemplates: true
artifacts: true
timezone: America/Los_Angeles
files:Helm will create configmaps from these files and mount them into the halyard container.
- path: /home/spinnaker/.hal/default/service-settings/deck.yml
content: |
env:
API_HOST: "http://spin-gate:8084"
- path: /home/spinnaker/.hal/default/service-settings/gate.yml
content: |
overrideBaseUrl: "/gate"
- path: /home/spinnaker/.hal/default/profiles/front50-local.yml
content: |
spinnaker.s3.versioning: false
Working with kubeconfig
Of course if you're deploying to Kubernetes you'll need to have the appropriate kubeconfig file. The Helm chart assumes you're deploying to the Kubernetes cluster Helm is running in, but you might want to deploy Spinnaker in another Kubernetes cluster. In that case, you won't need Helm to prepare a kubeconfig for the current cluster and you could disable that:prepareKubeconfig:Note that you'll also want to provide the new kubeconfig as a file, as we did in the previous section and point the halconfig towards it.
enabled: false
No matter what Kubernetes you are planning to use, I strongly suggest you set a separate namespace for the Spinnaker deployment if you are using the Kubernetes v1 provider, so you can delete Spinnaker without deleting everything else along with it.
prepareKubeconfig:
namespace: my-spinnaker-nsIf you don't set this value, Halyard will deploy Spinnaker in the same namespace in which Halyard itself is running.A few more handy tasks
Another thing provided by the chart that might be handy is Spinnaker service accounts creation.Just list the accounts and they will be created after Spinnaker installation:
serviceAccounts:Finally, there are also two variables in values.yaml you should know about:
- name: example
memberOf: ["example"]
daemon: false
command: "hal deploy apply"Leaving them with defaults will lead to a k8s job creation that will configure halyard, deploy the Spinnaker instance and shut itself down. This is useful when you have already prepared a halconfig file and want to use a helm release as storage for the halyard configuration. The halyard job is registered as a “post-install,post-upgrade” hook in Helm and will be rescheduled on each upgrade of the release. That means that Spinnaker configuration will be reapplied on each upgrade as well.Of course, you don't always have a prepared halconfig. You might want to run a halyard daemon, enter several “hal” commands, and then manually start the Spinnaker deployment. In this situation you'll find the following configuration useful:
daemon: true
command: "sleep infinity"In this case, Helm creates the Halyard daemon as a Kubernetes Deployment. You’ll be able to get into the halyard container with kubectl exec, experiment with configuration, deploy with hal deploy apply, fix the errors, deploy again… And you can later get the halconfig you’ve prepared inside the halyard container and use it in your future deployment.Cleanup
When you're finished and you want to clean up your Spinnaker deployment, nothing could be easier. Simply runhelm delete <halyard_release_name>Which will remove all objects associated with the release, as well as the deployed Spinnaker instance, by running hal deploy clean in the pre-delete hook. (You can get the <halyard_release_name> by running helm list.)Again, remember that if you are using Kubernetes v1 provider to deploy a Spinnaker instance, hal deploy clean will delete the entire Spinnaker namespace, even if you have other things in it. If you didn't create a namespace just for Spinnaker, you can prevent this by running
helm delete <halyard_release_name> --no-hooksThe --no-hooks flag prevents triggering of the pre-delete hook that is responsible for Spinnaker deployment cleanup. (But remember, you won't have Spinnaker deleted. Remember to set a namespace or use Kubernetes v2 provider for deployment.)Moving on from here
So that's a quicker, dirtier, and most of all easier way to deploy Spinnaker to Kubernetes.Once you've got it deployed, there's obviously a ton of things you can do with it, though some of them aren't always obvious. To learn more, please go ahead and sign up for our new Spinnaker Fundamentals course.
