Best of 2019 Blogs, Part 2: Building Your First Certified Kubernetes Cluster On-Premises
This week and next, we're bringing you the top 5 most popular blog posts about Docker Enterprise from 2019. The most popular post by far was a guest post by Ajeet Raina diving in to how to run Kubernetes on Docker Enterprise. Ajeet is a Docker Captain. You can follow Ajeet on Twitter @ajeetsraina and read his blog at http://www.collabnix.com.
There are now a number of options for running certified Kubernetes in the cloud. But let’s say you’re looking to adopt and operationalize Kubernetes for production workloads on-premises. What then? For an on-premises certified Kubernetes distribution, you need an enterprise container platform that allows you to leverage your existing team and processes.
Enter Docker Kubernetes Service
At DockerCon 2019, Docker announced the Docker Kubernetes Service (DKS). It is a certified Kubernetes distribution that is included with Docker Enterprise 3.0 and is designed to solve this fundamental challenge.
In this blog series, I’ll explain Kubernetes support and capabilities under Docker Enterprise 3.0, covering these topics:
Deploying certified Kubernetes Cluster using Docker Enterprise 3.0 running on a Bare Metal System
Implementing Persistent storage for Kubernetes workload using iSCSI
Implementing Cluster Ingress for Kubernetes
Deploying Istio Service Mesh under Docker Enterprise 3.0
Support of Kubernetes on Windows Server 2019 with Docker Enterprise 3.0
So About DKS...
DKS is the only offering that integrates Kubernetes from the developer desktop to production servers, with ‘sensible secure defaults’ out-of-the-box. Simply put, DKS makes Kubernetes easy to use and more secure for the entire organization. Here are three things that DKS does to simplify and accelerate Kubernetes adoption for the enterprise:
Consistent, seamless Kubernetes experience for developers and operators. With the use of Version Packs, developers’ Kubernetes environments running in Docker Desktop Enterprise stay in sync with production environments for a complete, seamless Kubernetes experience.
Streamlined Kubernetes lifecycle management (Day 1 and Day 2 operations). A new Cluster Management CLI Plugin allows operations teams to easily deploy, scale, backup and restore and upgrade a certified Kubernetes environment using a set of simple CLI commands.
Enhanced security with ‘sensible defaults.’ Teams get out-of-the-box configurations for security, encryption, access control, and lifecycle management — all without having to become Kubernetes experts.
DKS is compatible with Kubernetes YAML, Helm charts, and the Docker Compose tool for creating multi-container applications. It also provides an automated way to install and configure Kubernetes applications across hybrid and multi-cloud deployments. Capabilities include security, access control, and lifecycle management. Additionally, it uses Docker Swarm Mode to orchestrate Docker containers.
Kubernetes 1.14+ in Docker Enterprise
Docker Enterprise 3.0 comes with the following components:
Containerd 1.2.6
Docker Engine 19.03.1
Runc 1.0.0-rc8
docker-init 0.18.0
Universal Control Plane 3.2.0
Docker Trusted Registry 2.7
Kubernetes 1.14+
Calico v3.5.7
Docker UCP manager and worker nodes.
In this first post of the series, I will show you how to deploy a Certified Kubernetes cluster using Docker Enterprise 3.0 on bare metal (meaning you can deploy on-premises).
Pre-Requisites:
Ubuntu 18.04 (at least 2 Node to setup Multi-Node Cluster)
Minimal 4GB RAM is required for UCP 3.2.0
Click the Setup button for Docker Enterprise Edition for Ubuntu.
Copy the URL from the field labeled Copy and paste this URL to download your Edition.
Now you’re ready to start installing Docker Enterprise and Kubernetes. Let’s get started.
Step 1: Install packages to allow apt to use a repository over HTTPS
$sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
software-properties-commonStep 2: Add the $DOCKER_EE_URL variable into your environment
Replace with the URL you noted down in the prerequisites. Replace sub-xxx too.
$curl -fsSL https://storebits.docker.com/ee/m/sub-XXX-44fb-XXX-b6bf-XXXXXX/ubuntu/gpg |
sudo apt-key add -Step 3: Add the stable Repository
$sudo add-apt-repository \
"deb [arch=amd64] https://storebits.docker.com/ee/m/sub-XXX-44fb-XXX-b6bf-XXXXXX/ubuntu \
$(lsb_release -cs) \
stable-19.03"Step 4: Install Docker Enterprise
$sudo apt-get install docker-ee docker-ee-cli containerd.ioStep 5: Verifying Docker Enterprise Version
$ sudo docker version
Client: Docker Engine - Enterprise
Version: 19.03.1
API version: 1.40
Go version: go1.12.5
Git commit: f660560
Built: Thu Jul 25 20:59:23 2019
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Enterprise
Engine:
Version: 19.03.1
API version: 1.40 (minimum version 1.12)
Go version: go1.12.5
Git commit: f660560
Built: Thu Jul 25 20:57:45 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.6
GitCommit: 894b81a4b802e4eb2a91d1ce216b8817763c29fb
runc:
Version: 1.0.0-rc8
GitCommit: 425e105d5a03fabd737a126ad93d62a9eeede87f
docker-init:
Version: 0.18.0
GitCommit: fec3683
cse@ubuntu1804-1:~$Step 6: Test the Hello World Example
$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:6540fc08ee6e6b7b63468dc3317e3303aae178cb8a45ed3123180328bcc1d20f
Status: Downloaded newer image for hello-world:latest
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the executable
that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it to your
terminal.To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID by signing up or logging in to Docker Hub.
For more examples and ideas, visit the Docker Docs getting started page.
Step 7: Install Universal Control Plane v3.2.0
$ sudo docker container run --rm -it --name ucp \
> -v /var/run/docker.sock:/var/run/docker.sock \
> docker/ucp:3.2.0 install \
> --host-address 10.94.214.115 \
> --interactiveStep 8: Accessing the UCP
Now you should be able to access Docker Universal Control Plane via https://<node-ip>
Click on “Sign In” and upload the license file to access Docker Enterprise UCP 3.2.0 WebUI as shown below:
Step 9: Adding Worker Nodes to the Cluster
Let’s add worker nodes to the cluster. Click on “Shared Resources” on the left pane and Click on “Nodes”. Select “Add Nodes” and choose an orchestrator. You can also add either Linux or Windows nodes to the cluster here as shown below:
I assume that you have a worker node installed with Ubuntu 18.04 and the latest Docker binaries (it can be either the free version of Docker Engine or Docker Enterprise).
@ubuntu1804-1:~$ sudo curl -sSL https://get.docker.com/ | sh
$ sudo usermod -aG docker cs
$ sudo docker swarm join --token SWMTKN-1-3n4mwkzhXXXXXXt2hip0wonqagmjtos-bch9ezkt5kiroz6jncid
rz13x <managernodeip>:2377
This node joined a swarm as a worker.By now, you should be able to see both manager node and 1 worker node added under UCP.
If you see a warning on the UCP dashboard stating that you have a similar hostname on both the manager and worker node, change the hostname on the worker node and it will automatically get updated on UCP dashboard.
Step 10: Install the Docker Client Bundle
Click on Dashboard and scroll down to see the Docker CLI option. This option allows you to download a client bundle to create and manage services using the Docker CLI client. Once you click, you will be able to find a new window as shown below:
Click on “user profile page” and it should redirect you to https://<manager-ip-node/manage/profile/clientbundle page as seen in the below screenshot:
Click on “Generate Client Bundle” and it will download ucp-bundle-<username>.zip
$ unzip ucp-bundle-ajeetraina.zip
Archive: ucp-bundle-ajeetraina.zip
extracting: ca.pem
extracting: cert.pem
extracting: key.pem
extracting: cert.pub
extracting: kube.yml
extracting: env.sh
extracting: env.ps1
extracting: env.cmd
extracting: meta.json
extracting: tls/docker/key.pem
extracting: tls/kubernetes/ca.pem
extracting: tls/kubernetes/cert.pem
extracting: tls/kubernetes/key.pem
extracting: tls/docker/ca.pem
extracting: tls/docker/cert.pem
@ubuntu1804-1:~$ eval "$(<env.sh)"The env script updates the DOCKER_HOST and DOCKER_CERT_PATH environment variables to make the Docker CLI client interact with UCP and use the client certificates you downloaded. From now on, when you use the Docker CLI client, it includes your user specific client certificates as part of the request to UCP.
Step 11: Install Kubectl on Docker Enterprise 3.0
Once you have the Kubernetes version, install the kubectl client for the relevant operating system. As shown below, we need to install Kubectl version 1.14.3:
Step 12: Set the Kubectl version
@ubuntu1804-1:~$ k8sversion=v1.14.3
@ubuntu1804-1:~$ curl -LO https://storage.googleapis.com/kubernetes-release/release/
$k8sversion/bin/linux/amd64/kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 41.1M 100 41.1M 0 0 7494k 0 0:00:05 0:00:05 --:--:-- 9070k
@ubuntu1804-1:~$ chmod +x ./kubectl
@ubuntu1804-1:~$ sudo mv ./kubectl /usr/local/bin/kubectl
@ubuntu1804-1:~$Step 13: Verify the Kubectl Installation
~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:
"5e53fd6bc17c0dec8434817e69b04a25d8ae0ff0", GitTreeState:"clean", BuildDate:
"2019-06-06T01:44:30Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.3-docker-2",
GitCommit:"7cfcb52617bf94c36953159ee9a2bf14c7fcc7ba", GitTreeState:"clean",
BuildDate:"2019-06-06T16:18:13Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"Step 14: List out the Kubernetes Nodes
cse@ubuntu1804-1:~$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
node2 Ready <none> 23h v1.14.3-docker-2
ubuntu1804-1 Ready master 23h v1.14.3-docker-2Step 15: Enabling Helm and Tiller with UCP
$ kubectl create rolebinding default-view --clusterrole=view --serviceaccount=kube-system
:default --namespace=kube-system
rolebinding.rbac.authorization.k8s.io/default-view created
$ kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin
--serviceaccount=kube-system:default
clusterrolebinding.rbac.authorization.k8s.io/add-on-cluster-admin created
cse@ubuntu1804-1:~$Step 16: Install Helm
$ curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get > install-helm.sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 7001 100 7001 0 0 6341 0 0:00:01 0:00:01 --:--:-- 6347
$ chmod u+x install-helm.sh
$ ./install-helm.sh
Downloading https://get.helm.sh/helm-v2.14.3-linux-amd64.tar.gz
Preparing to install helm and tiller into /usr/local/bin
helm installed into /usr/local/bin/helm
tiller installed into /usr/local/bin/tiller
Run 'helm init' to configure helm.
cse@ubuntu1804-1:~$ helm init
Creating /home/cse/.helm
Creating /home/cse/.helm/repository
Creating /home/cse/.helm/repository/cache
Creating /home/cse/.helm/repository/local
Creating /home/cse/.helm/plugins
Creating /home/cse/.helm/starters
Creating /home/cse/.helm/cache/archive
Creating /home/cse/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /home/cse/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users'
policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/
#securing-your-helm-installation
cse@ubuntu1804-1:~$Step 17: Verify the Helm Installation
$ helm version
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085"
, GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085"
, GitTreeState:"clean"}Step 18: Deploying MySQL using Helm on Docker Enterprise 3.0
Let’s try out deploying MySQL using HelmPack.
$ helm install --name mysql stable/mysql
NAME: mysql
LAST DEPLOYED: Wed Aug 7 11:43:01 2019
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/ConfigMap
NAME DATA AGE
mysql-test 1 0s
==> v1/PersistentVolumeClaim
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
mysql Pending 0s
==> v1/Secret
NAME TYPE DATA AGE
mysql Opaque 2 0s
==> v1/Service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mysql ClusterIP 10.96.77.83 <none> 3306/TCP 0s
==> v1beta1/Deployment
NAME READY UP-TO-DATE AVAILABLE AGE
mysql 0/1 0 0 0s
NOTES:
MySQL can be accessed via port 3306 on the following DNS name from within your cluster:
mysql.default.svc.cluster.local
To get your root password run:
MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace default mysql -o jsonpath=
"{.data.mysql-root-password}" | base64 --decode; echo)
To connect to your database:
1. Run an Ubuntu pod that you can use as a client:
kubectl run -i --tty ubuntu --image=ubuntu:16.04 --restart=Never -- bash -il
2. Install the mysql client:
$ apt-get update && apt-get install mysql-client -y
3. Connect using the mysql cli, then provide your password:
$ mysql -h mysql -p
To connect to your database directly from outside the K8s cluster:
MYSQL_HOST=127.0.0.1
MYSQL_PORT=3306
# Execute the following command to route the connection:
kubectl port-forward svc/mysql 3306
mysql -h ${MYSQL_HOST} -P${MYSQL_PORT} -u root -p${MYSQL_ROOT_PASSWORD}
cse@ubuntu1804-1:~$Step 19: Listing out the Releases
The helm list command lists all of the releases. By default, it lists only releases that are deployed or failed. Flags like ‘–deleted’ and ‘–all’ will alter this behavior. Such flags can be combined: ‘–deleted –failed’. By default, items are sorted alphabetically. Use the ‘-d’ flag to sort by release date.
$ helm list
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
mysql 1 Wed Aug 7 11:43:01 2019 DEPLOYED mysql-1.3.0 5.7.14 default
$ kubectl get po,deploy,svc
NAME READY STATUS RESTARTS AGE
pod/mysql-6f6bff58d8-t2kwm 1/1 Running 0 5m35s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.extensions/mysql 1/1 1 0 5m35s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 28h
service/mysql ClusterIP 10.96.77.83 <none> 3306/TCP 5m35s
cse@ubuntu1804-1:~$With DKS, you can use Helm flawlessly with UCP under Docker Enterprise 3.0.
Kubernetes, On-Premises
Now you have Kubernetes running on-premises. You can do a lot from here, and I’ll cover some possibilities in the rest of this series.
You may also want to experiment with designing your first application in Kubernetes. Bill Mills from the Docker training team wrote a great blog series recently covering just that. I highly recommend checking it out starting with part 1 here.
This post originally appeared on the Docker blog on September 30, 2019.
