k0smotron v0.7.0 brings remote machines via Cluster API, client tunneling

The v0.7.0 release highlights two new features: client connection tunneling and the new Cluster API (CAPI) provider for “remote machines.”

Remote machine CAPI provider, a.k.a "k0smotron Anywhere"

Cluster API is a Kubernetes sub-project working to provide Kubernetes-native, declarative means for provisioning, upgrading, and operating multiple clusters on various infrastructures. One of the tenets of CAPI is that it can be easily used with cloud providers and other infrastructure that is API-powered. However, there are a lot of use cases where infrastructure is not API-powered (think: bare metal servers, IoT devices, etc.). In many of these, it doesn’t make sense to invest resources into making the infrastructure API powered.

There are some CAPI providers that can work with bare metal infrastructure, but all of them require some other components — adding complexity and potentially cost

One of the promises of k0s, the Kubernetes distro that powers k0smotron-managed clusters, is that it runs anywhere. So we started to look into how we could enable k0smotron clusters even in the simplest of infrastructures. What we mean by that is that k0smotron should be able to create and manage clusters and infrastructure admins should not need to run any additional tooling to make this possible.

When thinking about this, we remembered the success of our existing k0sctl tool. For those of you who have not used k0sctl, it’s a tool that can provision and manage a single cluster via YAML configuration. Essentially, you need to provide addresses and SSH credentials for each host you want to set up as part of the cluster. k0sctl connects to the hosts using SSH and orchestrates the whole cluster setup.

apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
  - ssh:
      address: 10.0.0.1
      user: root
      port: 22
    role: controller
  - ssh:
      address: 10.0.0.2
      user: root
      port: 22
    role: worker
  k0s:
    version: 1.28.2+k0s.0

Enter `RemoteMachines`

A RemoteMachine is a machine that can be provisioned remotely via SSH. k0smotron now controls the process of bootstrapping this kind of minimal node. Like in any other Cluster API provider, the scope for a machine controller is very limited. They “only” need to provision the machine using a bootstrap provider generated cloud-init configuration. So in the case of `RemoteMachine` controller, it simply does the following:

• Wait to see the bootstrap cloud-init secret for the machine

• Connect to the machine via SSH

• Execute the cloud-init steps via SSH

Let’s see an example:

apiVersion: cluster.x-k8s.io/v1beta1
kind: Machine
metadata:
  name:  remote-test-0
  namespace: default
spec:
  clusterName: remote-test
  bootstrap:
    configRef:
      apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
      kind: K0sWorkerConfig
      name: remote-test-0
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
    kind: RemoteMachine
    name: remote-test-0
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: K0sWorkerConfig
metadata:
  name: remote-test-0
  namespace: default
spec:
  version: v1.27.2+k0s.0
---
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: RemoteMachine
metadata:
  name: remote-test-0
  namespace: default
spec:
  address: 1.2.3.4
  port: 22
  user: root
  sshKeyRef:
    # This defines which SSH key to use for connecting to the machine. The Secret needs to have key 'value' with the SSH private key in it.
    name: footloose-key

So what you need to configure for your RemoteMachine is the address to connect to, the username, and which SSH key to use — via a Kubernetes secret, for security. As simple as that.

Of course, with RemoteMachines you cannot reap the full benefits of ClusterAPI. Things like automated updates via MachineDeployment are not possible. But what it does enable is, for example, managing your on-premise bare metal nodes via Cluster API. How cool is that!?

Client connection tunneling

In the previous k0smotron release, we introduced a control plane provider to also run the control plane using CAPI Machines. This enables you to use CAPI to provision more traditional clusters where both control and worker planes run on CAPI provider-managed infrastructure.

What if the infrastructure to run the Machines can’t be connected to directly by cluster admins? How can cluster admins connect to k0smotron child clusters with tools like kubectl and Lens?

For this use case, we’ve enabled tunneling configuration. What this does is set up a tunnel between the child cluster and management cluster in a way that can be used by clients. In the end, clients connect to a tunnel endpoint on the management cluster and via that, can access the child cluster’s API.

Read more in the k0smotron docs.

User survey

In case you’ve missed it, we launched a k0s user survey a couple weeks ago. By answering this survey, you’ll help us prioritize features that mean the most to YOU, and in general, you’ll help us to shape the future of k0s and k0smotron.