A Rancher and Kubernetes optimized immutable Linux distribution based on openSUSE

Related tags

DevOps Tools os2

RancherOS v2


RancherOS v2 is an immutable Linux distribution built to run Rancher and it's corresponding Kubernetes distributions RKE2 and k3s. It is built using the cOS-toolkit and based on openSUSE. Initial node configurations is done using only a cloud-init style approach and all further maintenance is done using Kubernetes operators.

Use Cases

RancherOS is intended to be ran as the operating system beneath a Rancher Multi-Cluster Management server or as a node in a Kubernetes cluster managed by Rancher. RancherOS also allows you to build stand alone Kubernetes clusters that run an embedded and smaller version of Rancher to manage the local cluster. A key attribute of RancherOS is that it is managed by Rancher and thus Rancher will exist either locally in the cluster or centrally with Rancher Multi-Cluster Manager.


OCI Image based

RancherOS v2 is an A/B style image based distribution. One first runs on a read-only image A and to do an upgrade pulls a new read only image B and then reboots the system to run on B. What is unique about RancherOS v2 is that the runtime images come from OCI Images. Not an OCI Image containing special artifacts, but an actual Docker runnable image that is built using standard Docker build processes. RancherOS is built using normal docker build and if you wish to customize the OS image all you need to do is create a new Dockerfile.


RancherOS v2 includes no container runtime, Kubernetes distribution, or Rancher itself. All of these assests are dynamically pulled at runtime. All that is included in RancherOS is rancherd which is responsible for bootstrapping RKE2/k3s and Rancher from an OCI registry. This means an update to containerd, k3s, RKE2, or Rancher does not require an OS upgrade or node reboot.


RancherOS v2 is initially configured using a simple version of cloud-init. It is not expected that one will need to do a lot of customization to RancherOS as the core OS's sole purpose is to run Rancher and Kubernetes and not serve as a generic Linux distribution.

RancherOS Operator

RancherOS v2 includes an operator that is responsible for managing OS upgrades and assiting with secure device onboarding (SDO).

openSUSE Leap

RancherOS v2 is based off of openSUSE Leap. There is no specific tie in to openSUSE beyond that RancherOS assumes the underlying distribution is based on systemd. We choose openSUSE for obvious reasons, but beyond that openSUSE Leap provides a stable layer to build upon that is well tested and has paths to commercial support, if one chooses.

  • Server is not automatically reboot after installation

    Server is not automatically reboot after installation

    I generated a RancherOS iso with elemental-image-build from https://github.com/rancher-sandbox/os2-opensuse-image, then installed a VM with it. Installation went fine but there was no automatic reboot at the end, and this should be the default behaviour.

    bug area/os2 
    opened by ldevulder 4
  • Is there any Roadmap for RancherOS v2?

    Is there any Roadmap for RancherOS v2?

    I'm very interested in RancherOS v2. I have also seen that the alpha release phase has already started :partying_face: . Is there any roadmap / timeframe for this release? When will any beta / rc / stable version be available? When could/should RancherOS v2 be used instead of RancherOS v1? Will there be any upgrade path from RancherOS v1 to RancherOS v2?

    Thank you

    ping: rancher/os#3063

    opened by MAFLO321 3
  • creating users with cloud init not functional

    creating users with cloud init not functional


    I'm unable to log in with the user created, I've tried both in isolation as well with no avail. I'm using the ros-image-build with an iso output and running on a physical machine. Seems potentially related to https://github.com/rancher/os2/issues/11 https://github.com/rancher/os2/issues/10

    Given this cloud-config:

    hostname: "ros-001"
    - name: "kampe"
      passwd: "password"
      groups: "users"
      - github:kampe
      - github:kampe
    - whoami
      kubernetesVersion: v1.22.2+k3s1
      rancherVersion: v2.6.0
        features: multi-cluster-management=false
        hostPort: 8443
          enabled: false
        noDefaultAdmin: true
        replicas: -3
        tls: external
      token: sometoken
          provider: "mdns"
          service: "rancher-server"
        expectedServers: 3
        serverCacheDuration: 1m
      role: cluster-init
      nodeName: ros-001
    opened by Kampe 2
  • Booting from openstack doesn't load userdata

    Booting from openstack doesn't load userdata

    I've tried to apply the following cloud-init in the custom user-data when creating a VM in openstack from the os2 qcow alpha12 release:

      role: cluster-init
        - multi-cluster-management=true
      - github:mudler
      - curl -fL https://raw.githubusercontent.com/rancher/rancherd/master/install.sh | sh -

    This seems to work locally, but when the machine starts from openstack I cannot login, seems ignoring my keys

    opened by mudler 2
  • TPM device is a hard requirement

    TPM device is a hard requirement

    Can't join nodes that don't have a TPM device, getting the following error when calling ros-installer -config-file /oem/userdata.yaml:

    ERRO[0000] failed to read registration URL https://xx.lan/v1-rancheros/registration/xxxx, retrying: opening tpm: TPM device not available

    This is an issue at least for running vagrant images as VirtualBox currently doesn't support TPM, but applies to other hypervisors including raspberrypi4 and baremetal which don't have the TPM hardware in general

    Note, this doesn't seem to be an issue for libvirt/qemu: https://documentation.suse.com/sles/15-SP3/html/SLES-all/tpm.html https://github.com/stefanberger/swtpm/issues/33

    See also: https://github.com/stefanberger/swtpm/issues/33

    opened by mudler 2
  • QR Code for bootstrapping

    QR Code for bootstrapping

    It would be really nice to be able to be able to scan a QR code that provides Rancher with all the onboarding data from a newly bootstrapped machine.

    The workflow would look like:

    • Boot installer on target machine
    • After laying down partitions and rebooting, display QR code with any data needed
    • User then logs into Rancher on their phone or tablet
    • User browses to bootstrapping tab and clicks on "add machine with QR code"
    • Rancher opens camera
    • User scans code
    • Rancher stores data and now can control device securely
    opened by agracey 2
  • Feature Request: Allow Operator to manage being a downstream

    Feature Request: Allow Operator to manage being a downstream "cattle"


    Problem: There's seemingly no great way to manage the registration of a "edge" rancher cluster with a cloud hosted "multicluster-managment" rancher - or if there is I'm missing something fundamental about how I should be managing these "cattle" clusters I create downstream of the Rancher.

    Current Solution: We manage this ourselves by creating a k8s job that runs at runtime and utilizes a "service account" with credentials retrieved from centralized secret store to create and register the cluster with rancher "multicluster-management" which then self applies the cattle-agent yaml.

    is there a more preferred way to manage these types of registrations with the rancher?

    opened by Kampe 1
  • Automatic installation not picked up in vsphere

    Automatic installation not picked up in vsphere

    I don't have logs to attach as I didn't had access to the machine, but seems that userdata from ISO didn't work correctly under vsphere. The automatic installation and ros-installer didn't catch the userdata and thus automatic installation was not kicking while joining nodes.

    The /oem/userdata files were populated correctly, workaround was to point the ros-installer to the userdata file directly with -config-file

    This card is mostly to try to reproduce and track the issue

    opened by mudler 1
  • Support container images as installation bootstrap

    Support container images as installation bootstrap

    cOS-toolkit gained the capability to bootstrap directly container images besides isos, so it should be possible to enhance the current mechanism that applies to ISOs to container images as well.

    See for a reference isoUrl

    opened by mudler 1
  • Broken cloud-init after install

    Broken cloud-init after install

    I'm trying to install os2 to bring a standalone K8s cluster to register other nodes to it, by using the following cloud-config file:

      role: cluster-init
        - multi-cluster-management=true
      - github:mudler
      - curl -fL https://raw.githubusercontent.com/rancher/rancherd/master/install.sh | sh -

    And by running after booting the ISO

    $> ros-installer -config-file cloud-init.yaml

    However, after installation, the generated cloud-init config in /oem/99_custom.yaml is the following: VirtualBox_test_13_01_2022_11_15_46

    which breaks the cloud-init parsing reference, hence can't login after boot, see https://rancher.github.io/os2/configuration/

    opened by mudler 1
  • Slack channel?

    Slack channel?

    Hi, interesting project and I've started a matchbox/terraform test of OS2. @ibuildthecloud ,will you consider adding a slack channel? (Sorry for a question issue)


    opened by hwaastad 1
Truly Minimal Linux Distribution for Containers

Statesman Statesman is a minimal Linux distribution, running from memory, that has just enough functionality to run OCI-compatible containers. Rationa

James Cunningham 3 Nov 12, 2021
A Kubernetes operator that allows for automatic provisioning and distribution of cert-manager certs across namespaces

cached-certificate-operator CachedCertificate Workflow When a CachedCertificate is created or updated the operator does the following: Check for a val

Weave Development Lab 7 Sep 6, 2022
Assigns floating ip addresses to Rancher Guest clusters.

kube-fip-operator The kube-fip-operator application manages the FloatingIP and FloatingIPRange Custom Resource Definition objects in a Rancher environ

null 0 Dec 6, 2021
repo de teste para executar á pipeline do rancher

pipeline-example-go This is a sample golang project to demonstrate the integration with rancher pipeline. Building go build -o ./bin/hello-server Runn

Kaique Yamamoto | RedBlock Solution 0 Dec 19, 2021
Fast docker image distribution plugin for containerd, based on CRFS/stargz

[ ⬇️ Download] [ ?? Browse images] [ ☸ Quick Start (Kubernetes)] [ ?? Quick Start (nerdctl)] Stargz Snapshotter Read also introductory blog: Startup C

containerd 763 Dec 29, 2022
immutable, fluent, builders for Kubernetes resources

Dies - immutable, fluent, builders for Kubernetes resources Using dies Common methods Creating dies diegen die markers +die This project contains dies

Scott Andrews 3 May 6, 2022
Open Source runtime scanner for Linux containers (LXD), It performs security audit checks based on CIS Linux containers Benchmark specification

lxd-probe Scan your Linux container runtime !! Lxd-Probe is an open source audit scanner who perform audit check on a linux container manager and outp

Chen Keinan 16 Dec 26, 2022
a small form factor OpenShift/Kubernetes optimized for edge computing

Microshift Microshift is OpenShift1 Kubernetes in a small form factor and optimized for edge computing. Edge devices deployed out in the field pose ve

Red Hat Emerging Technologies 450 Dec 29, 2022
resource manifest distribution among multiple clusters.

Providing content to managed clusters Support a primitive that enables resources to be applied to a managed cluster. Community, discussion, contributi

Open Cluster Management 26 Dec 26, 2022
Walker's alias method is an efficient algorithm to sample from a discrete probability distribution.

walker-alias Walker's alias method is an efficient algorithm to sample from a discrete probability distribution. This means given an arbitrary probabi

null 3 Jun 14, 2022
Apachedist-resource - A concourse resource to track updates of an apache distribution, e.g. tomcat

Apache Distribution Resource A concourse resource that can track information abo

Gareth Evans 0 Feb 2, 2022
Tape backup software optimized for large WORM data and long-term recoverability

Mixtape Backup software for tape users with lots of WORM data. Draft design License This codebase is not open-source software (or free, or "libre") at

Dave Anderson 16 Oct 30, 2022
Go library providing algorithms optimized to leverage the characteristics of modern CPUs

asm Go library providing algorithms optimized to leverage the characteristics of modern CPUs. Motivation With the development of Cloud technologies, a

Segment 778 Dec 29, 2022
Kubernetes OS Server - Kubernetes Extension API server exposing OS configuration like sysctl via Kubernetes API

KOSS is a Extension API Server which exposes OS properties and functionality using Kubernetes API, so it can be accessed using e.g. kubectl. At the moment this is highly experimental and only managing sysctl is supported. To make things actually usable, you must run KOSS binary as root on the machine you will be managing.

Mateusz Gozdek 3 May 19, 2021
Linux Controllers for Kubernetes

Tambourine Kubelet replacement with Built in Linux extensions Development Success: Install, Manage, and Observe a new systemd service from Kubernetes.

Kris Nóva 167 Jun 2, 2021
KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes

Kubernetes-based Event Driven Autoscaling KEDA allows for fine-grained autoscaling (including to/from zero) for event driven Kubernetes workloads. KED

KEDA 5.9k Jan 7, 2023
An Easy to use Go framework for Kubernetes based on kubernetes/client-go

k8devel An Easy to use Go framework for Kubernetes based on kubernetes/client-go, see examples dir for a quick start. How to test it ? Download the mo

null 10 Mar 25, 2022
Litmus helps Kubernetes SREs and developers practice chaos engineering in a Kubernetes native way.

Litmus Cloud-Native Chaos Engineering Read this in other languages. ???? ???? ???? ???? Overview Litmus is a toolset to do cloud-native chaos engineer

Litmus Chaos 3.4k Jan 1, 2023
vcluster - Create fully functional virtual Kubernetes clusters - Each cluster runs inside a Kubernetes namespace and can be started within seconds

Website • Quickstart • Documentation • Blog • Twitter • Slack vcluster - Virtual Clusters For Kubernetes Lightweight & Low-Overhead - Based on k3s, bu

Loft Labs 2.3k Jan 4, 2023