A Kubernetes CSI plugin to automatically mount SPIFFE certificates to Pods using ephemeral volumes

Overview

Go Report Card

csi-driver-spiffe

csi-driver-spiffe is a Container Storage Interface (CSI) driver plugin for Kubernetes to work along cert-manager. This CSI driver transparently delivers SPIFFE SVIDs in the form of X.509 certificate key pairs to mounting Kubernetes Pods.

The end result is all and any Pod running in Kubernetes can securely request their SPIFFE identity document from a Trust Domain with minimal configuration. These documents are:

  • automatically renewed; ✔️
  • private key never leaves the node's virtual memory; ✔️
  • each Pod's document is unique; ✔️
  • the document shares the same life cycle as the Pod and is destroyed on Pod termination. ✔️
...
          volumeMounts:
          - mountPath: "/var/run/secrets/spiffe.io"
            name: spiffe
      volumes:
        - name: spiffe
          csi:
            driver: spiffe.csi.cert-manager.io
            readOnly: true

SPIFFE documents can be used for mutual TLS (mTLS) or authentication by Pod's within its Trust Domain.

Components

The project is split into two components;

CSI Driver

The CSI driver runs as DaemonSet on the cluster which is responsible for generating, requesting, and mounting the certificate key pair to Pods on the node it manages. The CSI driver creates and manages a tmpfs directory which is used to create and mount Pod volumes from.

When a Pod is created with the CSI volume configured, the driver will locally generate a private key, and create a cert-manager CertificateRequest in the same Namespace as the Pod.

The driver uses CSI Token Request to both discover the Pod's identity to form the SPIFFE identity contained in the X.509 certificate signing request, as well as securely impersonate its ServiceAccount when creating the CertificateRequest.

Once signed by the pre-configured target signer, the driver will mount the private key and signed certificate into the Pod's Volume to be made available as a Volume Mount. This certificate key pair is regularly renewed based on the expiry of the signed certificate.

Approver

A distinct cert-manager approver Deployment is responsible for managing the approval and denial condition of created CertificateRequests that target the configured SPIFFE Trust Domain signer. The approver ensures that requests have:

  1. the correct key type (ECDSA P-521);
  2. acceptable key usages (Key Encipherment, Digital Signature, Client Auth, Server Auth);
  3. the requested duration matches the enforced duration (default 1 hour);
  4. no SANs or other identifiable attributes except a single URI SANs;
  5. the single URI SAN is the SPIFFE identity of the ServiceAccount who created the CertificateRequest;
  6. the SPIFFE ID Trust Domain is the same as configured.

If any of these checks do not pass, the CertificateRequest will be marked as Denied, else it will be marked as Approved. The approver will only manage CertificateRequests who request from the same IssuerRef that has been configured.

Installation

⚠️ Requires Kubernetes version v1.21+ or v1.20 with the --feature-gates=CSIServiceAccountToken=true flag.

⚠️ Requires cert-manager v1.3 or higher.

  1. cert-manager is required to be installed with csi-driver-spiffe.

⚠️

It is important that the default approver is disabled in cert-manager. If the default approver is not disabled in cert-manager, the csi-driver-spiffe approver will race with cert-manager and thus its policy enforcement becomes useless.

$ helm repo add jetstack https://charts.jetstack.io --force-update
$ helm upgrade -i -n cert-manager cert-manager jetstack/cert-manager --set extraArgs={--controllers='*\,-certificaterequests-approver'} --set installCRDs=true --create-namespace

⚠️

  1. Install or configure a ClusterIssuer to give cert-manager the ability to sign against your Trust Domain. If a namespace scoped Issuer is desired, then that Issuer must be created in every namespace that Pods will mount volumes from. You must use an Issuer type which is compatible with signing URI SAN certificates and the private does not need to be available to the signer, for example CA, Vault, Venafi, AWS PCA, Google CAS, Small Step. Issuers such as SelfSigned or ACME will not work.

    An example demo ClusterIssuer can be found here. This Trust Domain's root CA is self-signed by cert-manager and private key is stored in the cluster.

$ kubectl apply -f ./deploy/example/clusterissuer.yaml
# We must also approve the CertificateRequest since we have disabled the default approver
$ kubectl cert-manager approve -n cert-manager $(kubectl get cr -n cert-manager -ojsonpath='{.items[0].metadata.name}')
  1. Install csi-driver-spiffe into the cluster using the issuer we configured. We must also configure the issuer resource type and name of the issuer we configured so that the approver has permissions to approve referencing CertificateRequests.
  • Change signer name to match your issuer type.
  • Change name, kind, and group to your issuer.
$ helm upgrade -i -n cert-manager cert-manager-csi-driver-spiffe jetstack/cert-manager-csi-driver-spiffe --wait \
  --set "app.logLevel=1" \
  --set "app.trustDomain=my.trust.domain" \
  --set "app.approver.signerName=clusterissuers.cert-manager.io/csi-driver-spiffe-ca" \
  \
  --set "app.issuer.name=csi-driver-spiffe-ca" \
  --set "app.issuer.kind=ClusterIssuer" \
  --set "app.issuer.group=cert-manager.io"

Usage

Once the driver is successfully installed, Pods can begin to request and mount their key and SPIFFE certificate. Since the Pod's ServiceAccount is impersonated when creating CertificateRequests, every ServiceAccount must be given that permission which intends to use the volume.

Example manifest with a dummy Deployment:

$ kubectl apply -f ./deploy/example/example-app.yaml

$ kubectl exec -n sandbox $(kubectl get pod -n sandbox -l app=my-csi-app -o jsonpath='{.items[0].metadata.name}') -- cat /var/run/secrets/spiffe.io/tls.crt | openssl x509 --noout --text | grep Issuer:
        Issuer: CN = csi-driver-spiffe-ca
$ kubectl exec -n sandbox $(kubectl get pod -n sandbox -l app=my-csi-app -o jsonpath='{.items[0].metadata.name}') -- cat /var/run/secrets/spiffe.io/tls.crt | openssl x509 --noout --text | grep URI:
                URI:spiffe://foo.bar/ns/sandbox/sa/example-app

FS-Group

When running Pods with a specified user or group, the volume will not be readable by default due to Unix based file system permissions. The mounting volumes file group can be specified using the following volume attribute:

...
      securityContext:
        runAsUser: 123
        runAsGroup: 456
      volumes:
        - name: spiffe
          csi:
            driver: spiffe.csi.cert-manager.io
            readOnly: true
            volumeAttributes:
              spiffe.csi.cert-manager.io/fs-group: "456"
$ kubectl apply -f ./deploy/example/fs-group-app.yaml

$ kubectl exec -n sandbox $(kubectl get pod -n sandbox -l app=my-csi-app-fs-group -o jsonpath='{.items[0].metadata.name}') -- cat /var/run/secrets/spiffe.io/tls.crt | openssl x509 --noout --text | grep URI:
                URI:spiffe://foo.bar/ns/sandbox/sa/fs-group-app

Root CA Bundle

By default, the CSI driver will only mount the Pod's private key and signed certificate. csi-driver-spiffe can be optionally configured to also mount a statically defined CA bundle from a volume that will be written to all Pod volumes.

If the CSI driver detects this bundle has changed (through overwrite, renewal, etc), the new bundle will be written to all existing volumes.

The following example mounts the CA certificate used by the Trust Domain ClusterIssuer.

$ helm upgrade -i -n cert-manager cert-manager-csi-driver-spiffe jetstack/cert-manager-csi-driver-spiffe --wait \
  --set "app.logLevel=1" \
  --set "app.trustDomain=my.trust.domain" \
  --set "app.approver.signerName=clusterissuers.cert-manager.io/csi-driver-spiffe-ca" \
  \
  --set "app.issuer.name=csi-driver-spiffe-ca" \
  --set "app.issuer.kind=ClusterIssuer" \
  --set "app.issuer.group=cert-manager.io" \
  \
  --set "app.driver.volumes[0].name=root-cas" \
  --set "app.driver.volumes[0].secret.secretName=csi-driver-spiffe-ca" \
  --set "app.driver.volumeMounts[0].name=root-cas" \
  --set "app.driver.volumeMounts[0].mountPath=/var/run/secrets/cert-manager-csi-driver-spiffe" \
  --set "app.driver.sourceCABundle=/var/run/secrets/cert-manager-csi-driver-spiffe/ca.crt"
$ kubectl rollout restart deployment -n sandbox my-csi-app
$ kubectl exec -it -n sandbox $(kubectl get pod -n sandbox -l app=my-csi-app -o jsonpath='{.items[0].metadata.name}') -- ls /var/run/secrets/spiffe.io/
ca.crt   tls.crt  tls.key
Comments
  • Write go-spiffe compatible keys

    Write go-spiffe compatible keys

    We have been using csi-driver-spiffe with a workload based on go-spiffe and found that the EC private key created by the driver is not compatible. go-spiffe uses https://cs.opensource.google/go/go/+/go1.18.1:src/crypto/x509/pkcs8.go;l=33

    We are calling Load: https://github.com/spiffe/go-spiffe/blob/31de176038793c17cf7e77f23e61401160c7d6c9/v2/svid/x509svid/svid.go#L33

    This eventually calls: https://github.com/spiffe/go-spiffe/blob/31de176038793c17cf7e77f23e61401160c7d6c9/v2/internal/pemutil/pem.go#L110-L114

    This PR when complete will have the csi driver write a compatible key format instead so that keys can be used by go-spiffe based workloads.

    Signed-off-by: Charlie Egan [email protected]

    dco-signoff: yes size/L needs-rebase 
    opened by charlieegan3 7
  • Support for including imagePullSecrets in Helm

    Support for including imagePullSecrets in Helm

    Signed-off-by: Sitaram IYER [email protected]

    Add support for including imagePullSecret in Helm for csi-driver-spiffe to pull images from private repo. Fixes #17

    dco-signoff: yes lgtm approved size/S ok-to-test 
    opened by sitaramkm 6
  • Remove documentation in favour of cert-manager.io

    Remove documentation in favour of cert-manager.io

    This PR removes the docs in the README.md, in favor of them being on the cert-manager.io website.

    Paired with https://github.com/cert-manager/website/pull/1040

    dco-signoff: yes lgtm approved size/L 
    opened by JoshVanL 2
  • Make CSI driver spiffe writeFiles compatable with go-spiffe `x509svid.Parse`

    Make CSI driver spiffe writeFiles compatable with go-spiffe `x509svid.Parse`

    This PR ensures that the driver writes certificate key pairs which are compatible with the go-spiffe x509svid.Parse library function.

    To do this, the private key marshalling has been changed from MarshalECPrivateKey -> MarshalPKCS8PrivateKey and PEM header EC PRIVATE KEY -> PRIVATE KEY.

    In order to test this, the camanager driver component has been split out into its own struct. This was required since the camanager absolutely needs a filesystem storage (in order to read files back), however we need to use the store Interface for the writeKeypair func so InMemory can be used for tests. filesystem is incompatible with unit tests since it requires mounts.

    /cc @jakexks @charlieegan3

    dco-signoff: yes lgtm approved size/L 
    opened by JoshVanL 2
  • Adds verify-helm-docs.sh and update-helm-docs.sh scripts. Verify helm-docs during lint

    Adds verify-helm-docs.sh and update-helm-docs.sh scripts. Verify helm-docs during lint

    /assign @munnerz

    This PR adds a scripts, run at make lint, which verifies that the helm chart README.md file is kept up to date.

    An update script is also added which will update the helm chart README.md.

    dco-signoff: yes lgtm approved size/L 
    opened by JoshVanL 1
  • Add support for certificate expiry configuration

    Add support for certificate expiry configuration

    Currently the namespace csi.cert-manager.io supports many different volume configuration attributes. This set of configuration can enable for example certificate renew period using the attributes csi.cert-manager.io/duration and csi.cert-manager.io/renew-before. However I've noted that it is not implemented on spiffe.csi.cert-manager.io namespace. Instead the expiry period seem to be fixed in 1 hour like image below.

    image

    What would take to implement at least the spiffe.csi.cert-manager.io/duration and spiffe.csi.cert-manager.io/renew-before attributes to enable certificate renew for csi-spiffe-driver?

    opened by trvll 0
Releases(v0.2.0)
  • v0.2.0(May 4, 2022)

    What's Changed

    • Bump csi lib v0.2.2 by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/4
    • Update links to cert-manager/cert-manager in SECURITY.md and README.md by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/5
    • Updates to go v1.18 in Dockerfiles by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/6
    • Bumps bump-csi-node-driver-registrar to v2.5.0 by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/7
    • Adds liveness probe to csi-driver by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/8
    • Make CSI driver spiffe writeFiles compatable with go-spiffe x509svid.Parse by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/13
    • Configurable side cars by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/9
    • Adds verify-helm-docs.sh and update-helm-docs.sh scripts. Verify helm-docs during lint by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/11
    • Updates app version to v0.2.0 by @JoshVanL in https://github.com/cert-manager/csi-driver-spiffe/pull/10

    Full Changelog: https://github.com/cert-manager/csi-driver-spiffe/compare/v0.1.0...v0.2.0

    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Nov 16, 2021)

    Initial Release

    csi-driver-spiffe is a Container Storage Interface (CSI) driver plugin for Kubernetes to work along cert-manager. This CSI driver transparently delivers SPIFFE SVIDs in the form of X.509 certificate key pairs to mounting Kubernetes Pods.

    The end result is all and any Pod running in Kubernetes can securely request their SPIFFE identity document from a Trust Domain with minimal configuration. These documents are:

    automatically renewed; :heavy_check_mark: private key never leaves the node's virtual memory; :heavy_check_mark: each Pod's document is unique; :heavy_check_mark: the document shares the same life cycle as the Pod and is destroyed on Pod termination. :heavy_check_mark:

    ...
              volumeMounts:
              - mountPath: "/var/run/secrets/spiffe.io"
                name: spiffe
          volumes:
            - name: spiffe
              csi:
                driver: spiffe.csi.cert-manager.io
                readOnly: true
    

    SPIFFE documents can be used for mutual TLS (mTLS) or authentication by Pod's within its Trust Domain.

    Source code(tar.gz)
    Source code(zip)
Owner
null
Test-csi-driver - Amazon Elastic Block Store (EBS) CSI driver

Amazon Elastic Block Store (EBS) CSI driver Overview The Amazon Elastic Block St

Adi Vaknin 0 Feb 1, 2022
Kubectl plugin to ease sniffing on kubernetes pods using tcpdump and wireshark

ksniff A kubectl plugin that utilize tcpdump and Wireshark to start a remote capture on any pod in your Kubernetes cluster. You get the full power of

Eldad Rudich 2.5k Jan 4, 2023
Kubectl Locality Plugin - A plugin to get the locality of pods

Kubectl Locality Plugin - A plugin to get the locality of pods

John Howard 6 Nov 18, 2021
Dothill (Seagate) AssuredSAN dynamic provisioner for Kubernetes (CSI plugin).

Dothill-csi dynamic provisioner for Kubernetes A dynamic persistent volume (PV) provisioner for Dothill AssuredSAN based storage systems. Introduction

Enix 23 Oct 11, 2022
Kubectl plugin to run curl commands against kubernetes pods

kubectl-curl Kubectl plugin to run curl commands against kubernetes pods Motivation Sending http requests to kubernetes pods is unnecessarily complica

Segment 160 Dec 22, 2022
A plugin for Hashicorp Vault to create ephemeral users and API tokens for Jenkins CI

vault-plugin-secrets-jenkins This is a backend plugin to be used with Hashicorp Vault. This plugin generates ephemeral Jenkins Users and API tokens. v

Caleb Lemoine 6 Dec 15, 2022
Asynchronous data replication for Kubernetes volumes

VolSync VolSync asynchronously replicates Kubernetes persistent volumes between clusters using either rsync or rclone. It also supports creating backu

Backube 112 Jan 1, 2023
Container Storage Interface components for SPIFFE

SPIFFE CSI Driver WARNING: This project is in the "Development" phase of the SPIFFE Project Maturity Phases. A Container Storage Interface driver for

SPIFFE 29 Jan 3, 2023
A kubectl plugin to evict pods

kubectl-evict A kubectl plugin to evict pods. This plugin is good to remove a pod from your cluster or to test your PodDistruptionBudget. ?? Installat

Shin'ya Ueoka 15 Dec 7, 2022
K8s-cinder-csi-plugin - K8s Pod Use Openstack Cinder Volume

k8s-cinder-csi-plugin K8s Pod Use Openstack Cinder Volume openstack volume list

douyali 0 Jul 18, 2022
KinK is a helper CLI that facilitates to manage KinD clusters as Kubernetes pods. Designed to ease clusters up for fast testing with batteries included in mind.

kink A helper CLI that facilitates to manage KinD clusters as Kubernetes pods. Table of Contents kink (KinD in Kubernetes) Introduction How it works ?

Trendyol Open Source 362 Dec 10, 2022
gpu-memory-monitor is a metrics server for collecting GPU memory usage of kubernetes pods.

gpu-memory-monitor is a metrics server for collecting GPU memory usage of kubernetes pods. If you have a GPU machine, and some pods are using the GPU device, you can run the container by docker or kubernetes when your GPU device belongs to nvidia. The gpu-memory-monitor will collect the GPU memory usage of pods, you can get those metrics by API of gpu-memory-monitor

null 2 Jul 27, 2022
Viewnode displays Kubernetes cluster nodes with their pods and containers.

viewnode The viewnode shows Kubernetes cluster nodes with their pods and containers. It is very useful when you need to monitor multiple resources suc

NTTDATA-DACH 9 Nov 23, 2022
Kubernetes CSI driver for QNAP NAS's

QNAP CSI This is a very alpha QNAP Kubernetes CSI driver which lets you automatically provision iSCSI volumes on a QNAP NAS. Its only been tested on a

Terry Cain 3 Jul 29, 2022
An operator for managing ephemeral clusters in GKE

Test Cluster Operator for GKE This operator provides an API-driven cluster provisioning for integration and performance testing of software that integ

Isovalent 32 Oct 22, 2022
Mount encrypted drives for use with snapraid/mergerfs after boot.

Raid Mount This tool was designed to make it easy to mount encrypted hard drives for a snapraid/mergerfs configuration after a boot of a system. This

Mr. Gecko 0 Nov 28, 2021
Discover expired TLS certificates in the services of a kubernetes cluster

About verify-k8s-certs is a daemon (prometheus exporter) to discover expired TLS certificates in a kubernetes cluster. It exposes the informations as

Angelo Poerio 7 Feb 1, 2022
Manage Lets Encrypt certificates for a Kubernetes cluster.

Kubernetes Certificate Manager This project is loosely based on https://github.com/kelseyhightower/kube-cert-manager It took over most of its document

null 1 Mar 11, 2022
Deletes completed pods that are owned by ArgoWorkflow.

argoworkflow-pod-reaper Deletes completed pods that are owned by ArgoWorkflow. Usage: go test ./... -cover ok github.com/smallcase/workfl

smallcase 5 Sep 2, 2021