Kubernetes controller for backing up public container images to our own registry repository

Overview

image-clone-controller

Kubernetes controller which watches applications (Deployment and DaemonSet) and "caches" the images (public container images) by re-uploading to our own registry repository and reconfiguring the applications to use these copies.

Project's Motivation

  • We’d like to be safe against the risk of public container images disappearing from the registry while we use them, breaking our deployments.
  • Suppose, we have a Kubernetes cluster on which we can run applications. These applications will often use publicly available container images, like official images of popular programs, e.g. Jenkins, PostgreSQL, and so on. Since the images reside in repositories over which we have no control, it is possible that the owner of the repo deletes the image while our pods are configured to use it. In the case of a subsequent node rotation, the locally cached copies of the images would be deleted and Kubernetes would be unable to re-download them in order to re-provision the applications.
  • So, we want to have a controller which watches the applications and “caches” the images by re-uploading to our own registry repository and reconfiguring the applications to use these copies.

Demo

Use

Locally running the manager

  • clone this repo
  • open the repo locally
  • run make
  • run ./bin/manager
  • open another terminal and go to samples: cd config/samples
  • apply docker cred secret & sample deployment:
    • give in the auth: field of the docker-cred-k8s-secret
    • run kubectl apply -f docker-cred-secret.yaml
    • run kubectl apply -f sample-deployment.yaml
  • check in the sample deployment image, it will get cloned & pushed to your given docker registry and re-use in the deployment

InCluster manager running

  • export IMG="<your_registry>/<controller_image_name>:<tag>"
  • make docker-build
  • make docker-push (Note: for docker push you need to login in your dockerhub from the current terminal by docker login)
  • make deploy
  • verify the deployment by: kubectl get all -n image-clone-controller-system
  • open another terminal and go to samples: cd config/samples
  • apply docker cred secret & sample deployment:
    • give in the auth: field of the docker-cred-k8s-secret
    • run kubectl apply -f docker-cred-secret.yaml
    • run kubectl apply -f sample-deployment.yaml
  • check in the sample deployment image, it will get cloned & pushed to your given docker registry and re-use in the deployment
  • undeploy by: make undeploy

e2e test

  • Added e2e test for deployment controller, similarly will add for DaemonSet controller
  • For using Deployment controller test follow below steps:
    • run the controller (either locally or incluster running the manager)
    • in another terminal go to project's : cd tests/e2e
    • in the tests/e2e/framework/docker-cred-secret.go file provide your dockerhub "username:password" in the "auth" field
    • run ginkgo run --which-controller=<controller_name> --registry=<your_dockerhub_username>
    • ex: ginkgo run -- --which-controller=deployment --registry=shahincsejnu
    • Note: make sure you sync the namespace, registry name among test files & controllers

Disclaimer

  • It's a hobby project, not a production grade

What's Next?

  • make this controller code more generic
  • make helm chart of this operator

Resources:

Owner
Sahadat Hossain
Software Engineer at @appscode | Open-Source Enthusiast | Golang, Docker, Kubernetes, Kubeform, Terraform
Sahadat Hossain
Image clone controller is a kubernetes controller to safe guard against the risk of container images disappearing

Image clone controller image clone controller is a kubernetes controller to safe guard against the risk of container images disappearing from public r

Jayadeep KM 0 Oct 10, 2021
k8s-image-swapper Mirror images into your own registry and swap image references automatically.

k8s-image-swapper Mirror images into your own registry and swap image references automatically. k8s-image-swapper is a mutating webhook for Kubernetes

Enrico Stahn 300 May 1, 2022
registry-tools: Prints image digest from a registry

registry-tools: Prints image digest from a registry

Rashed K 1 Dec 23, 2021
A Controller written in kubernetes sample-controller style which watches a custom resource named Bookstore

bookstore-sample-controller A Controller written in kubernetes sample-controller style which watches a custom resource named Bookstore. A resource cre

Abdullah Al Shaad 0 Jan 20, 2022
Kubernetes Container Registry

k8scr A kubectl plugin for pushing OCI images through the Kubernetes API server. Quickstart Build kubectl-k8scr make build Move to location in PATH s

Daniel Mangum 110 Mar 25, 2022
Boxygen is a container as code framework that allows you to build container images from code

Boxygen is a container as code framework that allows you to build container images from code, allowing integration of container image builds into other tooling such as servers or CLI tooling.

nitric 5 Dec 13, 2021
A kubernetes controller that watches the Deployments and “caches” the images

image-cloner This is just an exercise. It's a kubernetes controller that watches

Luca Sepe 1 Dec 20, 2021
Kubernetes workload controller for container image deployment

kube-image-deployer kube-image-deployer는 Docker Registry의 Image:Tag를 감시하는 Kubernetes Controller입니다. Keel과 유사하지만 단일 태그만 감시하며 더 간결하게 동작합니다. Container, I

PUBG Corporation 2 Mar 8, 2022
A tool to check whether docker images exist in the remote registry.

Check Docker Image A tool to check whether docker images exist in the remote registry. Build project: go build -o check-image . Example usage: REGISTR

Hao-Ming, Hsu 2 May 19, 2022
Container Registry Synchronization made easy and fast

?? booster - Makes synchronization of container images between registries faster.

Silvio Moioli 11 May 12, 2022
Returns which registry from the container image name

Returns which registry from the container image name

Nozomu Ohki 0 Jan 23, 2022
network-node-manager is a kubernetes controller that controls the network configuration of a node to resolve network issues of kubernetes.

Network Node Manager network-node-manager is a kubernetes controller that controls the network configuration of a node to resolve network issues of ku

kakao 95 May 9, 2022
Carrier is a Kubernetes controller for running and scaling game servers on Kubernetes.

Carrier is a Kubernetes controller for running and scaling game servers on Kubernetes. This project is inspired by agones. Introduction Genera

Open Cloud-native Game-application Initiative 27 May 1, 2022
Kubei is a flexible Kubernetes runtime scanner, scanning images of worker and Kubernetes nodes providing accurate vulnerabilities assessment, for more information checkout:

Kubei is a vulnerabilities scanning and CIS Docker benchmark tool that allows users to get an accurate and immediate risk assessment of their kubernet

Portshift 585 May 17, 2022
An image server which automatically optimize non webp and avif images to webp and avif images

go-imageserver go-imageserver is an image server which automatically optimize no

DeltaLaboratory 4 Apr 18, 2022
At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.

School of SRE In early 2019, we started visiting campuses across India to recruit the best and brightest minds to ensure LinkedIn, and all the service

LinkedIn 5.6k May 14, 2022
Our library to interact with shopware6.

gosw6 Here you can find our library for shopware 6. We develop the API endpoints according to our demand and need. You are welcome to help us to furth

J&J Ideenschmiede GmbH 0 Jan 10, 2022
A simple Kubernetes Operator template that uses Golang, use it to build your own operators

A simple programmatic Kubernetes Operator template. Use this to create your own Kubernetes operators with golang. Build with KIND (Kubernetes in Docke

Cloud Native Skunkworks 7 May 13, 2022
KEDA is a Kubernetes-based Event Driven Autoscaling component. It provides event driven scale for any container running in Kubernetes

Kubernetes-based Event Driven Autoscaling KEDA allows for fine-grained autoscaling (including to/from zero) for event driven Kubernetes workloads. KED

KEDA 5k May 21, 2022