Kubernetes IN Docker - local clusters for testing Kubernetes

Related tags

DevOps Tools docker kubernetes golang kubeadm k8s-sig-testing podman
Overview

kind

Please see Our Documentation for more in-depth installation etc.

kind is a tool for running local Kubernetes clusters using Docker container "nodes". kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.

If you have go (1.11+) and docker installed GO111MODULE="on" go get sigs.k8s.io/[email protected] && kind create cluster is all you need!

kind consists of:

kind bootstraps each "node" with kubeadm. For more details see the design documentation.

NOTE: kind is still a work in progress, see the 1.0 roadmap.

Installation and usage

For a complete install guide see the documentation here.

You can install kind with GO111MODULE="on" go get sigs.k8s.io/[email protected].

NOTE: please use the latest go to do this, ideally go 1.13 or greater.

NOTE: go get should not be run from a Go [modules] enabled project directory, as go get inside a modules enabled project updates dependencies / behaves differently. Try for example cd $HOME first.

This will put kind in $(go env GOPATH)/bin. If you encounter the error kind: command not found after installation then you may need to either add that directory to your $PATH as shown here or do a manual installation by cloning the repo and run make build from the repository.

Without installing go, kind can be built reproducibly with docker using make build.

Stable binaries are also available on the releases page. Stable releases are generally recommended for CI usage in particular. To install, download the binary for your platform from "Assets" and place this into your $PATH:

On Linux:

curl -Lo ./kind "https://kind.sigs.k8s.io/dl/v0.11.1/kind-$(uname)-amd64"
chmod +x ./kind
mv ./kind /some-dir-in-your-PATH/kind

On macOS via Homebrew:

brew install kind

On macOS via MacPorts:

sudo port selfupdate && sudo port install kind

On macOS via Bash:

curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.11.1/kind-darwin-amd64
chmod +x ./kind
mv ./kind /some-dir-in-your-PATH/kind

On Windows:

curl.exe -Lo kind-windows-amd64.exe https://kind.sigs.k8s.io/dl/v0.11.1/kind-windows-amd64
Move-Item .\kind-windows-amd64.exe c:\some-dir-in-your-PATH\kind.exe

# OR via Chocolatey (https://chocolatey.org/packages/kind)
choco install kind

To use kind, you will need to install docker. Once you have docker running you can create a cluster with:

kind create cluster

To delete your cluster use:

kind delete cluster

To create a cluster from Kubernetes source:

  • ensure that Kubernetes is cloned in $(go env GOPATH)/src/k8s.io/kubernetes
  • build a node image and create a cluster with:
kind build node-image
kind create cluster --image kindest/node:latest

Multi-node clusters and other advanced features may be configured with a config file, for more usage see the docs or run kind [command] --help

Community

Please reach out for bugs, feature requests, and other issues! The maintainers of this project are reachable via:

Current maintainers are @BenTheElder, @munnerz, @aojea, and @amwat - feel free to reach out if you have any questions!

Pull Requests are very welcome! If you're planning a new feature, please file an issue to discuss first.

Check the issue tracker for help wanted issues if you're unsure where to start, or feel free to reach out to discuss. 🙂

See also: our own contributor guide and the Kubernetes community page.

Why kind?

  • kind supports multi-node (including HA) clusters
  • kind supports building Kubernetes release builds from source
    • support for make / bash or docker, in addition to pre-published builds
  • kind supports Linux, macOS and Windows
  • kind is a CNCF certified conformant Kubernetes installer

Code of conduct

Participation in the Kubernetes community is governed by the Kubernetes Code of Conduct.

Comments
  • Cluster doesn't restart when docker restarts

    Cluster doesn't restart when docker restarts

    When docker restarts or stop/start (for any reason), the kind node containers remain stopped and aren't restarted properly. When I tried to run docker restart <node container id> the cluster didn't start either.

    The only solution seems to recreate the cluster at this point.

    /kind bug

    kind/feature priority/important-soon lifecycle/active kind/design 
    opened by vincepri 92
  • Support arm64

    Support arm64

    Device under test is a Packet c1.large.arm 96-core arm64 machine running Ubuntu 18.04.

    ed@ed-2a-bcc-llvm:~$ go version
go version go1.11.2 linux/arm64
ed@ed-2a-bcc-llvm:~$ go get sigs.k8s.io/kind
ed@ed-2a-bcc-llvm:~$ go/bin/kind create cluster
Creating cluster 'kind-1' ...
 ✓ Ensuring node image (kindest/node:v1.12.2)  
 ✓ [kind-1-control-plane] Creating node container 📦 
 ✗ [kind-1-control-plane] Fixing mounts 🗻 
Error: failed to create cluster: exit status 1
Usage:  
  kind create cluster [flags]
        
Flags:  
      --config string   path to a kind config file
  -h, --help            help for cluster
      --image string    node docker image to use for booting the cluster
      --name string     cluster context name (default "1")
      --retain          retain nodes for debugging when cluster creation fails
      --wait duration   Wait for control plane node to be ready (default 0s)

Global Flags:
      --loglevel string   logrus log level [panic, fatal, error, warning, info, debug] (default "warning")

failed to create cluster: exit status 1
ed@ed-2a-bcc-llvm:~$ docker version
Client:
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.4
 Git commit:        4d60db4
 Built:             Wed Nov  7 00:52:41 2018
 OS/Arch:           linux/arm64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.4
  Git commit:       4d60db4
  Built:            Wed Nov  7 00:17:01 2018
  OS/Arch:          linux/arm64
  Experimental:     false
    kind/feature lifecycle/frozen priority/backlog lifecycle/active 
    opened by vielmetti 82
  • Add dual stack support

    Add dual stack support

    Add dual stack support to KIND, it also needs to add dual-stack support to KINDNET Depends on:

    • [x] https://github.com/kubernetes/kubernetes/pull/79033
    • [x] https://github.com/kubernetes/kubernetes/pull/79386
    • [x] https://github.com/kubernetes/kubernetes/pull/82462
    • [x] https://github.com/kubernetes/kubernetes/pull/82473
    • [x] https://github.com/kubernetes/kubernetes/pull/79993
    • [x] https://github.com/kubernetes/kubernetes/pull/78801
    • [x] https://github.com/kubernetes/kubernetes/pull/83123

    New dependency for kindnet so we don't need annotations on the nodes:

    • [x] https://github.com/kubernetes/enhancements/pull/1665
    approved cncf-cla: yes lgtm priority/important-soon size/XL area/provider/docker area/provider/podman 
    opened by aojea 67
  • mount node product_uuid and product_name in pod containers

    mount node product_uuid and product_name in pod containers

    When pods are running at kind cluster their product_uuid and product_name is the same since the share the kernel vfs, this PR add a new mount to OCI spec to bind mount node's product_uuid and product_name into pod's containers. This is the result

    $ kubectl exec nginx-kind-worker cat /sys/class/dmi/id/product_uuid
053ba73c-3a24-4cfe-b7ca-5a938a4600d7
$ kubectl exec nginx-kind-worker2 cat /sys/class/dmi/id/product_uuid
db9f435b-0316-4f66-92a0-8d3632d6f69c

    Closes #https://github.com/kubernetes-sigs/kind/issues/2318

    cncf-cla: yes size/M do-not-merge/hold ok-to-test 
    opened by qinqon 64
  • create HA cluster is flaky

    create HA cluster is flaky

    What happened: i started seeing odd failures in the kind-master and -1.14 kubeadm jobs: https://k8s-testgrid.appspot.com/sig-cluster-lifecycle-kubeadm#kubeadm-kind-master https://k8s-testgrid.appspot.com/sig-cluster-lifecycle-kubeadm#kubeadm-kind-1.14

    after switching to this HA config:

    # a cluster with 3 control-planes and 3 workers
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
- role: control-plane
- role: control-plane
- role: control-plane
- role: worker
- role: worker
- role: worker

    I0604 19:15:09.075770     760 join.go:480] [preflight] Retrieving KubeConfig objects
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
I0604 19:15:10.310249     760 round_trippers.go:438] GET https://172.17.0.2:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config 401 Unauthorized in 1233 milliseconds
error execution phase preflight: unable to fetch the kubeadm-config ConfigMap: failed to get config map: Unauthorized 
 ✗ Joining more control-plane nodes 🎮
DEBU[22:15:10] Running: /usr/bin/docker [docker ps -q -a --no-trunc --filter label=io.k8s.sigs.kind.cluster --format {{.Names}}\t{{.Label "io.k8s.sigs.kind.cluster"}} --filter label=io.k8s.sigs.kind.cluster=kind] 
$KUBECONFIG is still set to use /home/lubo-it/.kube/kind-config-kind even though that file has been deleted, remember to unset it
DEBU[22:15:10] Running: /usr/bin/docker [docker rm -f -v kind-control-plane2 kind-control-plane kind-control-plane3 kind-worker kind-worker3 kind-worker2 kind-external-load-balancer] 
⠈⠁ Joining more control-plane nodes 🎮 Error: failed to create cluster: failed to join a control plane node with kubeadm: exit status 1

    What you expected to happen: no errors.

    How to reproduce it (as minimally and precisely as possible):

    cd kind-src-path
GO111MODULE=on go build
# install the kind binary to PATH
cd kubernetes-src-path
kind build node-image --kube-root=$(pwd)
kind create cluster --config=<path-to-above-ha-config> --image kindest/node:latest

    Anything else we need to know?:

    • i cannot reproduce the bug without --loglevel=debug.
    • sometimes it fails during joining the extra CP nodes, something during joining the workers.

    Environment:

    • kind version: (use kind version): master at 43bf0e2594db
    • Kubernetes version: master at 1409ff38e5828f55
    • Docker version: (use docker info):
    Containers: 10
 Running: 7
 Paused: 0
 Stopped: 3
Images: 128
Server Version: 18.06.3-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 468a545b9edcd5932818eb9de8e72413e616e86e
runc version: a592beb5bc4c4092b1b1bac971afed27687340c5
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.13.0-41-generic
Operating System: Ubuntu 17.10
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.66GiB
Name: luboitvbox
ID: K2H6:2I6N:FSBZ:S77V:R5CQ:X22B:VYTF:WZ4R:UIKC:HGOT:UCHD:GCR2
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
    • OS (e.g. from /etc/os-release):
    NAME="Ubuntu"
VERSION="17.10 (Artful Aardvark)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 17.10"
VERSION_ID="17.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=artful
UBUNTU_CODENAME=artful

    /kind bug /priority important-soon (?)

    kind/bug priority/important-soon lifecycle/active 
    opened by neolit123 62
  • Add IPv6 support

    Add IPv6 support

    This PR adds allows creating IPv6 Kubernetes clusters with kind and have in mind a future dual-stack implementation, considering for simplicity, only one address of each protocol.

    It adds a new option ipFamily to the v1alpha3 API that allows choosing the IP family of the cluster. To avoid issues with the different networking options the podSubnet and the serviceSubnet kubeadm values are predefined with the following values:

    	Default PodSubnet          = "10.244.0.0/16"
	Default ServicesSubnet     = "10.96.0.0/12"
	Default PodSubnetIPv6      = "fd00:10:244::/64"
	Default ServicesSubnetIPv6 = "fd00:10:96::/112"

    We can create a Kubernetes IPv6 cluster with the following config:

    # necessary for conformance
kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
networking:
  ipFamily: ipv6
nodes:
# the control plane node
- role: control-plane
- role: worker
- role: worker

    Test results with IPv4 and IPv6

    References:

    • https://github.com/kubernetes-sigs/kind/issues/280
    • https://docs.google.com/document/d/17e3TWWLfnIZrsVxpln9wNi4x0JVn2oHIHDYjaeENdVE/edit?ts=5c9af5b4#heading=h.on33tp91ehzk

    Fixes #280

    cncf-cla: yes lgtm size/L do-not-merge/hold tide/merge-method-squash ok-to-test 
    opened by aojea 58
  • ARM64 CI

    ARM64 CI

    per discussion in #kind slack, we should setup some CI with openlab to get kind on arm64 xref #166

    @dims was able to get arm64 working, but we'll need some set this up to keep it working once that goes in, as the maintainers do not have access to suitable arm machines to test on otherwise.

    /assign /kind feature /priority important-longterm

    kind/feature priority/important-longterm 
    opened by BenTheElder 56
  • overlay network cannot be applied when host is behind a proxy

    overlay network cannot be applied when host is behind a proxy

    Environment

    Host OS: RHEL 7.4 Host Docker version: 18.09.0 Host go version: go1.11.2 Node Image: kindest/node:v1.12.2

    kind create cluster

    [root@localhost bin]# kind create cluster
Creating cluster 'kind-1' ...
 ✓ Ensuring node image (kindest/node:v1.12.2) 🖼
 ✓ [kind-1-control-plane] Creating node container 📦
 ✓ [kind-1-control-plane] Fixing mounts 🗻
 ✓ [kind-1-control-plane] Starting systemd 🖥
 ✓ [kind-1-control-plane] Waiting for docker to be ready 🐋
 ✗ [kind-1-control-plane] Starting Kubernetes (this may take a minute) ☸
FATA[07:20:43] Failed to create cluster: failed to apply overlay network: exit status 1

    Code below in pkg/cluster/context.go is trying to extract k8s version using kubectl version command in order to download the version-specific weave net.yaml. The code is not ok:-

            // TODO(bentheelder): support other overlay networks
        if err = node.Command(
                "/bin/sh", "-c",
                `kubectl apply --kubeconfig=/etc/kubernetes/admin.conf -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version --kubeconfig=/etc/kubernetes/admin.conf | base64 | tr -d '\n')"`,
        ).Run(); err != nil {
                return kubeadmConfig, errors.Wrap(err, "failed to apply overlay network")
        }

    Why is the output of kubectl version command, base64 encoded?

    kind/bug priority/important-soon 
    opened by senthilrch 53
  • Unstable cluster

    Unstable cluster

    When running it locally on my machine the cluster seems much more unstable than on our CI. So now cluster is created inside a privileged container, but then I am getting strange errors:

    $ kubectl cluster-info
Unable to connect to the server: unexpected EOF

    $ kubectl cluster-info
Error from server (InternalError): an error on the server ("") has prevented the request from succeeding (get services)

    $ kubectl cluster-info 
error: the server doesn't have a resource type "services"
    triage/needs-information 
    opened by mitar 49
  • WSL2 ERROR: failed to create cluster

    WSL2 ERROR: failed to create cluster

    ERROR: failed to create cluster: ERROR: failed to create cluster: failed to init node with kubeadm: command "docker exec --privileged kind-control-plane kubeadm init --skip-phases=preflight --config=/kind/kubeadm.conf --skip-token-print --v=6" failed with error: exit status 1 Command Output: I0622 15:16:13.468494 216 initconfiguration.go:246] loading configuration from "/kind/kubeadm.conf"

    What is Expected: cluster should be created without any error.

    How to reproduce it: run below command to reproduce it : $ kind create cluster

    Anything else we need to know?: I have recently install ubuntu as virtual machine on windows 10 as wsl 2. running ubuntu on windows terminal as admin user, also installed docker and set it as non root user. below I am providing environment related information.

    ENVIRONMENT:

    Ubuntu command used $lsb_release -a

    No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.2 LTS Release: 20.04 Codename: focal

    Kubectl Installation https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-using-native-package-management

    Kubectl Version command used kubectl version --client

    Client Version: version.Info{ Major:"1", Minor:"21", GitVersion:"v1.21.2", GitCommit:"092fbfbf53427de67cac1e9fa54aaa09a28371d7", GitTreeState:"clean", BuildDate:"2021-06-16T12:59:11Z", GoVersion:"go1.16.5", Compiler:"gc", Platform:"linux/amd64" }

    kind Installtion :

    curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.11.1/kind-linux-amd64 chmod +x ./kind sudo mv ./kind /usr/local/bin/kind

    kind version: : command used $kind version

    kind v0.11.1 go1.16.4 linux/amd64

    docker info

    command used $docker info

    Client: Context: default Debug Mode: false Plugins: app: Docker App (Docker Inc., v0.9.1-beta3) buildx: Build with BuildKit (Docker Inc., v0.5.1-docker) scan: Docker Scan (Docker Inc., v0.8.0)

    Server: Containers: 1 Running: 0 Paused: 0 Stopped: 1 Images: 2 Server Version: 20.10.7 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux Default Runtime: runc Init Binary: docker-init containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7 init version: de40ad0 Security Options: seccomp Profile: default Kernel Version: 5.4.72-microsoft-standard-WSL2 Operating System: Ubuntu 20.04.2 LTS OSType: linux Architecture: x86_64 CPUs: 12 Total Memory: 6.133GiB Name: LAPTOP-TN6NO0LS ID: JDCK:NRQ2:ML5P:EUMK:OBYG:76PM:5SXD:FMYK:KHCX:NDTB:IQ4R:KIBJ Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

    WARNING: No blkio throttle.read_bps_device support WARNING: No blkio throttle.write_bps_device support WARNING: No blkio throttle.read_iops_device support WARNING: No blkio throttle.write_iops_device support

    kind/bug 
    opened by piyushvj 46
  • feat: support for multiple images to kind load

    feat: support for multiple images to kind load

    closes #1905

    Usage:

    Multiple image names provided (all images present locally):

    ./kind load docker-image nginx,busybox --name master

    Multiple image names provided (all images are not present locally):

    ./kind load docker-image nginx,busybox,python --name master
ERROR: image: "python" not present locally

    One image name provided without commas (backwards compatibility):

    ./kind load docker-image nginx --name master
    approved cncf-cla: yes lgtm size/M ok-to-test 
    opened by yashvardhan-kukreja 46
  • [WIP] Iptables mess

    [WIP] Iptables mess

    As explained in https://bugzilla.netfilter.org/show_bug.cgi?id=1632 there is no compatibility guarantee for the userspace iptables binaries, this causes that the output of iptables-save may be different if the iptables host version doesn't match the version used inside the kind node.

    Since the iptables we need to process for doing the dns magic are well known and can not change without breaking a lot of users, instead of manipulating the existing rules, just obtain the parameters that we need and remove and add the necessary rules.

    Fixes: #3054

    approved cncf-cla: yes size/M do-not-merge/work-in-progress 
    opened by aojea 4
  • How do I access etcd container?

    How do I access etcd container?

    Hi all,
    I have a simple cluster with the following configuration:

    kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
  - containerPort: 22
    hostPort: 22
- role: worker
- role: worker

    I successfully deploy it, and then try to access the etcd container by doing:

    kubectl -n kube-system exec etcd-kind-control-plane -it -- sh

    But I get the following error:

    error: Internal error occurred: error executing command in container: failed to exec in container: failed to start exec "a22a7843cc56d0d4b6133e0eca2e1404bf80a683ac46aceff9a2f092de46e852": OCI runtime exec failed: exec failed: unable to start container process: open /dev/ptmx: operation not permitted: unknown

    I get the same error if I try to do it within the control plane container. Any help is much appreciated. Kind version:

    kind v0.17.0 go1.19.2 linux/amd64

    Docker version:

    Client:
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.19.4
 Git commit:        3a2c30b63a
 Built:             Tue Dec 20 20:43:40 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.22
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.19.4
  Git commit:       42c8b31499
  Built:            Tue Dec 20 20:42:46 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.6.14
  GitCommit:        9ba4b250366a5ddde94bb7c9d1def331423aa323.m
 runc:
  Version:          1.1.4
  GitCommit:
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
    kind/support 
    opened by Detoxify92 5
  • kind with podman remote

    kind with podman remote

    What happened: we running podman as sidecar container in gitlab-ci pipelines - we configured the following:

    $ export DOCKER_HOST=tcp://podman:8888
$ export DOCKER_DRIVER=fuse-overlayfs
$ export DOCKER_TLS_CERTDIR=""
$ export CONTAINER_CONNECTION=tcp://podman:8888
$ export CONTAINER_HOST=tcp://podman:8888

$ kind create cluster --config test/kind-config.yaml
enabling experimental podman provider
Creating cluster "kind" ...
 • Ensuring node image (kindest/node:v1.21.10) 🖼  ...
 ✓ Ensuring node image (kindest/node:v1.21.10) 🖼
 • Preparing nodes 📦   ...
 ✗ Preparing nodes 📦 
ERROR: failed to create cluster: command "podman run --name kind-control-plane --hostname kind-control-plane --label io.x-k8s.kind.role=control-plane --privileged --tmpfs /tmp --tmpfs /run --volume 011549994c011b0585520606d44fc241d77c97d4d4b88e3379e1b68aefec41e3:/var:suid,exec,dev --volume /lib/modules:/lib/modules:ro -e KIND_EXPERIMENTAL_CONTAINERD_SNAPSHOTTER --detach --tty --net kind --label io.x-k8s.kind.cluster=kind -e container=podman --publish=127.0.0.1:45791:6443/tcp -e KUBECONFIG=/etc/kubernetes/admin.conf docker.io/kindest/node:v1.21.10" failed with error: exit status [126](https://gitlab.dev.xxx.sh/aws-ops/xxx/-/jobs/8774672#L126)
Command Output: Error: crun: set xattr for `runc.sha256`: Permission denied: OCI permission denied

    podman configuration:

    #!/bin/bash
echo "starting podman ..."
unset CONTAINER_HOST
podman system service --time 0 unix:///var/run/docker.sock & 
podman system service --time 0 tcp://0.0.0.0:8888

    and containers.conf

    [containers]
netns="host"
userns="host"
ipcns="host"
utsns="host"
cgroupns="host"
cgroups="disabled"
default_sysctls = []
log_driver = "k8s-file"
[engine]
cgroup_manager = "cgroupfs"
events_logger="file"
runtime="crun"

    any idea what is the problem ?

    What you expected to happen:

    How to reproduce it (as minimally and precisely as possible):

    Anything else we need to know?:

    Environment:

    • kind version: (use kind version):
    • Runtime info: (use docker info or podman info):
    • OS (e.g. from /etc/os-release):
    • Kubernetes version: (use kubectl version):
    • Any proxies or other special environment settings?:
    kind/support kind/external 
    opened by haarchri 2
  • RFE: listing supported providers and currently configured provider

    RFE: listing supported providers and currently configured provider

    What would you like to be added:

    It would be great if the kind cli can support listing supported providers as well as an option/flag to show the currently configured provider.

    e.g.

    kind provider lists all supported provider (docker, podman, etc.) kind provider --current would show the currently configured provider (docker)

    Why is this needed:

    We have use cases where having this would benefit our internal tooling for determining what the current provider configured for kind is. And in the case of forks of kind that are setting the default provider to be not docker, then we cannot assume that if KIND_EXPERIMENTAL_PROVIDER env is missing, that the configured provider would be docker. So having this subcommand making the configured provider explicit is very useful.

    Very much open to UX suggestions and other feedback around this.

    kind/feature 
    opened by yibozhuang 0
  • Adding support for 'kind provider' command

    Adding support for 'kind provider' command

    I'm very much open to UX feedback, as we can do something like kind get provider, but keeping it at kind provider for now as this seems to be somewhat separate from the other operations.

    There are use cases where it would help to be able to list the supported providers as well as display the currently configured runtime provider.

    This change adds a kind provider command which will display all supported providers. The --current flag can be provided to show the currently configured provider which honors the KIND_EXPERIMENTAL_PROVIDER env if set.

    Addresses: #3056

    Signed-off-by: Yibo Zhuang [email protected]

    cncf-cla: yes size/L 
    opened by yibozhuang 4
  • Networking somewhat broken with iptables nf_tables mode 1.8.8 on the host

    Networking somewhat broken with iptables nf_tables mode 1.8.8 on the host

    Quick placeholder issue for https://github.com/kubernetes/minikube/issues/15573#issuecomment-1374286267

    We're going to have to get creative to deal with this properly. cc @aojea

    kind/bug 
    opened by BenTheElder 3
Releases(v0.17.0)

  • v0.17.0(Oct 26, 2022)

    v0.17.0 is a small release centered around fixing a bug loading docker hub / docker.io tagged images with identical content but different tags (including images with no explicit host) https://github.com/kubernetes-sigs/kind/pull/2955 and support for cross-architecture kind load ....

    This release also ships Kubernetes 1.25.3 and containerd 1.6.9 with their respective fixes.

    This release comes to you live from KubeCon NA 2022 😄

    Breaking Changes

    • The default node image is a Kubernetes v1.25.3 image: kindest/node:v1.25.3@sha256:f52781bc0d7a19fb6c405c2af83abfeb311f130707a0e219175677e366cc45d1
    • Internal config generation now defends against yaml-injection
      • This should not be a breaking change if you were using the config fields as documented
      • This does not apply to config patches which are applied after config generation, and by definition patch arbitrary yaml

    New Features

    • Support for loading cross-architecture images
      • When using kind load docker-image or kind load image-archive, kind now instructs containerd to import all architectures.
      • This means that if you have multi-arch docker run enabled on your host (binfmt_misc qemu-userspace), such as in the Docker Desktop application out-of-the box, you may be able to load and run pods with images for the wrong architecture (e.g. an amd64 image on an M1 mac).
    • containerd 1.6.9
    • go 1.19.2
    • upgraded metallb https://github.com/kubernetes-sigs/kind/pull/2973
    • overhauled docs code snippets https://github.com/kubernetes-sigs/kind/pull/2894

    New Node images have been built for kind v0.17.0, please use these exact images (IE like kindest/node:v1.25.3@sha256:f52781bc0d7a19fb6c405c2af83abfeb311f130707a0e219175677e366cc45d1 including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.25: kindest/node:v1.25.3@sha256:f52781bc0d7a19fb6c405c2af83abfeb311f130707a0e219175677e366cc45d1
    • 1.24: kindest/node:v1.24.7@sha256:577c630ce8e509131eab1aea12c022190978dd2f745aac5eb1fe65c0807eb315
    • 1.23: kindest/node:v1.23.13@sha256:ef453bb7c79f0e3caba88d2067d4196f427794086a7d0df8df4f019d5e336b61
    • 1.22: kindest/node:v1.22.15@sha256:7d9708c4b0873f0fe2e171e2b1b7f45ae89482617778c1c875f1053d4cef2e41
    • 1.21: kindest/node:v1.21.14@sha256:9d9eb5fb26b4fbc0c6d95fa8c790414f9750dd583f5d7cee45d92e8c26670aa1
    • 1.20: kindest/node:v1.20.15@sha256:a32bf55309294120616886b5338f95dd98a2f7231519c7dedcec32ba29699394
    • 1.19: kindest/node:v1.19.16@sha256:476cb3269232888437b61deca013832fee41f9f074f9bed79f57e4280f7c48b7

    Additional images known compatible with this release:

    • 1.26: kindest/node:v1.26.0@sha256:691e24bd2417609db7e589e1a479b902d2e209892a10ce375fab60a8407c7352

    NOTE: These node images support amd64 and arm64. It remains possible to build custom images for other architectures (see the docs).

    Fixes

    • Fix loading docker hub / docker.io tagged images with identical content but different tags (including images with no explicit host) https://github.com/kubernetes-sigs/kind/pull/2955
    • kindnetd (kind's lightweight networking daemonset) now supports removing wrong routes when nodes are added and removed
      • currently, kind does not explicitly have support for adding or removing nodes however, Cluster API Provider Docker (which is based on KIND), does support this.

    Contributors

    Thank you to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @aude
    • @BenTheElder
    • @chrischdi
    • @converge
    • @curtbushko
    • @flash-me
    • @hrittikhere
    • @k8s-ci-robot
    • @mdurand54
    • @raphaelauv
    • @Vlatombe

    And thank you very much to everyone else not listed here who contributed in other ways like filing issues, giving feedback, testing fixes, helping users in slack, etc. 🙏

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(6.56 MB)
    kind-darwin-amd64.sha256sum(84 bytes)
    kind-darwin-arm64(6.47 MB)
    kind-darwin-arm64.sha256sum(84 bytes)
    kind-linux-amd64(6.60 MB)
    kind-linux-amd64.sha256sum(83 bytes)
    kind-linux-arm64(6.49 MB)
    kind-linux-arm64.sha256sum(83 bytes)
    kind-linux-ppc64le(6.56 MB)
    kind-linux-ppc64le.sha256sum(85 bytes)
    kind-linux-s390x(7.06 MB)
    kind-linux-s390x.sha256sum(83 bytes)
    kind-windows-amd64(6.71 MB)
    kind-windows-amd64.sha256sum(85 bytes)

  • v0.16.0(Sep 22, 2022)

    v0.16.0 is a quick release centered around shipping Kubernetes v1.25.2 fixes by default. Additional fixes and features are listed below.

    Breaking Changes

    • The default node image is a Kubernetes v1.25.2 image: kindest/node:v1.25.2@sha256:9be91e9e9cdf116809841fc77ebdb8845443c4c72fe5218f3ae9eb57fdb4bace
    • kind no longer attempts misguided symlink /dev/kmsg to /dev/console when /dev/kmsg is missing. please ensure your host has /dev/kmsg https://github.com/kubernetes-sigs/kind/issues/662#issuecomment-1238911235
    • Kubernetes v1.15.X and lower are no longer supported, as warned in KIND v0.15.0

    New Features

    • open-iscsi / support for iSCSI volumes
    • containerd 1.6.8
    • crictl 1.25.0
    • go 1.19.1

    New Node images have been built for kind v0.16.0, please use these exact images (IE like kindest/node:v1.25.2@sha256:9be91e9e9cdf116809841fc77ebdb8845443c4c72fe5218f3ae9eb57fdb4bace including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.25: kindest/node:v1.25.2@sha256:9be91e9e9cdf116809841fc77ebdb8845443c4c72fe5218f3ae9eb57fdb4bace
    • 1.24: kindest/node:v1.24.6@sha256:97e8d00bc37a7598a0b32d1fabd155a96355c49fa0d4d4790aab0f161bf31be1
    • 1.23: kindest/node:v1.23.12@sha256:9402cf1330bbd3a0d097d2033fa489b2abe40d479cc5ef47d0b6a6960613148a
    • 1.22: kindest/node:v1.22.15@sha256:bfd5eaae36849bfb3c1e3b9442f3da17d730718248939d9d547e86bbac5da586
    • 1.21: kindest/node:v1.21.14@sha256:ad5b7446dd8332439f22a1efdac73670f0da158c00f0a70b45716e7ef3fae20b
    • 1.20: kindest/node:v1.20.15@sha256:45d0194a8069c46483a0e509088ab9249302af561ebee76a1281a1f08ecb4ed3
    • 1.19: kindest/node:v1.19.16@sha256:a146f9819fece706b337d34125bbd5cb8ae4d25558427bf2fa3ee8ad231236f2

    NOTE: These node images support amd64 and arm64. It remains possible to build custom images for other architectures (see the docs).

    Fixes

    • Fix for detecting new podman network overlap errors
    • Updated metallb docs to current

    Contributors

    Thank you to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @benmoss
    • @BenTheElder
    • @bornaivankovic
    • @fedepaol
    • @fukuta-tatsuya-intec
    • @k8s-ci-robot

    And thank you very much to everyone else not listed here who contributed in other ways like filing issues, giving feedback, testing fixes, helping users in slack, etc. 🙏

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(6.51 MB)
    kind-darwin-amd64.sha256sum(84 bytes)
    kind-darwin-arm64(6.42 MB)
    kind-darwin-arm64.sha256sum(84 bytes)
    kind-linux-amd64(6.55 MB)
    kind-linux-amd64.sha256sum(83 bytes)
    kind-linux-arm64(6.36 MB)
    kind-linux-arm64.sha256sum(83 bytes)
    kind-linux-ppc64le(6.49 MB)
    kind-linux-ppc64le.sha256sum(85 bytes)
    kind-linux-s390x(6.99 MB)
    kind-linux-s390x.sha256sum(83 bytes)
    kind-windows-amd64(6.66 MB)
    kind-windows-amd64.sha256sum(85 bytes)

  • v0.15.0(Sep 2, 2022)

    v0.15.0 contains important fixes for cluster reboots and various other improvements.

    Breaking Changes

    • The default node image is a Kubernetes v1.25.0 image: kindest/node:v1.25.0@sha256:428aaa17ec82ccde0131cb2d1ca6547d13cf5fdabcc0bbecf749baa935387cbf

    New Features

    • New single letter flag aliases -n for --name and -A for --all, in-line with kubectl etc.
    • Optimized image loading to re-tag images when the image contents are identical to previously loaded images but the tags are different
    • Support for Kubernetes 1.25, fix for handling rootless + 1.25
    • Containerd 1.6.7
    • Go 1.19
    • Updated base image distro to latest, NOTE: depend on the contents of the image at your own risk! our images enable running Kubernetes with KIND, we reserve the right to switch distros etc as needed
    • Support for Podman 4.0 / netavark
    • enhanced pre-release versions to include commit counts

    New Node images have been built for kind v0.15.0, please use these exact images (IE like kindest/node:v1.25.0@sha256:428aaa17ec82ccde0131cb2d1ca6547d13cf5fdabcc0bbecf749baa935387cbf including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.25: kindest/node:v1.25.0@sha256:428aaa17ec82ccde0131cb2d1ca6547d13cf5fdabcc0bbecf749baa935387cbf
    • 1.24: kindest/node:v1.24.4@sha256:adfaebada924a26c2c9308edd53c6e33b3d4e453782c0063dc0028bdebaddf98
    • 1.23: kindest/node:v1.23.10@sha256:f047448af6a656fae7bc909e2fab360c18c487ef3edc93f06d78cdfd864b2d12
    • 1.22: kindest/node:v1.22.13@sha256:4904eda4d6e64b402169797805b8ec01f50133960ad6c19af45173a27eadf959
    • 1.21: kindest/node:v1.21.14@sha256:f9b4d3d1112f24a7254d2ee296f177f628f9b4c1b32f0006567af11b91c1f301
    • 1.20: kindest/node:v1.20.15@sha256:d67de8f84143adebe80a07672f370365ec7d23f93dc86866f0e29fa29ce026fe
    • 1.19: kindest/node:v1.19.16@sha256:707469aac7e6805e52c3bde2a8a8050ce2b15decff60db6c5077ba9975d28b98
    • 1.18: kindest/node:v1.18.20@sha256:61c9e1698c1cb19c3b1d8151a9135b379657aee23c59bde4a8d87923fcb43a91

    NOTE: These node images support amd64 and arm64. It remains possible to build custom images for other architectures (see the docs).

    Fixes

    • Fixed rebooted node certificates
    • Fixed snapshotter selection on ZFS + overlayfs-fuse
    • Podman provider now includes node names in no_proxy env, matching the docker provider
    • Assorted documentation fixes
    • Fixed Kubernetes 1.13 configuration
      • NOTE: This will be the last release supporting Kubernetes versions below v1.15.0

    Contributors

    Thank you to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @a-palchikov
    • @AkihiroSuda
    • @aojea
    • @arielmorelli
    • @Benny-Git
    • @BenTheElder
    • @bryanasdev000
    • @cavokz
    • @cpanato
    • @danwinship
    • @harshanarayana
    • @jkremser
    • @k8s-ci-robot
    • @keypointt
    • @lepistone
    • @lixin963
    • @naveensrinivasan
    • @pacoxu
    • @rewanthtammana
    • @tnqn
    • @vanhtuan0409
    • @wherka-ama
    • @zaunist

    And thank you very much to everyone else not listed here who contributed in other ways like filing issues, giving feedback, testing fixes, helping users in slack, etc. 🙏

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(6.51 MB)
    kind-darwin-amd64.sha256sum(84 bytes)
    kind-darwin-arm64(6.42 MB)
    kind-darwin-arm64.sha256sum(84 bytes)
    kind-linux-amd64(6.55 MB)
    kind-linux-amd64.sha256sum(83 bytes)
    kind-linux-arm64(6.36 MB)
    kind-linux-arm64.sha256sum(83 bytes)
    kind-linux-ppc64le(6.49 MB)
    kind-linux-ppc64le.sha256sum(85 bytes)
    kind-linux-s390x(6.99 MB)
    kind-linux-s390x.sha256sum(83 bytes)
    kind-windows-amd64(6.66 MB)
    kind-windows-amd64.sha256sum(85 bytes)

  • v0.14.0(May 19, 2022)

    v0.14.0 is quick follow-up to v0.13.0, upgrading packages and fixing cgroups on some non-systemd-based hosts such as WSL2 and Alpine based tools like colima and rancher-desktop.

    Besides the cgroups fix, the update to the latest version of the local-path-provisioner may be a desirable upgrade worth noting with various downstream improvements.

    Be sure to see the previous release notes as well!

    Breaking Changes

    NOTE: The systemd change is from KIND v0.13.0, but we're re-iterating it here since v0.13.0 was so recent.

    • systemd cgroups driver will be used for Kubernetes v1.24.0+ (rather than 1.21.0+ when kubeadm changed the default, which we previously overrode).
      • NOTE: You must use kind v0.13.0+ with Kubernetes v1.24.0+ images, and if you built your own Kubernetes v1.24.0+ image with a previous kind version you will need to re-built when switching to kind v0.13.0+.
      • NOTE: You do not need to be using systemd on the host machine, In kind v0.13.0 we had a bug related to this, it should be fixed this release. systemd is used inside the kind node containers but should not be necessary on the host. We are now using it for Kubernetes pods in addition to running kubelet, containerd etc.
      • KIND will continue to use cgroupfs for Kubernetes versions prior to v1.24.0.
    • The default node image is a Kubernetes v1.24.0 image: kindest/node:v1.24.0@sha256:0866296e693efe1fed79d5e6c7af8df71fc73ae45e3679af05342239cdc5bc8e

    New Features

    • digest image references should be populated when loading images
    • base image updates
      • crictl v1.24.0
    • kind binary built with Go 1.18.2
    • all kindnetd dependencies updated to latest
    • updated haproxy image
    • latest local-path-provisioner v0.0.22 with updated packaging
    • support for Kubernetes v1.25.0-alpha pre-releases builds without the old kubeadm node taint
    • updated cgroupsv2 CI to Fedora 36

    New Node images have been built for kind v0.14.0, please use these exact images (IE like kindest/node:v1.24.0@sha256:0866296e693efe1fed79d5e6c7af8df71fc73ae45e3679af05342239cdc5bc8e including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.24: kindest/node:v1.24.0@sha256:0866296e693efe1fed79d5e6c7af8df71fc73ae45e3679af05342239cdc5bc8e
    • 1.23: kindest/node:v1.23.6@sha256:b1fa224cc6c7ff32455e0b1fd9cbfd3d3bc87ecaa8fcb06961ed1afb3db0f9ae
    • 1.22: kindest/node:v1.22.9@sha256:8135260b959dfe320206eb36b3aeda9cffcb262f4b44cda6b33f7bb73f453105
    • 1.21: kindest/node:v1.21.12@sha256:f316b33dd88f8196379f38feb80545ef3ed44d9197dca1bfd48bcb1583210207
    • 1.20: kindest/node:v1.20.15@sha256:6f2d011dffe182bad80b85f6c00e8ca9d86b5b8922cdf433d53575c4c5212248
    • 1.19: kindest/node:v1.19.16@sha256:d9c819e8668de8d5030708e484a9fdff44d95ec4675d136ef0a0a584e587f65c
    • 1.18: kindest/node:v1.18.20@sha256:738cdc23ed4be6cc0b7ea277a2ebcc454c8373d7d8fb991a7fcdbd126188e6d7

    NOTE: these node images support amd64 and arm64. It remains possible to build custom images for other architectures (see the docs).

    Fixes

    • fixed running kind with kubernetes v1.24.0+ on some non-systemd-based hosts
    • setting migrated kubelet flags in kubelet config where possible, for now we set both the old flags and the config, in some future release we'll set only kubelet config where possible
    • fixed kong ingress name in docs

    Contributors

    Thank you to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @AkihiroSuda
    • @AppalaKarthik
    • @BenTheElder
    • @k8s-ci-robot
    • @neolit123

    A special shoutout to these folks who helped report, test, and review the systemd cgroups fix:

    • @aojea
    • @cndoit18
    • @ckoenig
    • @jankoprowski
    • @longwuyuan
    • @mariomac
    • @stmcginnis
    • @Unveraenderbar

    And thank you very much to everyone else not listed here who contributed in other ways like filing issues, giving feedback, testing fixes, helping users in slack, etc. 🙏

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(6.30 MB)
    kind-darwin-amd64.sha256sum(84 bytes)
    kind-darwin-arm64(6.26 MB)
    kind-darwin-arm64.sha256sum(84 bytes)
    kind-linux-amd64(6.35 MB)
    kind-linux-amd64.sha256sum(83 bytes)
    kind-linux-arm64(6.29 MB)
    kind-linux-arm64.sha256sum(83 bytes)
    kind-linux-ppc64le(6.29 MB)
    kind-linux-ppc64le.sha256sum(85 bytes)
    kind-linux-s390x(6.73 MB)
    kind-linux-s390x.sha256sum(83 bytes)
    kind-windows-amd64(6.31 MB)
    kind-windows-amd64.sha256sum(85 bytes)

  • v0.13.0(May 10, 2022)

    v0.13.0 is all about cgroups -- We're making the switch to the systemd cgroup driver to align with current Kubernetes container runtime recommendations and kubeadm defaults.

    Breaking Changes

    • systemd cgroups driver will be used for Kubernetes v1.24.0+ (rather than 1.21.0+ when kubeadm changed the default, which we previously overrode).
      • NOTE: You must use kind v0.13.0+ with Kubernetes v1.24.0+ images, and if you built your own Kubernetes v1.24.0+ image with a previous kind version you will need to re-built when switching to kind v0.13.0+.
      • NOTE: ~You do not need to be using systemd on the host machine~, systemd is used inside the kind node containers. We are now using it for Kubernetes pods in addition to running kubelet, containerd etc.
        • UPDATE: There is a bug on hosts that are (cgroupv1, not-systemd) https://github.com/kubernetes-sigs/kind/issues/2765, a fix is pending and the next release will address this. See the linked issue for discussion and workarounds in the meantime.
        • UPDATE: v0.14.0 is released with a fix for this bug. Upgrading should be the preferred solution, but if you need to use v0.13.0 for some reason, see the issue linked above for other workarounds.
      • KIND will continue to use cgroupfs for Kubernetes versions prior to v1.24.0.
    • The default node image is a Kubernetes v1.24.0 image: kindest/node:v1.24.0@sha256:406fd86d48eaf4c04c7280cd1d2ca1d61e7d0d61ddef0125cb097bc7b82ed6a1

    New Features

    • Limited support for PPC64LE architecture (you will need to build your own node images)
    • kind export logs now dumps a list of images on each node
    • Base image updates
      • Containerd version 1.6.4
      • CNI plugins 1.1.1
    • kind binary built with Go 1.18
    • General Go dependency updates
    • registry.k8s.io is used as the primary mirror for k8s.gcr.io in kind nodes / image building
      • If registry.k8s.io is not reachable, the node runtime is configured to fallback to k8s.gcr.io directly as the next endpoint

    New Node images have been built for kind v0.13.0, please use these exact images (IE like kindest/node:v1.24.0@sha256:406fd86d48eaf4c04c7280cd1d2ca1d61e7d0d61ddef0125cb097bc7b82ed6a1 including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.24: kindest/node:v1.24.0@sha256:406fd86d48eaf4c04c7280cd1d2ca1d61e7d0d61ddef0125cb097bc7b82ed6a1
    • 1.23: kindest/node:v1.23.6@sha256:1af0f1bee4c3c0fe9b07de5e5d3fafeb2eec7b4e1b268ae89fcab96ec67e8355
    • 1.22: kindest/node:v1.22.9@sha256:6e57a6b0c493c7d7183a1151acff0bfa44bf37eb668826bf00da5637c55b6d5e
    • 1.21: kindest/node:v1.21.12@sha256:ae05d44cc636ee961068399ea5123ae421790f472c309900c151a44ee35c3e3e
    • 1.20: kindest/node:v1.20.15@sha256:a6ce604504db064c5e25921c6c0fffea64507109a1f2a512b1b562ac37d652f3
    • 1.19: kindest/node:v1.19.16@sha256:dec41184d10deca01a08ea548197b77dc99eeacb56ff3e371af3193c86ca99f4
    • 1.18: kindest/node:v1.18.20@sha256:38a8726ece5d7867fb0ede63d718d27ce2d41af519ce68be5ae7fcca563537ed

    NOTE: these node images support amd64 and arm64. It remains possible to build custom images for other architectures (see the docs).

    Fixes

    • In cgroup v1, unmount cgroups that are not supported by the runtime used to create nodes
    • Pinned metallb to a stable version in documentation

    Contributors

    Thank you to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    Contributors since v0.12.0:

    • @aojea
    • @BenTheElder
    • @chendave
    • @dims
    • @iamtodor
    • @jpmcb
    • @k8s-ci-robot
    • @kolyshkin
    • @mkumatag
    • @pacoxu
    • @stmcginnis
    • @yxxhero
    • @zaunist
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(6.30 MB)
    kind-darwin-amd64.sha256sum(84 bytes)
    kind-darwin-arm64(6.24 MB)
    kind-darwin-arm64.sha256sum(84 bytes)
    kind-linux-amd64(6.34 MB)
    kind-linux-amd64.sha256sum(83 bytes)
    kind-linux-arm64(6.22 MB)
    kind-linux-arm64.sha256sum(83 bytes)
    kind-linux-ppc64le(6.29 MB)
    kind-linux-ppc64le.sha256sum(85 bytes)
    kind-linux-s390x(6.73 MB)
    kind-linux-s390x.sha256sum(83 bytes)
    kind-windows-amd64(6.31 MB)
    kind-windows-amd64.sha256sum(85 bytes)

  • v0.12.0(Mar 7, 2022)

    v0.12.0 has been focused on stability, with improvements and fixes for support rootless and cgroupsv2, and bringing support for s390x architectures.

    Breaking Changes

    • The default node image is a Kubernetes v1.23.4 image: kindest/node:v1.23.4@sha256:0e34f0d0fd448aa2f2819cfd74e99fe5793a6e4938b328f657c8e3f81ee0dfb9
    • The haproxy image used for multiple control-plane cluster is distroless now

    New Features

    • Pods running at different nodes have different product_uuid
    • Support for s390x architectures
    • Adapt kind to handle the label migration from node-role.kubernetes.io/master to node-role.kubernetes.io/control-plane
    • Base image updates
      • Containerd version 1.5.10
      • Crictl 1.23.0
      • CNI plugins 1.1.0
      • Containerd containerd/fuse-overlayfs-snapshotter 1.0.4

    New Node images have been built for kind v0.12.0, please use these exact images (IE like kindest/node:v1.23.4@sha256:0e34f0d0fd448aa2f2819cfd74e99fe5793a6e4938b328f657c8e3f81ee0dfb9 including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.23: kindest/node:v1.23.4@sha256:0e34f0d0fd448aa2f2819cfd74e99fe5793a6e4938b328f657c8e3f81ee0dfb9
    • 1.22: kindest/node:v1.22.7@sha256:1dfd72d193bf7da64765fd2f2898f78663b9ba366c2aa74be1fd7498a1873166
    • 1.21: kindest/node:v1.21.10@sha256:84709f09756ba4f863769bdcabe5edafc2ada72d3c8c44d6515fc581b66b029c
    • 1.20: kindest/node:v1.20.15@sha256:393bb9096c6c4d723bb17bceb0896407d7db581532d11ea2839c80b28e5d8deb
    • 1.19: kindest/node:v1.19.16@sha256:81f552397c1e6c1f293f967ecb1344d8857613fb978f963c30e907c32f598467
    • 1.18: kindest/node:v1.18.20@sha256:e3dca5e16116d11363e31639640042a9b1bd2c90f85717a7fc66be34089a8169
    • 1.17: kindest/node:v1.17.17@sha256:e477ee64df5731aa4ef4deabbafc34e8d9a686b49178f726563598344a3898d5
    • 1.16: kindest/node:v1.16.15@sha256:64bac16b83b6adfd04ea3fbcf6c9b5b893277120f2b2cbf9f5fa3e5d4c2260cc
    • 1.15: kindest/node:v1.15.12@sha256:9dfc13db6d3fd5e5b275f8c4657ee6a62ef9cb405546664f2de2eabcfd6db778
    • 1.14: kindest/node:v1.14.10@sha256:b693339da2a927949025869425e20daf80111ccabf020d4021a23c00bae29d82

    NOTE: these node images support amd64 and arm64 now. It remains possible to build custom images for other architectures (see the docs).

    Fixes

    • Enable sysctl route_localnet on node images to allow nodes dns to work
    • Only mount /dev/fuse on rootless environments
    • Fixed node labels assignments from configuration
    • Fix docker+cgroup2+rootless initialization
    • Fix sysbox runtime regression on docker in docker
    • Fix support on distros without systemd and WSL2
    • Fix kind load docker-image if all images are already present

    Contributors

    Thank you to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    Contributors since v0.11.1:

    • @AkihiroSuda
    • @Alfa-Alex
    • @amwat
    • @aojea
    • @atoato88
    • @bagnaram
    • @BenTheElder
    • @chaodaiG
    • @csams
    • @dlipovetsky
    • @enyineer
    • @fedepaol
    • @fenggw-fnst
    • @flouthoc
    • @fstrudel
    • @gAmUssA
    • @gotzmann
    • @gprossliner
    • @guirish
    • @hs0210
    • @jayonlau
    • @jimangel
    • @juniorjbn
    • @k8s-ci-robot
    • @kerthcet
    • @knight42
    • @KrisJohnstone
    • @ktock
    • @marcosnils
    • @matzew
    • @ml-
    • @neolit123
    • @networkop
    • @oke-py
    • @pchan
    • @pmorch
    • @PushkarJ
    • @qinqon
    • @rahulii
    • @rikatz
    • @sestegra
    • @sethp
    • @simon-geard
    • @stevekuznetsov
    • @stmcginnis
    • @tao12345666333
    • @tgeoghegan
    • @valeneiko
    • @vdjagilev
    • @viveksyngh
    • @wanlerong
    • @xinydev
    • @YuviGold
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(6.32 MB)
    kind-darwin-amd64.sha256sum(84 bytes)
    kind-darwin-arm64(6.61 MB)
    kind-darwin-arm64.sha256sum(84 bytes)
    kind-linux-amd64(6.35 MB)
    kind-linux-amd64.sha256sum(83 bytes)
    kind-linux-arm64(6.39 MB)
    kind-linux-arm64.sha256sum(83 bytes)
    kind-linux-ppc64le(6.40 MB)
    kind-linux-ppc64le.sha256sum(85 bytes)
    kind-linux-s390x(6.71 MB)
    kind-linux-s390x.sha256sum(83 bytes)
    kind-windows-amd64(6.41 MB)
    kind-windows-amd64.sha256sum(85 bytes)

  • v0.11.1(May 28, 2021)

    v0.11.1 fixes a security vulnerability in runc <=1.0.0-rc94

    For full release notes please see v0.11.0

    New Features

    • Base image updates
      • Containerd sandbox image to pause v3.5
      • Containerd version 1.5.2 / runc rc95
      • Ubuntu 21.04
    • Documented support for installing kind via macports.

    New Node images have been built for kind v0.11.1, please use these exact images (IE like kindest/node:v1.21.1@sha256:fae9a58f17f18f06aeac9772ca8b5ac680ebbed985e266f711d936e91d113bad including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.21: kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6
    • 1.20: kindest/node:v1.20.7@sha256:cbeaf907fc78ac97ce7b625e4bf0de16e3ea725daf6b04f930bd14c67c671ff9
    • 1.19: kindest/node:v1.19.11@sha256:07db187ae84b4b7de440a73886f008cf903fcf5764ba8106a9fd5243d6f32729
    • 1.18: kindest/node:v1.18.19@sha256:7af1492e19b3192a79f606e43c35fb741e520d195f96399284515f077b3b622c
    • 1.17: kindest/node:v1.17.17@sha256:66f1d0d91a88b8a001811e2f1054af60eef3b669a9a74f9b6db871f2f1eeed00
    • 1.16: kindest/node:v1.16.15@sha256:83067ed51bf2a3395b24687094e283a7c7c865ccc12a8b1d7aa673ba0c5e8861
    • 1.15: kindest/node:v1.15.12@sha256:b920920e1eda689d9936dfcf7332701e80be12566999152626b2c9d730397a95
    • 1.14: kindest/node:v1.14.10@sha256:f8a66ef82822ab4f7569e91a5bccaf27bceee135c1457c512e54de8c6f7219f8

    Additionally the following image is known to work well:

    • 1.22: kindest/node:v1.22.0@sha256:b8bda84bb3a190e6e028b1760d277454a72267a5454b57db34437c34a588d047
    • 1.23: kindest/node:v1.23.0@sha256:49824ab1727c04e56a21a5d8372a402fcd32ea51ac96a2706a12af38934f81ac

    NOTE: these node images support amd64 and arm64 now. It remains possible to build custom images for other architectures (see the docs).

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(6.47 MB)
    kind-darwin-arm64(6.39 MB)
    kind-linux-amd64(6.50 MB)
    kind-linux-arm64(6.19 MB)
    kind-linux-ppc64le(6.26 MB)
    kind-windows-amd64(6.58 MB)

  • v0.11.0(May 18, 2021)

    v0.11.0 brings in much sought after support for multi-arch, rootless, dual stack along with a couple of performance fixes.

    Breaking Changes

    • The default node image is a Kubernetes v1.21.1 image: kindest/node:v1.21.1@sha256:fae9a58f17f18f06aeac9772ca8b5ac680ebbed985e266f711d936e91d113bad
    • Dropped support for building node images with bazel going forward as part of reducing the build maintenance for Kubernetes. The --type flag is deprecated.
    • The kind build node-image --kube-root flag is deprecated in favor of an argument, using the flag will print a warning. In a future release the flag will be removed.

    New Features

    • kind build node-image now supports --arch for cross compiling node images
    • pre-built node images now support arm64 in addition to amd64 (they are multi-arch)
    • Support for running kind with cgroupv2, rootless docker and rootless podman. Please checkout our guide at https://kind.sigs.k8s.io/docs/user/rootless/
    • kindnetd supports dual-stack networking now enabled by default since kubernetes v1.21
    • Internal kubeconfig is now automatically exposed on the control plane nodes for in-cluster access.

    New Node images have been built for kind v0.11.0, please use these exact images (IE like kindest/node:v1.21.1@sha256:fae9a58f17f18f06aeac9772ca8b5ac680ebbed985e266f711d936e91d113bad including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.21: kindest/node:v1.21.1@sha256:fae9a58f17f18f06aeac9772ca8b5ac680ebbed985e266f711d936e91d113bad
    • 1.20: kindest/node:v1.20.7@sha256:e645428988191fc824529fd0bb5c94244c12401cf5f5ea3bd875eb0a787f0fe9
    • 1.19: kindest/node:v1.19.11@sha256:7664f21f9cb6ba2264437de0eb3fe99f201db7a3ac72329547ec4373ba5f5911
    • 1.18: kindest/node:v1.18.19@sha256:530378628c7c518503ade70b1df698b5de5585dcdba4f349328d986b8849b1ee
    • 1.17: kindest/node:v1.17.17@sha256:c581fbf67f720f70aaabc74b44c2332cc753df262b6c0bca5d26338492470c17
    • 1.16: kindest/node:v1.16.15@sha256:430c03034cd856c1f1415d3e37faf35a3ea9c5aaa2812117b79e6903d1fc9651
    • 1.15: kindest/node:v1.15.12@sha256:8d575f056493c7778935dd855ded0e95c48cb2fab90825792e8fc9af61536bf9
    • 1.14: kindest/node:v1.14.10@sha256:6033e04bcfca7c5f2a9c4ce77551e1abf385bcd2709932ec2f6a9c8c0aff6d4f

    NOTE: these node images support amd64 and arm64 now. It remains possible to build custom images for other architectures (see the docs).

    Fixes

    • handle Kubernetes / kubeadm 1.21 breaking cgroups config change.
    • kindnetd now tolerates NoSchedule
    • Fixed kube-proxy CrashLoops by skipping setting nf_conntrack_max
    • Fixed an issue with SIGPIPE errors sometimes failing node startup
    • kind export logs now only exports the version once instead of per node
    • kind export logs always prints export path even if some logs fail to save
    • Fixed a bug where kubeadmConfigPatches weren't merging correctly with generated arguments.
    • Fixed base image build potentially breaking due to file permissions being affected by umask settings on the host.
    • Improved podman support for custom portmaps and zfs, btrfs storage drivers.
    • Fixes for zfs, btrfs storage drivers on docker

    Contributors

    Thank you to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @AkihiroSuda
    • @amwat
    • @aojea
    • @atoato88
    • @BenTheElder
    • @brb
    • @Colstuwjx
    • @houfangdong
    • @imjasonh
    • @imw
    • @jason-kane
    • @jsievenpiper
    • @k8s-ci-robot
    • @kjellkvinge
    • @knabben
    • @matzew
    • @mauilion
    • @MaXinjian
    • @micahhausler
    • @OhMyMndy
    • @pacoxu
    • @rudeigerc
    • @tao12345666333
    • @tokers
    • @vainu-lauri
    • @yashvardhan-kukreja
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(6.47 MB)
    kind-darwin-arm64(6.39 MB)
    kind-linux-amd64(6.50 MB)
    kind-linux-arm64(6.19 MB)
    kind-linux-ppc64le(6.26 MB)
    kind-windows-amd64(6.58 MB)

  • v0.10.0(Jan 23, 2021)

    In v0.10.0 the most important changes we're "shipping" are about improving sustainability:

    • Doubled the set of project owners, granting @aojea & @amwat full access to help maintain the project. A huge thank you to them for their existing contributions and for stepping up here :heart:
    • Rewritten and revamped contributor guide https://kind.sigs.k8s.io/docs/contributing/getting-started/

    Additionally we're trying a new approach to tracking priorities for v0.11.0 which you can see in our issue tracker.

    Otherwise we've made a number of bug fixes and minor improvements.

    Some users may appreciate the ability to run kind v0.10.0 (NOT the currently installed v0.7.0) in the free Google Cloud Shell for quick demos, workshops, etc.

    Simply install kind somewhere under $HOME and add it to your $PATH to use it in cloud shell:

    GO111MODULE=on go get sigs.k8s.io/[email protected]
export PATH="$(go env GOPATH)/bin:${PATH}"
kind create cluster

    To persist this PATH for future usage you can edit your $HOME/.bashrc in cloudshell.

    Breaking Changes

    • The default node image is a Kubernetes v1.20.2 image: kindest/node:v1.20.2@sha256:8f7ea6e7642c0da54f04a7ee10431549c0257315b3a634f6ef2fecaaedb19bab
    • We're only actively supporting Kubernetes v1.14.0+, however limited best effort functionality still encompases v1.13.0 for now.
    • Images from KIND v0.8.0+ should continue to work for now, but lack various improvements.
    • udev is explicitly disabled at the node level, though as always please try not to depend on the inner details of nodes beyond providing a particular Kubernetes version with kind
    • IPv6 pod subnet defaults to /56 instead of /64 (a necessary fix for newer Kubernetes, see #1903)

    New Features

    • kind export logs now includes the kind version
    • Node images compiled without dockershim for Kubernetes v1.19+ possible, reducing size
    • Reduced kind binary size further (~7MB) with improved build options
    • Improved zsh completion
    • Experimental github actions CI for podman, docker, cgroupsv2
    • Expanded WSL2 documentation
    • Revamped contributor guide
    • Improved docs site implementation, including table of contents generation
    • Updated dependencies
    • New Loadbalancer user guide
    • Better support for running in nested container environments when using images built with v0.10.0+
      • This enables cloud shell support

    New Node images have been built for kind v0.10.0, please use these exact images (IE like v1.20.2:@sha256:8f7ea6e7642c0da54f04a7ee10431549c0257315b3a634f6ef2fecaaedb19bab including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.20: kindest/node:v1.20.2@sha256:8f7ea6e7642c0da54f04a7ee10431549c0257315b3a634f6ef2fecaaedb19bab
    • 1.19: kindest/node:v1.19.7@sha256:a70639454e97a4b733f9d9b67e12c01f6b0297449d5b9cbbef87473458e26dca
    • 1.18: kindest/node:v1.18.15@sha256:5c1b980c4d0e0e8e7eb9f36f7df525d079a96169c8a8f20d8bd108c0d0889cc4
    • 1.17: kindest/node:v1.17.17@sha256:7b6369d27eee99c7a85c48ffd60e11412dc3f373658bc59b7f4d530b7056823e
    • 1.16: kindest/node:v1.16.15@sha256:c10a63a5bda231c0a379bf91aebf8ad3c79146daca59db816fb963f731852a99
    • 1.15: kindest/node:v1.15.12@sha256:67181f94f0b3072fb56509107b380e38c55e23bf60e6f052fbd8052d26052fb5
    • 1.14: kindest/node:v1.14.10@sha256:3fbed72bcac108055e46e7b4091eb6858ad628ec51bf693c21f5ec34578f6180

    Fixes

    • Fixed development scripts when CDPATH is in use
    • Fixed building node images with bazel when CWD is not within the source directory
    • Disabled fancy terminal output when in st (see #1924)
    • Fixed various typos
    • Improved cgroups handling
    • Improved some error messages

    Contributors

    Thank you to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @AkihiroSuda
    • @AndiDog
    • @aojea
    • @arangamani
    • @BenTheElder
    • @FedericoAntoniazzi
    • @frelon
    • @Git-Jiro
    • @goneri
    • @gunadhya
    • @Hellcatlk
    • @Huang-Wei
    • @jbarrick-mesosphere
    • @jessehu
    • @jlucktay
    • @jsoref
    • @k8s-ci-robot
    • @kensipe
    • @knabben
    • @micarlise
    • @mjpitz
    • @nicholasdille
    • @nicks
    • @yangy2000
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(7.05 MB)
    kind-linux-amd64(7.10 MB)
    kind-linux-arm64(6.64 MB)
    kind-linux-ppc64le(6.71 MB)
    kind-windows-amd64(7.04 MB)

  • v0.9.0(Sep 15, 2020)

    v0.9.0 Focuses on stability enhancements following v0.8.0 / v0.8.1, a new wave of features will ship in v0.10.0.

    Breaking changes have been kept to a minimum, primarily that Kubernetes v1.12.X is no longer supported to make way for some fixes requiring beta-grade kubeadm.

    Breaking Changes

    • The default node image is a Kubernetes v1.19.1 image: kindest/node:v1.19.1@sha256:98cf5288864662e37115e362b23e4369c8c4a408f99cbc06e58ac30ddc721600
    • Node images built with kind v0.9.0 require using kind v0.9.0+.
      • These images should mostly work with older releases but may not work offline.
      • Images from kind v0.8.0+ should still work but lack some internal improvements
    • Support has been dropped for Kubernetes older than v1.13.0.
      • A detailed support policy is in the works. The Kubernetes project only supports v1.16+ currently.
      • UPDATE: Kubernetes v1.14+ is required for multi-node clusters. See https://github.com/kubernetes-sigs/kind/pull/1744#issuecomment-692905480
    • Building Kubernetes with bazel will only work with Kubernetes v1.15+
      • Building without bazel still works back to v1.13.0
    • v1alpha3 kind config is no longer supported. Please upgrade to v1alpha4 (which was already the current version in previous releases)
    • Capital letters in cluster names are explicitly rejected by kind, in order to prevent encountering upstream issues with name validation

    UPDATE: If you are hitting label key and value greater than maximum size (4096 bytes), key: containerd: invalid argument (https://github.com/containerd/cri/pull/1572), then try the following KIND config:

    kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".containerd]
  disable_snapshot_annotations = true

    See: https://kind.sigs.k8s.io/docs/user/configuration/#getting-started for how to use a KIND config file.

    New Features

    • NFS volumes should now work
    • Kubernetes RuntimeConfig is now first class in kind config
    • Upgraded dependencies across the board
    • Nodes now have a tenative providerID value set. We commit to the kind:// prefix but the rest of the contents may change.
    • The base image is tentatively multi-arch by default. The node image is still not as of yet.
    • More values are automatically included in no_proxy when proxies are detected
    • The default CNI shoult automatically match MTU to the underlying bridge
    • Binaries are now stripped of debugger (not stacktrace) info for smaller binaries

    New Node images have been built for kind v0.9.0, please use these exact images (IE like v1.19.1:@sha256:98cf5288864662e37115e362b23e4369c8c4a408f99cbc06e58ac30ddc721600 including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.19: kindest/node:v1.19.1@sha256:98cf5288864662e37115e362b23e4369c8c4a408f99cbc06e58ac30ddc721600
    • 1.18: kindest/node:v1.18.8@sha256:f4bcc97a0ad6e7abaf3f643d890add7efe6ee4ab90baeb374b4f41a4c95567eb
    • 1.17: kindest/node:v1.17.11@sha256:5240a7a2c34bf241afb54ac05669f8a46661912eab05705d660971eeb12f6555
    • 1.16: kindest/node:v1.16.15@sha256:a89c771f7de234e6547d43695c7ab047809ffc71a0c3b65aa54eda051c45ed20
    • 1.15: kindest/node:v1.15.12@sha256:d9b939055c1e852fe3d86955ee24976cab46cba518abcb8b13ba70917e6547a6
    • 1.14: kindest/node:v1.14.10@sha256:ce4355398a704fca68006f8a29f37aafb49f8fc2f64ede3ccd0d9198da910146
    • 1.13: kindest/node:v1.13.12@sha256:1c1a48c2bfcbae4d5f4fa4310b5ed10756facad0b7a2ca93c7a4b5bae5db29f5

    Fixes

    • Limited fixes related to HA mode and restart support
    • Fixed issues with ipv6 network overlap
    • Fixed bugs with NO_PROXY generation
    • Mitigated issues with concurrent cluster creation on clean hosts
    • KUBECONFIG writing has retries to mitigate concurrency / locking issues
    • Docker data root on ZFS should be fixed
    • Fixed building with bazel in kubernetes 1.20 development
    • Implemented assorted workarounds for breaking bugs in podman v2.X
    • Upstream CNI fixes identified by the project have been upstreamed and picked up to mitigate excessive iptables calls in testing
    • Replaced broken component IP auto-detection with explicit addresses to work around upstream Kubernetes limitations (pending an agreement on how to move forward upstream)
    • Fixed some issues with userns-remap support
    • Fixed port forwarding in some cases

    Contributors

    Thanks again to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @afbjorklund
    • @aledbf
    • @amwat
    • @aojea
    • @babilen5
    • @BenTheElder
    • @damdo
    • @dims
    • @ericsyh
    • @faiq
    • @giuseppe
    • @HadesArchitect
    • @Hellcatlk
    • @inercia
    • @izzyleung
    • @jayunit100
    • @k8s-ci-robot
    • @liggitt
    • @MaXinjian
    • @MaxRenaud
    • @mikedanese
    • @oomichi
    • @peteroneilljr
    • @prasadkatti
    • @rcarrillocruz
    • @rikatz
    • @SataQiu
    • @sozercan
    • @steigr
    • @tao12345666333
    • @vdice
    • @yashbhutwala
    • @zhijianli88
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(7.08 MB)
    kind-linux-amd64(7.07 MB)
    kind-linux-arm64(6.64 MB)
    kind-linux-ppc64le(6.71 MB)
    kind-windows-amd64(7.02 MB)

  • v0.8.1(May 2, 2020)

  • v0.8.0(May 1, 2020)

    v0.8.0 Most importantly brings persistent clusters that automatically come back up after dockerd / host restarts. It also brings a number of fixes to networking in general, experimental podman backend support, improved failure output, and more.

    Breaking Changes

    • The default node image is a Kubernetes v1.18.2 image: kindest/node:v1.18.2@sha256:7b27a6d0f2517ff88ba444025beae41491b016bc6af573ba467b70c5e8e0d85f
    • KIND v0.8.0 requires node images built with v0.8.0+.
      • These new images should be mostly backwards compatible for kind v0.5.0+
    • The default docker network for nodes is now kind instead of the default docker bridge
      • This enabled many fixes, however if you were running things alongside kind you may need to pass --net=kind
      • Your host DNS resolver should just work now :tm: in many cases
      • IPv6 clusters no longer require modifying docker daemon config
      • Docker networks must have a free IPv6 /64 in the [ULA] range, that is you may not create other docker networks consuming the entire ULA range. You can work around this by precreating the kind network with some totally other range if you wish.
    • You may need to migrate to the new featureGates config field when setting feature gates, this should be easier to use though!

    NOTE: If you've been experimenting with kind > v0.7.0 installed from HEAD, you may need to docker network rm kind before creating IPv6 clusters.

    New Features

    • Clusters should not be broken by dockerd restarts / restarting your host!
    • Host DNS should just work :tm:
    • IPv6 Clusters no longer require modifying docker daemon settings
    • Kubernetes 1.18+ should work on hosts using the iptables nft backend
    • kind delete clusters --all for deleting all your clusters
    • New top level featureGates field in Cluster config for enabling feature gates
    • Upgraded dependencies broadly, including library packages, go, containerd, CNI, etc...
    • NO_COLOR (no-color.org) support
    • Experimental podman support
      • These clusters do not survive host reboots
      • Requires rootful mode
      • Experimental
    • Improved output on failures, including command output without needing to set -v
    • Experimentally cross-building the base image for AMD64 + ARM by default
    • More Documentation! Guide to working with KIND offline, Ingress and more!
    • New download host and experimental nightly builds
    • Support for Dynamic PV in Kubernetes 1.11.X
    • New top level featureGates config option for controlling cluster featuregates
    • Shiny new WorkGroup-K8s-Infra based infrastructure for Kubernetes CI builds
    • Support for KIND_CLUSTER_NAME environment variable isntead of --name

    New Node have been Images for kind v0.8.0, please use these exact images (IE like v1.18.2:@sha256:7b27a6d0f2517ff88ba444025beae41491b016bc6af573ba467b70c5e8e0d85f including the digest) or build your own as we may need to change the image format again in the future :sweat_smile:

    Images built for this release:

    • 1.18: kindest/node:v1.18.2@sha256:7b27a6d0f2517ff88ba444025beae41491b016bc6af573ba467b70c5e8e0d85f
    • 1.17: kindest/node:v1.17.5@sha256:ab3f9e6ec5ad8840eeb1f76c89bb7948c77bbf76bcebe1a8b59790b8ae9a283a
    • 1.16: kindest/node:v1.16.9@sha256:7175872357bc85847ec4b1aba46ed1d12fa054c83ac7a8a11f5c268957fd5765
    • 1.15: kindest/node:v1.15.11@sha256:6cc31f3533deb138792db2c7d1ffc36f7456a06f1db5556ad3b6927641016f50
    • 1.14: kindest/node:v1.14.10@sha256:6cd43ff41ae9f02bb46c8f455d5323819aec858b99534a290517ebc181b443c6
    • 1.13: kindest/node:v1.13.12@sha256:214476f1514e47fe3f6f54d0f9e24cfb1e4cda449529791286c7161b7f9c08e7
    • 1.12: kindest/node:v1.12.10@sha256:faeb82453af2f9373447bb63f50bae02b8020968e0889c7fa308e19b348916cb
    • 1.11: kindest/node:v1.11.10@sha256:74c8740710649a3abb169e7f348312deff88fc97d74cfb874c5095ab3866bb42

    EDIT: 1.11.X does not work. See: https://github.com/kubernetes-sigs/kind/issues/1565

    Fixes

    • Avoid potentially slow sync during bootstrap
    • Fixed some occasional flakes when exporting logs
    • Simple output mode automatically activated on Travis CI to avoid broken PTY behavior
    • Fixed image import on ARM
    • Fixed abspath resolution in cluster config on Windows
    • Fixed DNS resolution on some systems
    • Fixed building Kubernetes with Bazel
    • Fixed using kind on ZFS / BTRFS backed host docker
    • Updated Conformance Certification

    Contributors

    Thanks again to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @aledbf
    • @amwat
    • @Anuj2512
    • @aojea
    • @BenTheElder
    • @boddumanohar
    • @chuckha
    • @darkowlzz
    • @eightlimbed
    • @gurjeet
    • @JamesLaverack
    • @jroper
    • @k8s-ci-robot
    • @liggitt
    • @longkb
    • @maelvls
    • @narendrakangralkar
    • @oke-py
    • @panzhc
    • @rajalokan
    • @ryanhartje
    • @sayboras
    • @slimm609
    • @stevesloka
    • @tao12345666333
    • @zegl
    • @zhlhahaha
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(9.54 MB)
    kind-linux-amd64(9.66 MB)
    kind-linux-arm(8.62 MB)
    kind-linux-arm64(9.26 MB)
    kind-linux-ppc64le(9.39 MB)
    kind-windows-amd64(9.31 MB)

  • v0.7.0(Jan 14, 2020)

    v0.7.0 notably brings greatly improved support for dynamic PersistentVolumes by way of rancher.io/localhost-path integration for clusters with Kubernetes v1.12.X or greater.

    It is also worth noting that kind is now committed to IPv6 PR testing and running 80% of the GCE presubmit test cases on Kubernetes Pull Requests.

    Breaking Changes

    • The default node image is a Kubernetes v1.17.0 image kindest/node:v1.17.0@sha256:9512edae126da271b66b990b6fff768fbb7cd786c7d39e86bdf55906352fdf62
    • Node images built with kind v0.7.0 have many improvements and require kind v0.5.0+, images built with older releases should continue to work.
      • To get the new dynamic PV support you must upgrade both kind and your node image(s) to this release or newer. The images are compatible with older releases but only both a newer kind CLI and newer images together will enable the updated PV support.
      • When the new PV support enabled by these versions the default storage class is changed from the previous k8s.io/host-path based implementation.

    New Features

    • Dynamic PersistentVolume Support is greatly enhanced with rancher.io/localhost-path integration out of the box
    • Upgraded dependencies around the board, notably including fixes to the CNI portmap plugin to reduce flakiness
    • Documentation Enhancements
      • We have started an effort to comprehensively cover configuration in the docs
      • We have a dedicated guide for using Ingress
      • Added more resources
      • Improved site code, in particular added copyable snippets
      • Other assorted smaller improvements
    • Again show one box emoji per node being provisioned

    New Node have been Images for kind v0.7.0, please use these exact images (IE like kindest/node:v1.16.4@sha256:b91a2c2317a000f3a783489dfb755064177dbc3a0b2f4147d50f04825d016f55) or build your own as we may need to change the image format again in the future :sweat_smile:

    Kubernetes Version @ digest

    • v1.18.0 @ sha256:0e20578828edd939d25eb98496a685c76c98d54084932f76069f886ec315d694
    • v1.17.0 @ sha256:9512edae126da271b66b990b6fff768fbb7cd786c7d39e86bdf55906352fdf62
    • v1.16.4 @ sha256:b91a2c2317a000f3a783489dfb755064177dbc3a0b2f4147d50f04825d016f55
    • v1.15.7 @ sha256:e2df133f80ef633c53c0200114fce2ed5e1f6947477dbc83261a6a921169488d
    • v1.14.10 @ sha256:81ae5a3237c779efc4dda43cc81c696f88a194abcc4f8fa34f86cf674aa14977
    • v1.13.12 @sha256:5e8ae1a4e39f3d151d420ef912e18368745a2ede6d20ea87506920cd947a7e3a
    • v1.12.10 @ sha256:68a6581f64b54994b824708286fafc37f1227b7b54cbb8865182ce1e036ed1cc
    • v1.11.10 @ sha256:e6f3dade95b7cb74081c5b9f3291aaaa6026a90a977e0b990778b6adc9ea6248

    Fixes

    • Improved messaging in a number of user facing errors
    • Disabled GINKGO_FLAKE_ATTEMPTS when testing
    • Fixed local image check
    • When the default kubeconfig is empty / non-existent, kind create cluster and kind export kubeconfig will preserve extra fields from kubeadm, notably apiVersion and kind (though they are not strictly necessary...)
    • Reduced cruft in base image

    Contributors

    Thanks again to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @amwat
    • @aojea
    • @BenTheElder
    • @danielhelfand
    • @darkowlzz
    • @daxmc99
    • @dustinspecker
    • @HeavyWombat
    • @jpreese
    • @k8s-ci-robot
    • @kandros
    • @kezmorris
    • @longkb
    • @marcindulak
    • @MasayaAoyama
    • @matti
    • @truongnh1992
    • @ytsarev
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(8.81 MB)
    kind-linux-amd64(8.92 MB)
    kind-linux-arm(7.90 MB)
    kind-linux-arm64(8.73 MB)
    kind-linux-ppc64le(8.80 MB)
    kind-windows-amd64(9.01 MB)

  • v0.6.1(Dec 4, 2019)

    This is a small patch release over v0.6.0.

    Notably:

    • Relaxed protocol and propagation in v1alpha4 config to have defaulting like v1alpha3 without specifying a value.
    • Fixed containerd config patching on clusters with multiple control plane nodes.
    • Fixed containerd config patching on slow hosts (do not attempt to restart containerd if it has not started yet).
    • Fixed airgapped support for node images (corrected kindnetd image preloading).

    This last point brings a new list of node images, fully compatible with v0.6.0, but with kindnetd properly pre-loaded onto them for use in air-gapped / offline environments:

    • kindest/node:v1.16.3@sha256:70ce6ce09bee5c34ab14aec2b84d6edb260473a60638b1b095470a3a0f95ebec
    • kindest/node:v1.15.6@sha256:18c4ab6b61c991c249d29df778e651f443ac4bcd4e6bdd37e0c83c0d33eaae78
    • kindest/node:v1.14.9@sha256:bdd3731588fa3ce8f66c7c22f25351362428964b6bca13048659f68b9e665b72
    • kindest/node:v1.13.12@sha256:1fe072c080ee129a2a440956a65925ab3bbd1227cf154e2ade145b8e59a584ad
    • kindest/node:v1.12.10@sha256:c5aeca1433e3230e6c1a96b5e1cd79c90139fd80242189b370a3248a05d77118
    • kindest/node:v1.11.10@sha256:8ebe805201da0a988ee9bbcc2de2ac0031f9264ac24cf2a598774f1e7b324fe1

    The default node image is now kindest/node:v1.16.3@sha256:70ce6ce09bee5c34ab14aec2b84d6edb260473a60638b1b095470a3a0f95ebec

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(9.14 MB)
    kind-linux-amd64(9.30 MB)
    kind-linux-arm(8.24 MB)
    kind-linux-arm64(9.07 MB)
    kind-linux-ppc64le(9.20 MB)
    kind-windows-amd64(8.99 MB)

  • v0.6.0(Nov 18, 2019)

    v0.6.0 brings major internal rework and some important breaking changes that we hope will make kind easier to use.

    It is also worth noting that kind is now used for blocking Kubernetes pull request testing, in addition to already being used for release signal. This is a major milestone for the project.

    Breaking Changes

    • KUBECONFIG is now automatically merged in the same style as kops, minikube, GKE, etc. Please see #1060 for a detailed migration guide.

      • You do not need to and should not do export KUBECONFIG="$(kind get kubeconfig-path)". This command is deprecated and will be removed in a future release.
      • kind will set your current-context when creating a cluster
      • You can configure using a different file by either setting the KUBECONFIG environment variable before creating a cluster, or by setting the --kubeconfig flag on cluster creation

    • Config files are now parsed in strict mode, which should help catch invalid config files

    • kind is moving all external (labels, config) namespaced objects into x-k8s.io to comply with the guidelines for API review. For this release kind will support and use both labels without breaking anything, however in a future release we will only use the new x-k8s.io labels. This note is primarily for @danderson 🙃

    • The default node image is a Kubernetes v1.16.3 image kindest/node:v1.16.3@sha256:bced4bc71380b59873ea3917afe9fb35b00e174d22f50c7cab9188eac2b0fb88

    • Node images built with kind v0.6.0 have many improvements and require kind v0.5.0+, images built with v0.4.0 / v0.3.0 should continue to work with v0.6.0 however.

    New Features

    • There is a new kind.x-k8s.io/v1alpha4 config format.
      • Support for per-node kubeadm config patches
      • Support for patching the kind nodes's containerd config, which will be used in an upcoming new guide for local registries
    • Automatic KUBECONFIG export and merge to simplify using kind.
      • kind clusters will always have a kind- prefix in kubeconfig file entries and now can be identified by this.
    • kind export kubeconfig to re-export credentials in the way kind create cluster now does.
    • Significant logging rework
      • -q / --quiet flag to silence status messages / error output
      • Logging with -v 1 / --verbosity 1 or greater will result in detailed error output including stacktraces and command output
    • Colored terminal output and slightly better spinners.
      • You can disable both by setting the environment variable TERM to dumb
    • Enhanced kind version output with more details
      • Builds with make build or make install include git commit
      • The simplified version can be obtained with kind version -q
    • Mounts in extraMounts may have relative paths on the host, resolved against the working directory of kind
    • kubeadm config patches no longer need to include metadata.name and may optionally exclude the apiVersion, allowing more flexible matching of patches to config objects
      • The kind binary is now 75% smaller as a result of eliminating dependencies and writing a patch runtime specific to our needs.
    • kindnetd now has improved logging with klog
    • A random salutation is printed at the end of kind create cluster 🙃
    • It should be possible to install kind via chocolatey with choco install kind. We're also hoping to get a homebrew package shortly!

    New Node have been Images for kind v0.5.0, please use these exact images (IE like kindest/node:v1.16.3@sha256:bced4bc71380b59873ea3917afe9fb35b00e174d22f50c7cab9188eac2b0fb88) or build your own as we may need to change the image format again in the future :sweat_smile:

    • v1.16.3 sha256:bced4bc71380b59873ea3917afe9fb35b00e174d22f50c7cab9188eac2b0fb88
    • v1.15.6 sha256:1c8ceac6e6b48ea74cecae732e6ef108bc7864d8eca8d211d6efb58d6566c40a
    • v1.14.9 sha256:00fb7d424076ed07c157eedaa3dd36bc478384c6d7635c5755746f151359320f
    • v1.13.12 sha256:ad1dd06aca2b85601f882ba1df4fdc03d5a57b304652d0e81476580310ba6289
    • v1.12.10 sha256:e93e70143f22856bd652f03da880bfc70902b736750f0a68e5e66d70de236e40
    • v1.11.10 sha256:44e1023d3a42281c69c255958e09264b5ac787c20a7b95caf2d23f8d8f3746f2

    The new node images for recent Kubernetes versions should be > 200 MB smaller thanks to a KEP we implemented upstream to allow building without most of the in-tree cloud provider code.

    https://github.com/kubernetes/enhancements/blob/master/keps/sig-cloud-provider/20190729-building-without-in-tree-providers.md

    Fixes

    • Various kind CLI output was corrected to be to stdout or stderr where appropriate
    • We have robust new containerd nightly build infrastructure and are shipping the lastest upstream containerd 1.3 + backports in all node images
    • Removed unnecessary dependencies and upgraded all dependencies
    • Fixed building Kubernetes master branch with Bazel
    • Upgraded to build with Go 1.13.4
    • Upgraded all dependencies
    • kindnetd now mounts the iptables xtables lock to ensure coordination with kube-proxy
    • kind will attempt to recreate the original cgroup environment inside the node, this eliminates some error output from kubelet
    • fixed a race condition in command execution, eliminating a source of flakiness of some kind commands
    • kind get kubeconfig does not depend on an existing config file on the host
    • the product_uuid is now faked to be different per-node
    • switched to GA storageclass annotation on the default storage class
      • removed unnecessary addon manager annotation
    • kindnetd is now more robust with backoff on failures
    • kind is now relatively decoupled from docker internally for node management
    • deflaked our cross build and fixed running it on macOS

    Contributors

    Thanks again to everyone who contributed to this release! ❤️

    Users whose commits are in this release (alphabetically by user name)

    • @alejandrox1
    • @amwat
    • @aojea
    • @beautytiger
    • @BenTheElder
    • @cofyc
    • @danderson
    • @dustinspecker
    • @fabiand
    • @hwdef
    • @jbrette
    • @jieyu
    • @k8s-ci-robot
    • @luksa
    • @lwr20
    • @maelvls
    • @manics
    • @mauilion
    • @paol
    • @PatrickLang
    • @RafalKorepta
    • @robertojrojas
    • @sivanzcw
    • @sufuf3
    • @tanjunchen
    • @tao12345666333
    • @u5surf
    • @WalkerGriggs
    • @ysz
    • @yuxiaobo96
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(9.14 MB)
    kind-linux-amd64(9.29 MB)
    kind-linux-arm(8.24 MB)
    kind-linux-arm64(9.07 MB)
    kind-linux-ppc64le(9.20 MB)
    kind-windows-amd64(8.99 MB)

  • v0.5.1(Aug 21, 2019)

    This release is a minor patch to upgrade kustomize to v3.1.1-0.20190821175718-4b67a6de1296, fixing builds for Windows. This release also contains fixes to our release tooling & CI to ensure we don't regress on this.

    Otherwise, please see the release notes for v0.5.0.

    See: https://github.com/kubernetes-sigs/kind/issues/792 for more details on what went wrong and how we fixed it.

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(36.44 MB)
    kind-linux-amd64(39.18 MB)
    kind-linux-arm(34.09 MB)
    kind-linux-arm64(37.82 MB)
    kind-linux-ppc64le(38.07 MB)
    kind-windows-amd64(36.63 MB)

  • v0.5.0(Aug 20, 2019)

    Please see v0.5.1 for a small update with Windows fixes (!)

    v0.5.0 primarily brings bug fixes and documentation + code quality improvements

    Breaking Changes

    • The default node image is a Kubernetes v1.15.3 image kindest/node:v1.15.3@sha256:27e388752544890482a86b90d8ac50fcfa63a2e8656a96ec5337b902ec8e5157
    • Node images built with kind v0.5.0 will require kind v0.5.0+, images built with v0.4.0 / v0.3.0 should continue to work with v0.5.0 however.

    New Features

    • Port forwards now support UDP and SCTP
    • We now have documentation for using kind with WSL2 (shoutout to @PatrickLang)
    • We now have a resources page where you can find talks, guides, and examples
    • Limited arm32 support
    • Nodes perform some more startup logging to the container TTY, which should help with debugging
    • Building node images should be slightly faster due to streamlined image pre-loading

    New Node have been Images for kind v0.5.0, please use these exact images or build your own as we may need to change the image format again in the future :sweat_smile:

    • kindest/node:v1.15.3@sha256:27e388752544890482a86b90d8ac50fcfa63a2e8656a96ec5337b902ec8e5157
    • kindest/node:v1.14.6@sha256:464a43f5cf6ad442f100b0ca881a3acae37af069d5f96849c1d06ced2870888d
    • kindest/node:v1.13.10@sha256:2f5f882a6d0527a2284d29042f3a6a07402e1699d792d0d5a9b9a48ef155fa2a
    • kindest/node:v1.12.10@sha256:e43003c6714cc5a9ba7cf1137df3a3b52ada5c3f2c77f8c94a4d73c82b64f6f3
    • kindest/node:v1.11.10@sha256:bb22258625199ba5e47fb17a8a8a7601e536cd03456b42c1ee32672302b1f909

    Fixes

    • Worked around Kubernetes build changes pending in v1.16, kind again supports building Kubernetes at HEAD
    • kind load docker-image now correctly avoids loading images that are already loaded :warning:
    • More Node runtime written storage is now on volume(s), this should fix some bugs (#771) and improve performance
    • New node images will attempt to work around missing /dev/kmsg causing kubelet to not start
    • / is rshared in new node (images) for mount propagation (more mounts should be shared now)
    • LimitNOFILE in the containerd service now matches the updated upstream value (1048576) in new node images
    • All node level services should be passed proxy environment variables by default now with new node images
    • Upgraded to build with Go 1.12.9
    • Upgraded Kustomize to v3.1.0
    • Bugs in the documentation for using private registries were corrected
    • The kind CLI's own build scripts support proxy environment variables properly
    • We've improved / clarified the roadmap a bit

    Contributors

    Thanks again to everyone who contributed to this release! :smile:

    Users whose commits are in this release (alphabetically by user name)

    • @alfsuse
    • @aojea
    • @barkbay
    • @BenTheElder
    • @bonifaido
    • @electrofelix
    • @ethan-daocloud
    • @jbarrick-mesosphere
    • @k8s-ci-robot
    • @munnerz
    • @PatrickLang
    • @tao12345666333
    • @WalkerGriggs
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(36.42 MB)
    kind-linux-amd64(39.16 MB)
    kind-linux-arm(34.09 MB)
    kind-linux-arm64(37.69 MB)
    kind-linux-ppc64le(38.07 MB)

  • v0.4.0(Jun 26, 2019)

    v0.4.0 brings improved networking features in particular, including initial IPv6 support and additional node port forwards. It also continues to improve speed and reliability (hopefully! :upside_down_face:)

    Breaking Changes

    • The default node image is a Kubernetes v1.15.0 image kindest/node:v1.15.0@sha256:b4d092fd2b507843dd096fe6c85d06a27a0cbd740a0b32a880fe61aba24bb478
    • The deprecated kind.sigs.k8s.io/v1alpha2 version of config was removed, please switch to kind.sigs.k8s.io/v1alpha3
    • kind build node-image --type=apt was removed. Please use kind build node-image or kind build node-image --type=bazel instead, or one of the pre-built images. A future release will re-work these commands and add support for building from upstream release tarballs.

    New Features

    • Additional node port forwards may be configured with the nodes[].extraPortMappings of Cluster configuration in kind.sigs.k8s.io/v1alpha3. This can be used to access workloads more easily from the host. Expect guides using this soon!
    • Limited IPv6 support. The networking.ipFamily may be set to ipv6 in kind.sigs.k8s.io/v1alpha3 Cluster configuration to create an ipv6 enabled cluster on Linux. We have set up continuous Kubernetes conformance testing with IPv6 enabled and are working to fix the tests
    • A warning is emitted if the chosen cluster name is too long and likely to cause the generated node names to be too long.
    • make install now supports overriding INSTALL to specify an alternate tool to install and ensures the output directory
    • Support / documentation for using kind on WSL2.
    • The kind load ... sub-commands now avoid loading images that are already present
    • kubeadm v1beta2 config is now use opportunistically for recent enough Kubernetes versions
    • Reduced startup time for single-node clusters in particular

    New Node have been Images for kind v0.4.0, please use these exact images or build your own as we may need to change the image format again in the future :sweat_smile:

    • kindest/node:v1.15.0@sha256:b4d092fd2b507843dd096fe6c85d06a27a0cbd740a0b32a880fe61aba24bb478
    • kindest/node:v1.14.3@sha256:583166c121482848cd6509fbac525dd62d503c52a84ff45c338ee7e8b5cfe114
    • kindest/node:v1.13.7@sha256:f3f1cfc2318d1eb88d91253a9c5fa45f6e9121b6b1e65aea6c7ef59f1549aaaf
    • kindest/node:v1.12.9@sha256:bcb79eb3cd6550c1ba9584ce57c832dcd6e442913678d2785307a7ad9addc029
    • kindest/node:v1.11.10@sha256:176845d919899daef63d0dbd1cf62f79902c38b8d2a86e5fa041e491ab795d33

    Fixes

    • The generated KUBECONFIG file references the API Server by listen address (IP) instead of localhost (domain), which is more correct and should work on systems without a valid localhost entry (!)
    • The new kind images should work on ipv4 only hosts with ipv6 fully disabled
    • The node subnet is properly added included in NO_PROXY on the nodes when proxy settings are detected
    • Clusters with multiple control plane nodes now have proper healthchecks of the API servers in the external loadbalancer
    • Minor cleanup and typo fixes

    Contributors

    Thanks again to everyone who committed to this release! You all are the best! :heart:

    Alphabetically by user name:

    • @alisondy
    • @amwat
    • @aojea
    • @BenTheElder
    • @fabriziopandini
    • @fllaca
    • @jieyu
    • @k8s-ci-robot
    • @kaspernissen
    • @mistydemeo
    • @PatrickLang
    • @praseodym
    • @xhzhf
    • @yeya24
    • @ytinirt
    • @zegl
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(34.35 MB)
    kind-linux-amd64(34.58 MB)
    kind-linux-arm64(33.28 MB)
    kind-linux-ppc64le(33.54 MB)
    kind-windows-amd64(34.40 MB)

  • v0.3.0(May 17, 2019)

    This release focused on improving the speed and reliability of cluster creation, networking, and bug fixes. A number of breaking changes were required, but Nodes should be simpler, smaller, faster, and more reliable.

    Breaking Changes

    IMPORTANT: Breaking Image Changes

    kind v0.3+ requires new node images versus v0.2.1 and earlier. A number of important internal improvements were made that required changing how cluster bootup is performed. The following images we have previously published are incompatible with kind v0.3+:

    • kindest/node:v1.11.3
    • kindest/node:v1.12.2
    • kindest/node:v1.12.3
    • kindest/node:v1.13.2
    • kindest/node:v1.13.4
    • kindest/node:v1.14.0

    The following images require unreleased versions of kind between v0.3.0 and v0.2.1, please do not use them:

    • kindest/node:v1.14.1

    For 0.3+ we are pushing the following images:

    • kindest/nodev1.14.2@sha256:33539d830a6cf20e3e0a75d0c46a4e94730d78c7375435e6b49833d81448c319
    • kindest/node:v1.13.6@sha256:9e07014fb48c746deb98ec8aafd58c3918622eca6063e643c6e6d86c86e170b4
    • kindest/node:v1.12.8@sha256:cc6e1a928a85c14b52e32ea97a198393fb68097f14c4d4c454a8a3bc1d8d486c
    • kindest/node:v1.11.10@sha256:abd0275ead5ddfd477b7bc491f71957d7dd75408a346834ffe8a9bee5fbdc15b

    We hope not to make further breaking changes to the images, but strongly recommend pinning images by sha256, the default image is pinned to a particular sha. This can be done with --image=kindest/node:v1.X.Y@sha256:hash. For example: kind create cluster --image=kindest/node:v1.12.8@sha256:cc6e1a928a85c14b52e32ea97a198393fb68097f14c4d4c454a8a3bc1d8d486c

    • Default clusters will now be Kubernetes v1.14.2, the default node image is now kindest/nodev1.14.2@sha256:33539d830a6cf20e3e0a75d0c46a4e94730d78c7375435e6b49833d81448c319

    • Kubernetes on the nodes now uses CRI (containerd) instead of dockershim (dockerd) inside the "node" docker containers. Please note that details underlying Kubernetes are not guaranteed and may change again in the future, however we are still warning about this particular change as some advanced users depended on these details anyhow.

    • Similarly, the default CNI / networking setup changed, kind now sets a podSubnet and masquerades outbound pod traffic with a new CNI setup. This subnet is defaulted but is now configurable and the entire CNI setup can be disabled (see features below).

    • Building kind now requires go modules (using any upstream supported go version) or using the new Makefile. Pre-built binaries for all platforms are included with this release. Please consider using one of the binaries in your CI setups, or at least pinning to a tagged release.

    New Features

    • The default CNI can be disabled with the Cluster.networking.disableDefaultCNI config field, making it easier to experiment with your own CNI.
    • The Pod Subnet is now configurable with Cluster.networking.podSubnet
    • kind completion bash is now supported to source shell completions.
      • WARNING: we are considering renaming this to align better with the [verb] [noun] Kubernetes style CLI structure.
    • Limited PPC64LE support (requires building your own images)
    • The kind command line binary is now built reproducibly
      • building now only requires make / docker on the host as an alternative to go 1.11+ with modules
    • kind now runs a minimal and lightweight CNI setup
    • internal progress on IPv6 support
    • internal progress on cluster reboot support
    • kind is now versioned with go modules

    Fixes

    • We've improved auto-detecting the kubernetes source location for kind build node-image
    • Cluster bootup should be even faster and more reliable in constrained environments.
      • simplified startup logic with less waiting
      • Kubernetes images are fully preloaded at build time
      • our networking setup is now somewhat custom, and is simpler and lighter
    • kind export logs should be less likley to flake with large numbers of nodes or log files, internally we've combined most of the streams.
    • More improvements to proxy handling, though kind may still not work correctly in proxied environments.
    • /sys/class/dmi/id/product_name is faked to kind where applicable, this avoids Kubernetes detecting if kind is running on GCE.

    Contributors

    Thanks again to everyone who committed to this release! :heart:

    Alphabetically by user name:

    • @alejandrox1
    • @ameukam
    • @aojea
    • @BenTheElder
    • @dims
    • @fiunchinho
    • @fstab
    • @k8s-ci-robot
    • @mkumatag
    • @neolit123
    • @paranoidaditya
    • @sohankunkerkar
    • @steffengy
    • @tao12345666333
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(34.35 MB)
    kind-linux-amd64(34.56 MB)
    kind-linux-arm64(33.22 MB)
    kind-linux-ppc64le(33.55 MB)
    kind-windows-amd64(34.40 MB)

  • 0.2.1(Mar 28, 2019)

    0.2.1 is a bug fix release

    Breaking Changes

    NONE

    New Features

    • The hostpath provisioner is now enabled by default (#397)

    Fixes

    • fix kind build node-image on macOS, previously --type=bazel and --type=docker (the default) did not work properly on not-Linux in 0.2.0 (#413)
    • fix possible panic in failed kind create cluster calls, previously if multiple nodes failed to come up kind could panic (#407)

    Contributors

    Thanks to everyone who committed to this release! ❤️

    • @BenTheElder
    • @joejulian
    • @k8s-ci-robot
    • @neolit123
    • @akutz
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(33.48 MB)
    kind-linux-amd64(33.69 MB)
    kind-linux-arm64(32.42 MB)
    kind-windows-amd64(33.53 MB)

  • 0.2.0(Mar 20, 2019)

    Breaking Changes

    • Default clusters will now be Kubernetes v1.13.4, the default node image is now kindest/node:v1.13.4
    • v1alpha1 config is no longer suppported, please upgrade to v1alpha3 (v1alpha2 is also suppported in this release)
    • v1alpha3 is now the default config version
      • The Config type was renamed to Cluster
      • 127.0.0.1 is the default address for binding the API server port to the host, if you wish to use 0.0.0.0 please set this in the networking section of Cluster config in v1alpha3
      • replicas is no longer a field on nodes, given the low node count, please write one entry per node
      • kubeadmConfigPatches and kubeadmConfigPatchesJSON6902 have moved to the Cluster level from nodes (though they were applied cluster-wide previously, and only from the bootstrap control plane node)

    New Features

    • Cluster bootup should be faster on Linux in particular, many tasks are now handled in parallel
    • The CNI (weave) is bundled in the node-image, clusters should work without internet access if kindest/node is pulled ahead of time
    • support for multiple-control planes with an external-load balancer, I.E. """HA""" clusters
    • v1alpha3 now supports cluster-wide networking options, which allow for setting the API server listen address and port on the host
    • proxy support is improved
      • http_proxy, https_proxy, and no_proxy are supported instead of their uppercase equivilants if the uppercase values are not set
      • the nodes's docker daemons will be configured with proxy settings from the host (thanks
    • Paths on the host may be bind-mounted to nodes using the new extraMounts field on nodes, to E.G. share a directory with the host and a node
    • We have a new documentation site https://kind.sigs.k8s.io/ :tada:
      • we have a logo now
    • kind is CNCF certified conformant
    • kind should work on ARM, though kubernetes images will need to be built locally for now
    • kind load docker-image and kind load image-archive are now supported for side-loading images into your cluster
    • kubeadm output from init and join will be printed when the kind loglevel is debug
    • kind is now developed with go modules instead of dep
    • Improved usage output for kind create cluster on Windows

    Fixes

    • CNI was upgraded to 0.7.5 in the base image
    • Docker was upgraded to 18.06.3 with patched runc
    • kubelet disk eviction and image GC are disabled for kind clusters, this prevents various failures when the host has low disk space. As kind clusters do not run on dedicated hosts, we configure Kubernetes not to reserve disk overhead.
    • Docker image archives are chowned to root:root in the node image, fixing user namespace remapping support
    • No emoji are re-used in CLI output, every action now has a unique emoji

    Contributors

    Thanks to everyone who committed to this release! :heart:

    Alphabetically by user name:

    • @alejandrox1
    • @ameukam
    • @aojea
    • @bart0sh
    • @BenTheElder
    • @devaii
    • @dims
    • @dlespiau
    • @fabriziopandini
    • @idealhack
    • @k8s-ci-robot
    • @Katharine
    • @lucperkins
    • @mitar
    • @munnerz
    • @neolit123
    • @pablochacin
    • @phisco
    • @RafalKorepta
    • @rimusz
    • @rlenferink
    • @steffengy
    • @stevesloka
    • @tao12345666333
    • @vasrem
    • @ZachEddy
    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(33.46 MB)
    kind-linux-amd64(33.68 MB)
    kind-linux-arm64(32.43 MB)
    kind-windows-amd64(33.52 MB)

  • 0.1.0(Jan 16, 2019)

    This release contains many exciting features! Amongst other things we now have support for multi-node clusters thanks to @fabriziopandini, and fixes have been made for kind on Windows.

    UPDATE 2019-02-11

    Due to CVE-2019-5736 ("the runc escape")

    kindest/node:v1.13.2@sha256:d070e091e7c0e515a37d0834ea72828b2338acdc4bc7a13cbb84274fb14e5e83 or: kindest/node:v1.13.3@sha256:d1af504f20f3450ccb7aed63b67ec61c156f9ed3e8b0d973b3dee3c95991753c

    Should be used instead of the default image in this release. Example:

    kind create cluster --image=kindest/node:v1.13.3@sha256:d1af504f20f3450ccb7aed63b67ec61c156f9ed3e8b0d973b3dee3c95991753c

    In the future we are looking at not pinning to an exact image by default, so that minimal fixes can be silently pushed. We are additionally hoping to adopt rootless containers once support lands in Kubernetes.

    Breaking changes since 0.1.0

    • Default clusters will now be Kubernetes v1.13.2, and the default node image is now kindest/node:v1.13.2@sha256:e14edfa4950e009fe560499c9db6e89daae8bd18bcb372caca6d321a86c52cda
    • nodeLifecycleHooks are no longer supported in the config, we believe this feature was unused and will revisit it later, if you need this, please file an issue with your use case!
    • Deleting a cluster now attempts to delete the cluster specific KUBECONFIG file on the host (thanks @blakestoddard!)

    New Features

    • The current kind Config is now at kind.sigs.k8s.io/v1alpha2, with support for multi-node clusters! (thanks @fabriziopandini!)
    • The default image was upgraded to Kubernetes v1.13.2 (see above)
    • kind should now work on hosts where userns remap is enabled in the docker daemon (thanks @vasrem!)
    • Improved documentation (thanks @alejandrox1, @jimangel, @tao12345666333, @k8k, @timoreimann, @chuckha, @superbrothers, @benmoss!)
    • kubeadm's config is now internally v1beta1 when new enough Kubernetes images are used, this should improve stable support for future Kubernetes versions. Additionally all config objects are populated, so they may be patched via patches in kind's config.
    • When removing a KUBECONFIG file, we attempt to detect if the environment variable is set and warn about unsetting it (thanks @blakestoddard, @manparvesh!)
    • The base image now contains conntrack, in order to support installing recent kubeadm packages. by default it is now kindest/base:v20181203-d055041
    • kind build node-image now supports a --kube-root flag to explicitly specify the path to Kubernetes's source code, if unset kind will continue to auto-detect this via go's build tooling. (thanks @inercia!)

    Fixes

    • On Windows the KUBECONFIG files should now be placed in the default directory correctly (we now use the homedir util from client-go)
    • On Windows patching kubeadm config should work correctly
    • Recent kubeadm debian packages should be installable in the base image after adding conntrack (see above)

    Committers to this release are:

    • @BenTheElder
    • @fabriziopandini
    • @kris-nova
    • @blakestoddard
    • @jimangel
    • @tao12345666333
    • @alejandrox1
    • @manparvesh
    • @inercia
    • @superbrothers
    • @chuckha
    • @timoreimann
    • @k8k
    • @vincepri
    • @k8s-ci-robot

    Thank you for your contributions! Additionally, special thanks to @neolit123, @krzyzacy, @munnerz, and @amwat for PR reviews!

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(36.00 MB)
    kind-linux-amd64(36.29 MB)
    kind-windows-amd64(36.13 MB)

  • 0.0.1(Nov 29, 2018)

    UPDATE 2019-02-11

    Due to CVE-2019-5736 ("the runc escape")

    kindest/node:v1.12.5@sha256:2cc6e8153dbe32db0b962cda060e193615951ae8d2a0b808535726a9b6b5e583 or kindest/node:v1.13.2@sha256:d070e091e7c0e515a37d0834ea72828b2338acdc4bc7a13cbb84274fb14e5e83 or: kindest/node:v1.13.3@sha256:d1af504f20f3450ccb7aed63b67ec61c156f9ed3e8b0d973b3dee3c95991753c

    Should be used instead of the default image in this release. Example:

    kind create cluster --image=kindest/node:v1.13.3@sha256:d1af504f20f3450ccb7aed63b67ec61c156f9ed3e8b0d973b3dee3c95991753c

    In the future we are looking at not pinning to an exact image by default, so that minimal fixes can be silently pushed. We are additionally hoping to adopt rootless containers once support lands in Kubernetes.

    Please consider upgrading to kind 0.1.0 or newer as well.

    UPDATE: Kubernetes v1.12.2 is affected by CVE-2018-1002105; when using kind create cluster please set the image to a non-default image via the --image flag. The following Kubernetes v1.12.3 image should be suitable: kindest/node:v1.12.3@sha256:f0ecb1066697d9417365ca58410132e512ce2010763470bb28c1e8f7fef55464

    A patch release will be made to upgrade this default, see https://github.com/kubernetes-sigs/kind/issues/180.

    kind is still alpha-grade software, and as such breaking changes will be made to future releases.

    This release allows early users to pin to these pre-built binaries and avoid these changes until they intend to upgrade, rather than installing from HEAD with go get ...

    NOTE: the default kind create cluster pins to a specific node image in order to avoid breakage there.

    This image is: kindest/node:v1.12.2@sha256:6ac1dc1750fc0efd13d4e294115f9012a21282957e4380a5535bd32154193d4d on the current official registry.

    Contributors to this release are:

    • @BenTheElder
    • @munnerz
    • @alejandrox1
    • @tao12345666333
    • @neolit123
    • @fabriziopandini
    • @Lion-Wei
    • @mooncak
    • @radu-matei
    • @AdamDang
    • @cblecker

    Thank you for your contributions!

    Source code(tar.gz)
    Source code(zip)
    kind-darwin-amd64(35.77 MB)
    kind-linux-amd64(36.07 MB)
Owner
Kubernetes SIGs
Org for Kubernetes SIG-related work
Kubernetes SIGs
GitHub https://kind.sigs.k8s.io/
KinK is a helper CLI that facilitates to manage KinD clusters as Kubernetes pods. Designed to ease clusters up for fast testing with batteries included in mind.

kink A helper CLI that facilitates to manage KinD clusters as Kubernetes pods. Table of Contents kink (KinD in Kubernetes) Introduction How it works ?

Trendyol Open Source 362 Dec 10, 2022
PolarDB Stack is a DBaaS implementation for PolarDB-for-Postgres, as an operator creates and manages PolarDB/PostgreSQL clusters running in Kubernetes. It provides re-construct, failover swtich-over, scale up/out, high-available capabilities for each clusters.

PolarDB Stack开源版生命周期 1 系统概述 PolarDB是阿里云自研的云原生关系型数据库，采用了基于Shared-Storage的存储计算分离架构。数据库由传统的Share-Nothing，转变成了Shared-Storage架构。由原来的N份计算+N份存储，转变成了N份计算+1份存储

null 23 Nov 8, 2022
vcluster - Create fully functional virtual Kubernetes clusters - Each cluster runs inside a Kubernetes namespace and can be started within seconds

Website • Quickstart • Documentation • Blog • Twitter • Slack vcluster - Virtual Clusters For Kubernetes Lightweight & Low-Overhead - Based on k3s, bu

Loft Labs 2.3k Jan 4, 2023
provider-kubernetes is a Crossplane Provider that enables deployment and management of arbitrary Kubernetes objects on clusters

provider-kubernetes provider-kubernetes is a Crossplane Provider that enables deployment and management of arbitrary Kubernetes objects on clusters ty

International Business Machines 2 Dec 14, 2022
Crossplane provider to provision and manage Kubernetes objects on (remote) Kubernetes clusters.

provider-kubernetes provider-kubernetes is a Crossplane Provider that enables deployment and management of arbitrary Kubernetes objects on clusters ty

Crossplane Contrib 69 Jan 3, 2023
Dotnet-appsettings-env - Convert .NET appsettings.json file to Kubernetes, Docker and Docker-Compose environment variables

dotnet-appsettings-env Convert .NET appsettings.json file to Kubernetes, Docker

Daniel Dias de Assumpção 18 Dec 30, 2022
Dotnet-appsettings-env - Convert .NET appsettings.json file to Kubernetes, Docker and Docker-Compose environment variables

dotnet-appsettings-env Convert .NET appsettings.json file to Kubernetes, Docker

Daniel Dias de Assumpção 1 Feb 16, 2022
Local Storage is one of HwameiStor components. It will provision the local LVM volume.

Local Storage Module English | Simplified_Chinese Introduction Local Storage is one of modules of HwameiStor which is a cloud native local storage sys

HwameiStor 165 Aug 6, 2022
WIP - Pin - local pipeline project with Docker Golang API.

pin ?? WIP - Local pipeline project with Docker Golang API. ?? Installation Download latest release You can download latest release from here Install

Muhammed İKİNCİ 12 May 28, 2022
kubequery is a Osquery extension that provides SQL based analytics for Kubernetes clusters

kubequery powered by Osquery kubequery is a Osquery extension that provides SQL based analytics for Kubernetes clusters kubequery will be packaged as

Uptycs Inc 88 Dec 27, 2022
🐶 Kubernetes CLI To Manage Your Clusters In Style!

K9s - Kubernetes CLI To Manage Your Clusters In Style! K9s provides a terminal UI to interact with your Kubernetes clusters. The aim of this project i

Fernand Galiana 19.2k Jan 9, 2023
Validation of best practices in your Kubernetes clusters

Best Practices for Kubernetes Workload Configuration Fairwinds' Polaris keeps your clusters sailing smoothly. It runs a variety of checks to ensure th

Fairwinds 2.8k Jan 9, 2023
Manage large fleets of Kubernetes clusters

Introduction Fleet is GitOps at scale. Fleet is designed to manage up to a million clusters. It's also lightweight enough that it works great for a si

Rancher 1.2k Dec 31, 2022
A best practices checker for Kubernetes clusters. 🤠

Clusterlint As clusters scale and become increasingly difficult to maintain, clusterlint helps operators conform to Kubernetes best practices around r

DigitalOcean 500 Dec 29, 2022
Kubernetes operator to autoscale Google's Cloud Bigtable clusters

Bigtable Autoscaler Operator Bigtable Autoscaler Operator is a Kubernetes Operator to autoscale the number of nodes of a Google Cloud Bigtable instanc

RD Station 22 Nov 5, 2021
Nebula Operator manages NebulaGraph clusters on Kubernetes and automates tasks related to operating a NebulaGraph cluster

Nebula Operator manages NebulaGraph clusters on Kubernetes and automates tasks related to operating a NebulaGraph cluster. It evolved from NebulaGraph Cloud Service, makes NebulaGraph a truly cloud-native database.

vesoft inc. 58 Dec 31, 2022
Client extension for interacting with Kubernetes clusters from your k6 tests.

⚠️ This is a proof of concept As this is a proof of concept, it won't be supported by the k6 team. It may also break in the future as xk6 evolves. USE

k6 25 Jan 2, 2023
Managing your Kubernetes clusters (including public, private, edge, etc) as easily as visiting the Internet

Clusternet Managing Your Clusters (including public, private, hybrid, edge, etc) as easily as Visiting the Internet. Clusternet (Cluster Internet) is

Clusternet 1.1k Dec 30, 2022
Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration, and automating updates to configuration when there is new code to deploy.

Flux is a tool for keeping Kubernetes clusters in sync with sources of configuration (like Git repositories), and automating updates to configuration when there is new code to deploy.

Flux project 4.3k Jan 8, 2023