Run Amazon EKS on your own infrastructure ๐Ÿš€

Overview

Amazon EKS Anywhere

Go Report Card

Conformance test status: BuildStatus

Amazon EKS Anywhere is a new deployment option for Amazon EKS that enables you to easily create and operate Kubernetes clusters on-premises with your own virtual machines. It brings a consistent AWS management experience to your data center, building on the strengths of Amazon EKS Distro, the same distribution of Kubernetes that powers EKS on AWS. Its goal is to include full lifecycle management of multiple Kubernetes clusters that are capable of operating completely independently of any AWS services.

Here are the steps for getting started with EKS Anywhere. Full documentation for releases can be found on https://anywhere.eks.amazonaws.com.

Development

The EKS Anywhere is tested using Prow, the Kubernetes CI system. EKS operates an installation of Prow, which is visible at https://prow.eks.amazonaws.com/. Please read our CONTRIBUTING guide before making a pull request. Refer to our end to end guide to run E2E tests locally.

The dependencies which make up EKS Anywhere are defined and built via the build-tooling repo. To update dependencies please review the Readme for the specific dependency before opening a PR.

Security

If you discover a potential security issue in this project, or think you may have discovered a security issue, we ask that you notify AWS Security via our vulnerability reporting page. Please do not create a public GitHub issue.

License

This project is licensed under the Apache-2.0 License.

Comments
  • eksctl plan cluster upgrade command reports incorrect current versions

    eksctl plan cluster upgrade command reports incorrect current versions

    What happened: when running eksctl anywhere upgrade plan cluster -f eksa-cluster.yaml the returned current versions are incorrect

    What you expected to happen: eksctl anywhere upgrade plan cluster -f eksa-cluster.yaml should report correct current and next versions

    How to reproduce it (as minimally and precisely as possible): run eksctl anywhere upgrade plan cluster -f eksa-cluster.yaml twice and you will see the 2nd run will report the same old versions under current

    Anything else we need to know?:

    $ eksctl anywhere upgrade plan cluster -f eksa-cluster.yaml
    Warning: VSphereDatacenterConfig configured in insecure mode
    Checking new release availability...
    NAME                 CURRENT VERSION   NEXT VERSION
    EKS-A                v0.10.1+c59d2cc   v0.11.2+cd46e1d
    Flux                 v0.29.4+e670ae4   v0.31.3+5738265
    cert-manager         v1.7.2+d4fa7e0    v1.8.2+543ab1d
    cluster-api          v1.1.3+d32ac08    v1.2.0+172e2ab
    kubeadm              v1.1.3+973f8a2    v1.2.0+525dbd6
    vsphere              v1.1.1+628bf01    v1.1.1+2fc04d0
    kubeadm              v1.1.3+9094d93    v1.2.0+115b0d5
    etcdadm-bootstrap    v1.0.3+bfb8d15    v1.0.5+77e4d45
    etcdadm-controller   v1.0.1+842bc86    v1.0.4+05b4294
    cilium               v1.10.11-eksa.2   v1.10.14-eksa.1
    $ flux version
    flux: v0.31.5
    helm-controller: v0.22.2-eks-a-16
    kustomize-controller: v0.26.3-eks-a-16
    notification-controller: v0.24.1-eks-a-16
    source-controller: v0.25.9-eks-a-16
    

    Environment:

    • EKS Anywhere Release: 0.11.2
    • EKS Distro Release: 1.23
    external ack team/cli area/cli 
    opened by echel0n 33
  • Adding token refresher to EKS-A release bundle

    Adding token refresher to EKS-A release bundle

    Signed-off-by: jonahjon [email protected]

    Current tests are failing since the new image isn't being pushed to Public ECR. This is adding it to the bundle.

    approved lgtm size/L needed-next-release 
    opened by jonahjon 25
  • Add Gloo Edge, and Gloo Mesh.

    Add Gloo Edge, and Gloo Mesh.

    Solo.io is an EKS anywhere partner.

    https://aws.amazon.com/eks/eks-anywhere/partners/

    Issue #, if available:

    Description of changes:

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    approved lgtm ok-to-test size/XS 
    opened by murphye 20
  • cilium not deployed

    cilium not deployed

    What happened:

    following https://anywhere.eks.amazonaws.com/docs/getting-started/local-environment/ to create local dev cluster, the cluster is up, but cilium is not deployed

    dev-cluster.yaml

    apiVersion: anywhere.eks.amazonaws.com/v1alpha1
    kind: Cluster
    metadata:
      name: dev-cluster
    spec:
      clusterNetwork:
        cni: cilium
        pods:
          cidrBlocks:
          - 10.0.0.0/8
        services:
          cidrBlocks:
          - 10.96.0.0/12
      controlPlaneConfiguration:
        count: 1
      datacenterRef:
        kind: DockerDatacenterConfig
        name: dev-cluster
      externalEtcdConfiguration:
        count: 1
      kubernetesVersion: "1.21"
      workerNodeGroupConfigurations:
      - count: 1
    
    ---
    apiVersion: anywhere.eks.amazonaws.com/v1alpha1
    kind: DockerDatacenterConfig
    metadata:
      name: dev-cluster
    spec: {}
    

    pod

    # kubectl  get po -A
    NAMESPACE                           NAME                                                              READY   STATUS              RESTARTS   AGE
    capd-system                         capd-controller-manager-659dd5f8bc-vdrb5                          2/2     Running             0          26m
    capi-kubeadm-bootstrap-system       capi-kubeadm-bootstrap-controller-manager-69889cb844-h9fpc        2/2     Running             0          26m
    capi-kubeadm-control-plane-system   capi-kubeadm-control-plane-controller-manager-6ddc66fb75-w5b9w    2/2     Running             0          26m
    capi-system                         capi-controller-manager-db59f5789-k4qdr                           2/2     Running             0          26m
    capi-webhook-system                 capi-controller-manager-64b8c548db-plfrs                          2/2     Running             0          26m
    capi-webhook-system                 capi-kubeadm-bootstrap-controller-manager-68b8cc9759-clmtg        2/2     Running             0          26m
    capi-webhook-system                 capi-kubeadm-control-plane-controller-manager-7dc88f767d-5zzjl    2/2     Running             0          26m
    cert-manager                        cert-manager-5f6b885b4-r2mhb                                      1/1     Running             0          27m
    cert-manager                        cert-manager-cainjector-bb6d9bcb5-tg8qg                           1/1     Running             0          27m
    cert-manager                        cert-manager-webhook-56cbc8f5b8-lnjkx                             1/1     Running             0          27m
    etcdadm-bootstrap-provider-system   etcdadm-bootstrap-provider-controller-manager-54476b7bf9-s5sv2    2/2     Running             0          26m
    etcdadm-controller-system           etcdadm-controller-controller-manager-d5795556-xpzzh              2/2     Running             0          26m
    kube-system                         coredns-7c68f85774-fq4jx                                          0/1     ContainerCreating   0          8m59s
    kube-system                         coredns-7c68f85774-wv77r                                          0/1     ContainerCreating   0          9m14s
    kube-system                         etcd-dev-cluster-eks-a-cluster-control-plane                      1/1     Running             0          27m
    kube-system                         kindnet-j4m6z                                                     1/1     Running             0          27m
    kube-system                         kube-apiserver-dev-cluster-eks-a-cluster-control-plane            1/1     Running             0          27m
    kube-system                         kube-controller-manager-dev-cluster-eks-a-cluster-control-plane   1/1     Running             0          27m
    kube-system                         kube-proxy-s545n                                                  1/1     Running             0          27m
    kube-system                         kube-scheduler-dev-cluster-eks-a-cluster-control-plane            1/1     Running             0          27m
    local-path-storage                  local-path-provisioner-666bfc797f-8wpv8                           1/1     Running             0          27m
    

    What you expected to happen:

    expect cilium to be deployed

    How to reproduce it (as minimally and precisely as possible):

    follow steps in https://anywhere.eks.amazonaws.com/docs/getting-started/local-environment/

    Anything else we need to know?:

    Environment:

    • EKS Anywhere Release: v0.5.0

    • EKS Distro Release:

    status/needinfo 
    opened by vincentmli 19
  • Move eks-a controller build

    Move eks-a controller build

    Issue #, if available:

    Description of changes: We currently keep the logic for the eks-a controller in the build-tooling repo. This PR moves that logic directly to this repository.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    approved lgtm ok-to-test size/L 
    opened by taneyland 17
  • Refactor artifact table generation for release assets

    Refactor artifact table generation for release assets

    Refactor artifact table generation logic for EKS-A releases into a single file. This will reduce code duplication and make it easier to add more assets.

    /hold

    Additions

    Modifications

    • [x] Update spec generation methods to use new asset methods
    • [x] Release branch integrations (1-20, 1-21, ...)
    • [x] Move remaining files existing under pkg to sub-packages based on functionality

    Deletions

    • [x] Remove all assets_X.go files
    • [x] Cleanup methods

    Testing

    • [x] Dev-release dry-run succeeds
    • [x] Source S3 and ECR URIs are preserved
    • [x] Release S3 and ECR URIs are preserved
    • [x] Testing generated bundle against golden file (#1743)

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    approved lgtm size/XXL 
    opened by abhay-krishna 16
  • Avoid downloading artifacts when running release tooling unit tests

    Avoid downloading artifacts when running release tooling unit tests

    This PR removes the step of downloading and renaming artifacts from the release-tooling unit tests. This step was being run since the test code was common with the main release code that ran download/rename/upload artifacts. But we don't need to do this since it's just unit tests.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    approved lgtm size/L 
    opened by abhay-krishna 15
  • Import images command

    Import images command

    Part of #1166

    What are we doing

    In order to facilitate disconnected environments for Snow devices, we need to be able to store all container images needed for EKS-A in an AMI (Snow admin AMI), which will be deployed to a device and later used to populate a private registry.

    The idea is that we will we use the CLI to both download and store the images and later populate the registry with them.

    Current state

    The CLI already provides an import-images command. However, this one performs the pulling (downloading images from their origin) and pushing (populating the private registry) in one step. This means that internet connection is required to populate the registry. As explained previously, this won't work for Snow.

    Proposal

    Summary

    • Add a new command import
    • Add a sub-command import images
    • Add a sub-command download images
    • Deprecate the existing command import-images

    How do these command work

    A couple of examples that are the most relevant to the Snow usecase:

    • This will download the images from the original source (mostly ECR) and store them in a tarball. This command will require internet connection.
    eksctl anywhere download images --output path/file_images.tar
    
    • And then to populate the registry:
    eksctl anywhere import images --input path/file_images.tar --registry https://djfkdks.xy/whatever
    

    All the necessary input will be provided directly through flags and env vars and not through the cluster config file (like the current import-image and other commands do). This is more flexible, more straight forward, easier to script and it doesn't require users to create and understand the syntax of a Cluster object, which in my opinion belongs to a different subdomain.

    Why storing images on disk

    An alternative could have been to just rely on docker to store the container images. Since the "pull" command will be run before "committing" the AMI, once this one is deployed to a device and the VM is started, we should be able to just do docker push against the private registry since the images should still be there.

    While this works for the Snow design, it is a bit inflexible and it doesn't allow for other disconnected environments where the push and the pull are not run in the same machine. Using a tarball allows for easier portability.

    Why a new command

    The current behavior of import-images is not flexible enough. Since the use of this command is not very extended (I don't have data to support this though), it is easier to deprecate it and create a new one, incorporating what have learned since we first introduce it. As opposed to trying to add more flags to alter how it works while keeping backwards compatibility.

    Moreover, the CLI command tries to follow a verb + resource pattern, so import images follows this better than import-images. Previous literature on this topic.

    Implementation notes

    • The eks-a tools image needs to be imported separetly since it's needed in order to initialize the helm executable, which is needed to push the helm charts to the registry.

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    approved lgtm size/XL 
    opened by g-gaston 15
  • vsphere-csi-* pods related pods not creating

    vsphere-csi-* pods related pods not creating

    Hello there,

    We finished installing eksa and was able to deploy "https://anywhere.eks.amazonaws.com/manifests/hello-eks-a.yaml" successfully. However we just noticed the following:

    vsphere-csi-controller-765894dd54-sh8pp 0/5 ContainerCreating 0 4d vsphere-csi-node-g2bxf 0/3 ContainerCreating 0 3d23h vsphere-csi-node-p69j8 0/3 ContainerCreating 0 3d23h vsphere-csi-node-xl6x2 0/3 ContainerCreating 0 4d vsphere-csi-node-zxgr4 0/3 ContainerCreating 0 3d23h

    Also

    Events: Type Reason Age From Message


    Warning FailedMount 7m47s (x1465 over 4d) kubelet Unable to attach or mount volumes: unmounted volumes=[vsphere-config-volume], unattached volumes=[socket-dir kube-api-access-47njc vsphere-config-volume]: timed out waiting for the condition Warning FailedMount 3m22s (x2840 over 4d) kubelet MountVolume.SetUp failed for volume "vsphere-config-volume" : secret "csi-vsphere-config" not found

    Where do we need to specify this csi-related config in the original cluster.yaml file? What are we missing here?

    Thanks!

    opened by qwangne 15
  • vSphere Provider setup is valid error on template deployment: please specify a datacenter

    vSphere Provider setup is valid error on template deployment: please specify a datacenter

    What happened: When trying to deploy a cluster to vSphere, it will error out on the step of deploying the template to a Content Library with: Validation failed {"validation": "vsphere Provider setup is valid", "error": "failed deploying template: error deploying template: govc: please specify a datacenter\n", "remediation": ""}

    What you expected to happen: It succeeds. From what I can see, I have a datacenter specified. My vCenter has multiple datacenters in it with clusters.

    How to reproduce it (as minimally and precisely as possible): Attempt to deploy a cluster to vSphere. vCenter version is 7.0U3 build 18700403. My vCenter has multiple datacenters in it with clusters.

    My config YAML:

    apiVersion: anywhere.eks.amazonaws.com/v1alpha1
    kind: Cluster
    metadata:
      name: mmg-test
    spec:
      clusterNetwork:
        cni: cilium
        pods:
          cidrBlocks:
          - 10.69.0.0/16
        services:
          cidrBlocks:
          - 10.112.0.0/12
      controlPlaneConfiguration:
        count: 2
        endpoint:
          host: "10.96.78.5"
        machineGroupRef:
          kind: VSphereMachineConfig
          name: mmg-test-cp
      datacenterRef:
        kind: VSphereDatacenterConfig
        name: mmg-test
      externalEtcdConfiguration:
        count: 3
        machineGroupRef:
          kind: VSphereMachineConfig
          name: mmg-test-etcd
      kubernetesVersion: "1.21"
      workerNodeGroupConfigurations:
      - count: 2
        machineGroupRef:
          kind: VSphereMachineConfig
          name: mmg-test
    
    ---
    apiVersion: anywhere.eks.amazonaws.com/v1alpha1
    kind: VSphereDatacenterConfig
    metadata:
      name: mmg-test
    spec:
      datacenter: "ZB"
      insecure: true
      network: "eks-mmgtest"
      server: "vcenter-fqdn"
      thumbprint: ""
    
    ---
    apiVersion: anywhere.eks.amazonaws.com/v1alpha1
    kind: VSphereMachineConfig
    metadata:
      name: mmg-test-cp
    spec:
      datastore: "ESXi 3/ESXi3 SSD 3"
      diskGiB: 25
      folder: ""
      memoryMiB: 8192
      numCPUs: 2
      osFamily: bottlerocket
      resourcePool: "COMG3/Resources"
      users:
      - name: ec2-user
        sshAuthorizedKeys:
        - ssh-rsa MYPUBLICKEY
    
    ---
    apiVersion: anywhere.eks.amazonaws.com/v1alpha1
    kind: VSphereMachineConfig
    metadata:
      name: mmg-test
    spec:
      datastore: "ESXi 3/ESXi3 SSD 3"
      diskGiB: 25
      folder: ""
      memoryMiB: 8192
      numCPUs: 2
      osFamily: bottlerocket
      resourcePool: "COMG3/Resources"
      users:
      - name: ec2-user
        sshAuthorizedKeys:
        - ssh-rsa MYPUBLICKEY
    
    ---
    apiVersion: anywhere.eks.amazonaws.com/v1alpha1
    kind: VSphereMachineConfig
    metadata:
      name: mmg-test-etcd
    spec:
      datastore: "ESXi 3/ESXi3 SSD 3"
      diskGiB: 25
      folder: ""
      memoryMiB: 8192
      numCPUs: 2
      osFamily: bottlerocket
      resourcePool: "COMG3/Resources"
      users:
      - name: ec2-user
        sshAuthorizedKeys:
        - ssh-rsa MYPUBLICKEY
    
    ---
    
    

    Anything else we need to know?: N/A

    Environment:

    • EKS Anywhere Release: v0.5.0
    • EKS Distro Release: Unknown
    opened by MasterWayZ 15
  • Release brew trigger for v0.9.2

    Release brew trigger for v0.9.2

    Issue #, if available:

    Description of changes:

    Testing (if applicable):

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    approved lgtm size/XS 
    opened by mitalipaygude 14
  • Allow users to choose between `LinkedClone` or `FullClone` mode for vSphere provider

    Allow users to choose between `LinkedClone` or `FullClone` mode for vSphere provider

    What would you like to be added: Add option in EKS Anywhere on vSphere to allow users to specify whether to use LinkedClone or FullClone mode for OS images.

    Why is this needed: Currently, EKS Anywhere on vSphere is hardcoded to use LinkedClone mode. This makes the VM provisioning times faster but doesn't allow users to configure the disk size on VMs.

    Because of the way LinkedClone works, if the template has snapshots, then the VM size isn't configurable. So, if the users want to modify the disk size, they have to delete the template snapshots for the VMs to fall back to FullClone.

    kind/enhancement good first issue area/providers/vsphere 
    opened by abhinavmpandey08 1
  • refactor(CLI): Use the new kubeconfig functions in CP CLI commands

    refactor(CLI): Use the new kubeconfig functions in CP CLI commands

    Refactors the curated packages CLI commands to use the new functions introduced in ac5db7f. The non curated packages commands were not refactored, because I'm not 100% sure it's safe to do so. I'll leave that exercise for a later PR.

    This helps simplify our k8s go client creation for CP CLI commands, which will be coming in a follow-up commit.

    do-not-merge/hold size/L 
    opened by ewollesen 3
  • refactor(CLI): DRYs up kubeconfig checking in packages commands

    refactor(CLI): DRYs up kubeconfig checking in packages commands

    This bit of repetitive code was lifted into the kubeconfig package, where it can be modified once to prevent someone from having to do more shotgun commits like this in the future.

    The behavior for kubeconfig flags remains the same as previously in aws/eks-anywhere#3304 and aws/eks-anywhere#3299.

    Issue #, if available:

    Description of changes:

    Testing (if applicable):

    By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

    approved lgtm size/M 
    opened by ewollesen 11
Releases(weekly.2022-09-22)
Amazon ECS Container Agent: a component of Amazon Elastic Container Service

Amazon ECS Container Agent The Amazon ECS Container Agent is a component of Amazon Elastic Container Service (Amazon ECS) and is responsible for manag

null 0 Dec 28, 2021
ecsk is a CLI tool to interactively use frequently used functions of docker command in Amazon ECS. (docker run, exec, cp, logs, stop)

English / ๆ—ฅๆœฌ่ชž ecsk ECS + Task = ecsk ?? ecsk is a CLI tool to interactively use frequently used functions of docker command in Amazon ECS. (docker run

null 107 Aug 31, 2022
A golang tool to list out all EKS clusters with active nodegroups in all regions in json format

eks-tool A quick and dirty tool to list out all EKS clusters with active nodegro

null 0 Dec 18, 2021
Simple-go-api - This porject deploys a simple go app inside a EKS Cluster

SimpleGoApp This porject deploys a simple go app inside a EKS Cluster Prerequisi

null 0 Jan 19, 2022
A Terraform module to manage cluster authentication (aws-auth) for an Elastic Kubernetes (EKS) cluster on AWS.

Archive Notice The terraform-aws-modules/eks/aws v.18.20.0 release has brought back support aws-auth configmap! For this reason, I highly encourage us

Aidan Melen 27 Sep 14, 2022
k8s-image-swapper Mirror images into your own registry and swap image references automatically.

k8s-image-swapper Mirror images into your own registry and swap image references automatically. k8s-image-swapper is a mutating webhook for Kubernetes

Enrico Stahn 316 Sep 19, 2022
A simple Kubernetes Operator template that uses Golang, use it to build your own operators

A simple programmatic Kubernetes Operator template. Use this to create your own Kubernetes operators with golang. Build with KIND (Kubernetes in Docke

Cloud Native Skunkworks 7 May 13, 2022
Make any web accessible from your own host / domain

Web Mirror Based on reverseproxy Solution for: Website that only set 'X-Frame-Options' to 'sameorigin'. Hide website real url Content interception & m

Rubi 3 May 31, 2022
An extensible tool for creating your own in cluster health endpoints

healthyk8s an extensible tool for creating your own "in cluster" health endpoints Why? allows for creating a health endpoint for anything - external r

Nic Grobler 0 Oct 26, 2021
Latest block exporter to monitor your own nodes !

Ethereum Block Prometheus Exporter Deeply copied from 31z4/ethereum-prometheus-exporter Thanks a lot for his work ! This service exports the latest bl

iderr 0 Nov 5, 2021
Dynamic DNS on your own Domain, written in Go

dyngo Dynamic DNS on your own Domain, written in Go. It will determine the external IP of the system it is running on and updates a given domain recor

Matthias 3 May 2, 2022
Fast, concurrent, streaming access to Amazon S3, including gof3r, a CLI. http://godoc.org/github.com/rlmcpherson/s3gof3r

s3gof3r s3gof3r provides fast, parallelized, pipelined streaming access to Amazon S3. It includes a command-line interface: gof3r. It is optimized for

Randall McPherson 1.1k Sep 27, 2022
Prometheus exporter for Amazon Elastic Container Service (ECS)

ecs_exporter ?? ?? ?? This repo is still work in progress and is subject to change. This repo contains a Prometheus exporter for Amazon Elastic Contai

Prometheus Monitoring Community 43 Sep 18, 2022
Amazon Web Services (AWS) providerAmazon Web Services (AWS) provider

Amazon Web Services (AWS) provider The Amazon Web Services (AWS) resource provider for Pulumi lets you use AWS resources in your cloud programs. To us

William Garcia Jacobo 0 Nov 10, 2021
Amazon Elastic Container Service Agent

Amazon ECS Container Agent The Amazon ECS Container Agent is a component of Amazon Elastic Container Service (Amazon ECS) and is responsible for manag

Amazon Web Services 1.9k Sep 20, 2022
Test-csi-driver - Amazon Elastic Block Store (EBS) CSI driver

Amazon Elastic Block Store (EBS) CSI driver Overview The Amazon Elastic Block St

Adi Vaknin 0 Feb 1, 2022
Godart - Amazon Alexa skill in Go to read train times out loud

GODART Alexa skill to have DART times for the requested station. build and deplo

Patrick O'Sullivan 1 Apr 13, 2022
Continuous Delivery for Declarative Kubernetes, Serverless and Infrastructure Applications

Continuous Delivery for Declarative Kubernetes, Serverless and Infrastructure Applications Explore PipeCD docs ยป Overview PipeCD provides a unified co

PipeCD 613 Sep 23, 2022