Create AWS Auto Scaling groups from running instances or AMIs automatically.

Overview

AWS Auto Scaling Groups Builder

AWS Auto Scaling group is a great way of managing Amazon EC2 instances. AWS Auto Scaling group watches the corresponding instances' health and launches new instances whenever needed: to replace instances that became unhealthy or to scale the group. Furthermore, with an AWS Elastic Load Balancer attached, AWS Auto Scaling Group also takes ELB health checks into account, making it possible to replace the instances based on service-level signals.

However, creating AWS Auto Scaling Groups might be tricky. To create a new group, one needs to perform many actions using either AWS Management Console or AWS CLI. In addition, the user needs to wait until the corresponding resources become available for some steps. It might not be easy to recall all the steps required if preparing new services is not a regular job.

So, this is a job for an automation tool, such as AWS Auto Scaling Groups Builder. It uses AWS EC2 API, AWS EC2 Auto Scaling API, and AWS ELB API to perform the following operations automatically:

  • register an AMI from a running instance;
  • create a launch template using this AMI;
  • register a target group;
  • create a load balancer with a listener that forwards traffic to this target group;
  • create an Auto Scaling group with both EC2 and ELB health checks;
  • handle all the errors that might happen along the way; in case of failure, all the created artifacts are deleted.

The tool uses the default AWS credentials config. Run aws configure or set up the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION before running the tool.

Only AWS EC2 Auto Scaling is supported. The service is supposed to handle health checks via HTTP.

The tool offers some default settings for all the services, which are not available for changing via command-line arguments. However, it's always possible to change them manually with the default AWS tooling after the service is ready. These defaults are:

  • All the instances of the created service inherit the following settings of the source instance:
    • instance type;
    • kernel ID;
    • key name;
    • licence specifications;
    • placement;
    • network interfaces.
  • The service only uses spot instances, despite the source instance lifecycle option.
  • The tool creates and AMI with name $GROUP_NAME v1. The launch template name is just $GROUP_NAME. The names of target group and balancer are both $GROUP_NAME with substitution: - instead of _. E.g., if you're going to create an Auto Scaling group with name my_service, the following resources will be created:
    • AMI with my_service v1;
    • launch template my_service;
    • target group my-service;
    • load balancer my-service;
    • Auto Scaling group my_service.
  • The balancer is created in all availability zones, in the default subnets. So, if one has six availability zones in their AWS account, all these zones will be used for placing the load balancer.
  • The instances for the service are also placed within the default subnets of all the availability zones of the AWS account.
  • The Auto Scaling group has the following capacity settings:
    • maximum capacity = 2 * desired capacity = 2 * minimum capacity;
    • capacity rebalancing is turned on;
    • no dynamic or predictive scaling policies, no scheduled actions.

Examples of Use

The following command will create an Auto Scaling group my_service_group from an instance i-0699803d818227e16:

aws_asg_builder --instance i-0699803d818227e16 --group my_service_group --port 8080 --instances 10

The output looks like that then:

2022/01/07 10:42:01 will create an AMI "my_service_group v1" from the instance i-0699803d818227e16
2022/01/07 10:42:01 will create a launch template "my_service_group"
2022/01/07 10:42:01 will create a target group "my-service-group" in VPC vpc-c832bdb5
2022/01/07 10:42:01 will create a load balancer "my-service-group" in subnets subnet-7e353733, subnet-bb0d39b5, subnet-6bafd034, subnet-6b7c0e0d, subnet-66ca4b57, subnet-dd8ef7fc
2022/01/07 10:42:01 will create an auto scaling group "my_service_group" with 10 t2.small spot instances
2022/01/07 10:42:01 ami-00c78efe33a162b36 ("my_service_group v1"): pending
2022/01/07 10:43:02 ami-00c78efe33a162b36 ("my_service_group v1"): pending
2022/01/07 10:44:02 ami-00c78efe33a162b36 ("my_service_group v1"): pending
2022/01/07 10:45:02 ami-00c78efe33a162b36 ("my_service_group v1"): available
2022/01/07 10:45:02 created launch template "my_service_group"
2022/01/07 10:45:03 created target group "my-service-group" (arn:aws:elasticloadbalancing:us-east-1:046898261394:targetgroup/my-service-group/9ff35ebad2a86a0d)
2022/01/07 10:45:05 load balancer "my-service-group": provisioning
2022/01/07 10:46:05 load balancer "my-service-group": provisioning
2022/01/07 10:47:05 load balancer "my-service-group": active
2022/01/07 10:47:09 group "my_service_group": 0 instances in total, 0 instances are in service and healthy (10 needed)
2022/01/07 10:48:09 group "my_service_group", instance "i-0071e23d5f723ca78": InService, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-0250bc8b1ef1173d2": InService, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-02c51ec311822e072": InService, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-0415988f40e7cb87e": InService, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-0470f10de375fc978": Pending, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-047cd43b77947afce": InService, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-087d57ef33e6c26b0": Pending, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-08d426cade3c3bf8e": Pending, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-09a68825c258faf9f": InService, Healthy
2022/01/07 10:48:09 group "my_service_group", instance "i-0bb9b9061e7970004": InService, Healthy
2022/01/07 10:48:09 group "my_service_group": 10 instances in total, 7 instances are in service and healthy (10 needed)
2022/01/07 10:49:10 group "my_service_group", instance "i-0071e23d5f723ca78": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-0250bc8b1ef1173d2": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-02c51ec311822e072": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-0415988f40e7cb87e": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-0470f10de375fc978": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-047cd43b77947afce": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-087d57ef33e6c26b0": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-08d426cade3c3bf8e": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-09a68825c258faf9f": InService, Healthy
2022/01/07 10:49:10 group "my_service_group", instance "i-0bb9b9061e7970004": InService, Healthy
2022/01/07 10:49:10 group "my_service_group": 10 instances in total, 10 instances are in service and healthy (10 needed)
2022/01/07 10:49:10 successfully created an auto scaling group "my_service_group"
2022/01/07 10:49:10 enabled metrics collection for the group "my_service_group"
2022/01/07 10:49:10 AMI link: https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#ImageDetails:imageId=ami-00c78efe33a162b36
2022/01/07 10:49:10 Launch template link: https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchTemplateDetails:launchTemplateId=lt-0ce496d3c72c69e9d
2022/01/07 10:49:10 Target group link: https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#TargetGroup:targetGroupArn=arn:aws:elasticloadbalancing:us-east-1:046898261394:targetgroup/my-service-group/9ff35ebad2a86a0d
2022/01/07 10:49:10 Balancer link: https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LoadBalancers:search=my-service-group
2022/01/07 10:49:10 Auto Scalingr group link: https://console.aws.amazon.com/ec2autoscaling/home?region=us-east-1#/details/my_service_group
2022/01/07 10:49:10 check out the health status: http://my-service-group-1643996769.us-east-1.elb.amazonaws.com:8080/health

Program Arguments

  • group: the name of the Auto Scaling group to create; required.
  • instance: AWS EC2 instance ID to create the service from; required.
  • health-path: the health HTTP handler for the service; optional, default: /health.
  • port: the HTTP traffic port for the service; optional, default: 80.
  • instances: the number of instances to create within the group; optional, default: 1.
  • health-check-grace-period: the time needed for the instance to become healthy after the launch; optional, default: 1m. Use the Golang duration strings to override, see https://pkg.go.dev/time#ParseDuration.
  • update-timeout: the time limit to complete the instance refresh; optional, default: 30m. Use the Golang duration strings to override, see https://pkg.go.dev/time#ParseDuration.
  • update-tick: the time between status updates in the log file; optional, default: 1m. Making this parameter lower might speed up the overall execution. Use the Golang duration strings to override, see https://pkg.go.dev/time#ParseDuration.

Installation

Assuming you already have Golang installed on the machine, simply run:

go install github.com/ashagraev/[email protected]

The tool will then appear in your golang binary folder (e.g., ~/go/bin/aws_asg_builder). If you don't have Golang yet, consider installing it using the official guide https://go.dev/doc/install.

Alternatively, you can download the pre-built binaries from the latest release: https://github.com/ashagraev/aws_asg_builder/releases/latest.

You might also like...
Automatically create global & local Rate Limit in Istio, support EnvoyFilter versioning!

istio-ratelimit-operator Istio ratelimit operator provide an easy way to configure Global or Local Ratelimit in Istio mesh. Istio ratelimit operator a

A plugin for running Open Policy Agent (OPA) in AWS Lambda as a Lambda Extension.

opa-lambda-extension-plugin A custom plugin for running Open Policy Agent (OPA) in AWS Lambda as a Lambda Extension. To learn more about how Lambda Ex

The AWS Enumerator was created for service enumeration and info dumping for investigations of penetration testers during Black-Box testing. The tool is intended to speed up the process of Cloud review in case the security researcher compromised AWS Account Credentials. Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)
Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)

terraform-provider-awsutils Terraform provider for performing various tasks that cannot be performed with the official AWS Terraform Provider from Has

Infrastructure testing helper for AWS Resources that uses AWS SSM to remotely execute commands on EC2 machines.
Infrastructure testing helper for AWS Resources that uses AWS SSM to remotely execute commands on EC2 machines.

Infrastructure testing helper for AWS Resources that uses AWS SSM to remotely execute commands on EC2 machines, to enable infrastructure engineering teams to write tests that validate behaviour.

Amazon Web Services (AWS) providerAmazon Web Services (AWS) provider

Amazon Web Services (AWS) provider The Amazon Web Services (AWS) resource provider for Pulumi lets you use AWS resources in your cloud programs. To us

Aws-secretsmanager-caching-extension - Cache server for AWS Secrets Manager
Aws-secretsmanager-caching-extension - Cache server for AWS Secrets Manager

AWS Lambda Extension / Sidecar Container Cache Server The cache server is writte

Pulumi-aws-iam - Reusable IAM modules for AWS

xyz Pulumi Component Provider (Go) This repo is a boilerplate showing how to cre

A Terraform module to manage cluster authentication (aws-auth) for an Elastic Kubernetes (EKS) cluster on AWS.

Archive Notice The terraform-aws-modules/eks/aws v.18.20.0 release has brought back support aws-auth configmap! For this reason, I highly encourage us

Owner
Alexey Shagraev
Alexey Shagraev
Rotate is a tool for rotating out AWS Auto-Scaling Groups within a k8s cluster

k8s-r8 rotate is a tool for rotating out AWS Auto-Scaling Groups within a k8s cluster. It was developed to make upgrading AMIs as a one command experi

maikxchd 0 Mar 27, 2022
Service-scaling - Microservice scaling with example

Scaling examples This is the repo for my (Hungarian) Twitch streams where I spea

Gergely Radics 0 Jan 8, 2022
This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances

Session Manager Plugin This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances. Sess

Amazon Web Services 175 Nov 15, 2022
Carrier is a Kubernetes controller for running and scaling game servers on Kubernetes.

Carrier is a Kubernetes controller for running and scaling game servers on Kubernetes. This project is inspired by agones. Introduction Genera

Open Cloud-native Game-application Initiative 30 Jul 28, 2022
Automatically stop GCE instances

auto-stop-gce-instances Architecture How to deploy 1. Deploy Pub/Sub $ gcloud pubsub topics create $PUBSUB_NAME 2. Deploy Cloud Functions $ gcloud fu

Tomomasa Uchida 0 Nov 28, 2021
Git-auto-push - Auto commit and push to github repositories

Auto commit and push to github repositories. How to use git clone https://github

Z.Q.K 4 Oct 30, 2022
go-awssh is a developer tool to make your SSH to AWS EC2 instances easy.

Describing Instances/VPCs data, select one or multiple instances, and make connection(s) to selected instances. Caching the response of API calls for 1day using Tmpfs.

kenju 5 Oct 11, 2021
Kubegres is a Kubernetes operator allowing to create a cluster of PostgreSql instances and manage databases replication, failover and backup.

Kubegres is a Kubernetes operator allowing to deploy a cluster of PostgreSql pods with data replication enabled out-of-the box. It brings simplicity w

Reactive Tech Ltd 1.1k Nov 21, 2022
A Kubernetes Operator used for pre-scaling applications in anticipation of load

Pre-Scaling Kubernetes Operator Built out of necessity, the Operator helps pre-scale applications in anticipation of load. At its core, it manages a c

Container Solutions 24 Oct 14, 2021
GoBinClassify - A library that makes it easy to classify into groups

GoBinClassify GoBinClassify is a library that makes it easy to classify into gro

null 0 Feb 12, 2022