Build a retractable ECS load balance network through aliyun openapi.

Overview

ECSEquilizer Build Status

通过阿里云OpenAPI建立一个可伸缩的负载均衡网络调度器。

简介

为确保代理集群网络和计算能力可以通过ECS云服务动态伸缩,所以制定实现以下策略。

节点分为static和dynamic两种,static节点是通过配置文件(config.yaml)实现预设的,固定不变。

dynamic由调度器根据节点集群健康状态调用ECS节点动态伸缩。

健康状态评估

每个节点有三个状态:great,normal,bad

评估方法用tcping,对每个节点的业务TCP端口ping四次求平均值,延迟小于100ms为great,小于1s为normal,超过1s为bad

负载均衡策略

调度器每10分钟调度一次,每次调度遵循三个规则。

  1. 当static节点状态均大于等于normal状态(且持续超过1个小时),则释放所有dynamic节点。

  2. 当所有节点当中,超过50%的节点状态为bad,则调度器自动创建N个dynamic节点,使网络状态恢复至50%节点大于等于normal。

  3. 当所有节点中,超过60%的节点状态为大于等于normal,则持续释放dynamic至小于60%。

新创建的节点由于ECS初始化延迟的缘故,健康状态默认为normal,实际状态由下次调度更新实际状态。

编译/测试/运行

$> make tidy

$> make test

$> make 过程中会创建https自签证书

编辑 ./target/config.yaml 修改配置信息

$> cd target

$> ./ecsEquilizer

HTTP 身份认证

修改config.yaml配置文件中的authorization参数添加Key

在HTTP请求头中添加 Authorization : Bearer [vaild-key]

Restful API

GET /v1/nodes

Result:

{
	"nodes":[
		{
			"id":"S0",
			"ip":"127.0.0.1",
			"port":"1234",
			"type":"static",
			"status":"normal"
		},
		{
			"id":"S1",
			"ip":"127.0.0.1",
			"port":"4567",
			"type":"dynamic",
			"status":"normal"
		}
	]
}

回调接口

修改config.yaml配置文件中的https_callback参数添加回调地址,回调格式如下。

设置 https_callback_auth 参数,将在回调请求的HTTP头参数中加入Authorization进行身份认证

如果https_callback参数为空字符串,则不使用回调。


POST [https_callback]

{
    "id": "D1", 
    "ip": "127.0.0.1", 
    "action": "dropped"
}

action 参数值有两种,createddropped,分别表示对该结点的创建和释放。

业务接入

业务可通过调度器接口,拿到所有节点状态,优先拿出健康状态更好的节点为客户提供服务。

调度器提供回调,当有节点创建或者被释放,将通知业务节点。

注意事项

修改config.yaml配置文件中的work_internal_second参数设置调度间隔时间,要确保时长足够所需调度的节点启动完成。

You might also like...
OpenAPI Terraform Provider that configures itself at runtime with the resources exposed by the service provider (defined in a swagger file)
OpenAPI Terraform Provider that configures itself at runtime with the resources exposed by the service provider (defined in a swagger file)

Terraform Provider OpenAPI This terraform provider aims to minimise as much as possible the efforts needed from service providers to create and mainta

Kusk makes your OpenAPI definition the source of truth for API resources in your cluster
Kusk makes your OpenAPI definition the source of truth for API resources in your cluster

Kusk - use OpenAPI to configure Kubernetes What is Kusk? Developers deploying their REST APIs in Kubernetes shouldn't have to worry about managing res

Monitor your Website and APIs from your Computer. Get Notified through Slack, E-mail when your server is down or response time is more than expected.
Monitor your Website and APIs from your Computer. Get Notified through Slack, E-mail when your server is down or response time is more than expected.

StatusOK Monitor your Website and APIs from your computer.Get notified through Slack or E-mail when your server is down or response time is more than

Command-line tool to remotely execute commands on Windows machines through WinRM

winrm-cli This is a Go command-line executable to execute remote commands on Windows machines through the use of WinRM/WinRS. Note: this tool doesn't

Parallel processing through go routines, copy and delete thousands of key within some minutes

redis-dumper CLI Parallel processing through go routines, copy and delete thousands of key within some minutes copy data by key pattern from one redis

Igo Agent is the agent of Igo, a command-line tool, through which you can quickly start Igo

igo agent 英文 | 中文 Igo Agent is the agent of Igo, a command-line tool, through which you can quickly start Igo, and other capabilities may be added lat

A toy project to play music (aha, Nose in fact) through Prometheus metrics

tiplay A toy project to play music (aha, mostly Noise in fact) through Prometheu

Blast is a simple tool for API load testing and batch jobs

Blast Blast makes API requests at a fixed rate. The number of concurrent workers is configurable. The rate may be changed interactively during executi

:rocket: Modern cross-platform HTTP load-testing tool written in Go
:rocket: Modern cross-platform HTTP load-testing tool written in Go

English | 中文 Cassowary is a modern HTTP/S, intuitive & cross-platform load testing tool built in Go for developers, testers and sysadmins. Cassowary d

Owner
B23r0
If the code cant run , thats not what i wrote. :b
B23r0
provide api for cloud service like aliyun, aws, google cloud, tencent cloud, huawei cloud and so on

cloud-fitter 云适配 Communicate with public and private clouds conveniently by a set of apis. 用一套接口,便捷地访问各类公有云和私有云 对接计划 内部筹备中,后续开放,有需求欢迎联系。 开发者社区 开发者社区文档

null 23 May 8, 2022
exporter for Aliyun CloudMonitor. Written in Golang.

Aliyun CloudMonitor Exporter exporter for Aliyun CloudMonitor. Written in Golang. inspired by aliyun-exporter Develop cd aliyun-exporter make tidy Bui

IAOTW 33 Nov 21, 2022
Endpoints-operator - Kubernetes endpoints balance for outsite apiserver

endpoints-operator 对于集群内访问集群外部服务场景使用固定的endpoint维护增加探活功能 背景 在实际使用中,两个K8s集群内的服务经常有

sealyun 58 Nov 16, 2022
network-node-manager is a kubernetes controller that controls the network configuration of a node to resolve network issues of kubernetes.

Network Node Manager network-node-manager is a kubernetes controller that controls the network configuration of a node to resolve network issues of ku

kakao 99 Oct 4, 2022
K8s-network-config-operator - Kubernetes network config operator to push network config to switches

Kubernetes Network operator Will add more to the readme later :D Operations The

Daniel Hertzberg 6 May 16, 2022
ecsk is a CLI tool to interactively use frequently used functions of docker command in Amazon ECS. (docker run, exec, cp, logs, stop)

English / 日本語 ecsk ECS + Task = ecsk ?? ecsk is a CLI tool to interactively use frequently used functions of docker command in Amazon ECS. (docker run

null 109 Nov 15, 2022
A tool to automate some of my tasks in ECS/ECR.

severinoctl A tool to automate some tasks in ECS/ECR. Work in progress... Prerequisites awscli working aws credentials environment AWS_REGION exported

Edson C. 7 Feb 19, 2022
Prometheus exporter for Amazon Elastic Container Service (ECS)

ecs_exporter ?? ?? ?? This repo is still work in progress and is subject to change. This repo contains a Prometheus exporter for Amazon Elastic Contai

Prometheus Monitoring Community 48 Nov 11, 2022
🥝 Mini ECS CLI Command 🥝

miniecs ?? miniecs is a CLI tool for AWS ECS. ?? Requirement go 1.17.x or later �Installation go install github.com/jedipunkz/[email protected] Usage $ m

Tomokazu HIRAI 2 Oct 26, 2022
Amazon ECS Container Agent: a component of Amazon Elastic Container Service

Amazon ECS Container Agent The Amazon ECS Container Agent is a component of Amazon Elastic Container Service (Amazon ECS) and is responsible for manag

null 0 Dec 28, 2021