K8s cluster simulator for workload scheduling.

Overview

Open-Simulator

Go Report Card workflow build

Motivation

概念定义

Open-Simulator 是 K8s 下的仿真调度组件。用户准备一批待创建 Workload 资源,Workload 资源指定好资源配额、绑核规则、亲和性规则、优先级等,通过 Open-Simulator 的仿真调度能力可判断当前集群是否能够满足 Workload 资源,以及添加多少资源可保证资源部署成功。

原生 Kubernetes 缺少仿真调度能力,且社区并没有相关项目供参考。Open-Simulator 可解决资源规划问题,通过Workload 调度要求计算出最少物理资源数量,进而提高资源使用率,为用户节省物理成本和运维成本。

Use Case

两类场景需要资源规划:

  • 交付前:评估产品最少物理资源,通过仿真系统计算出交付需要的特定规格节点数量、磁盘数量(类似朱雀系统);
  • 运行时:用户新建 or 扩容 Workload,仿真调度系统会给出当前集群物理资源是否满足,并给出集群扩容建议(详细到扩容节点数)

Run

使用

添加节点

执行命令

./simon apply --kubeconfig=[kubeconfig文件目录] -f [Yaml文件夹目录]

Yaml文件夹参考./example目录,包含如下文件:

  • Deployment yamls
  • Statefulset yamls
  • Node yaml

执行后输出一个名为configmap-simon.yaml的文件,用以保存结果。

apiVersion: v1
kind: ConfigMap
metadata:
  name: simulator-plan
  namespace: kube-system
data:
  Deployment: '{"vivo-test-namespace/suppress-memcache-lsr":["simulator-node1","simulator-node1","node3","node2"],"vivo-test-namespace/suppress-memcache-be":["simulator-node1","simulator-node1","node3","node2"]}'
  StatefulSet: '{"vivo-test-namespace/suppress-memcache-lsr":["simulator-node1","simulator-node1","node3","node2"],"vivo-test-namespace/suppress-memcache-be":["simulator-node1","simulator-node1","node3","node2"]}'

效果图

Deployment

以 MacBook 为例

步骤

# 克隆项目
mkdir $(GOPATH)/github.com/alibaba
cd $(GOPATH)/github.com/alibaba
git clone https://github.com/alibaba/open-simulator.git
cd open-simulator

# 安装minikube并运行
minikube start

# 拷贝 kubeconfig 文件到项目目录
cp ~/.kube/config  ./kubeconfig

# 项目编译及运行
make
bin/simon apply --kubeconfig=./kubeconfig -f ./example/simple_example_by_huizhi
Issues
Owner
Alibaba
Alibaba Open Source
Alibaba
nano-gpu-scheduler is a Kubernetes scheduler extender for GPU resources scheduling.

Nano GPU Scheduler About This Project With the continuous evolution of cloud native AI scenarios, more and more users run AI tasks on Kubernetes, whic

Nano GPU 32 Oct 14, 2021
Distributed Task Scheduling System|分布式定时任务调度平台

Crocodile Distributed Task Scheduling System English | 中文 Introduction A distributed task scheduling system based on Golang that supports http request

labulaka521 734 Oct 14, 2021
Easy and fluent Go cron scheduling

goCron: A Golang Job Scheduling Package. goCron is a Golang job scheduling package which lets you run Go functions periodically at pre-determined inte

Go Co Op 1.2k Oct 22, 2021
goCron: A Golang Job Scheduling Package.

goCron: A Golang Job Scheduling Package.

辣椒面 2.7k Oct 20, 2021
A persistent and flexible background jobs library for go.

Jobs Development Status Jobs is no longer being actively developed. I will still try my best to respond to issues and pull requests, but in general yo

Alex Browne 483 Oct 17, 2021
gron, Cron Jobs in Go.

gron Gron provides a clear syntax for writing and deploying cron jobs. Goals Minimalist APIs for scheduling jobs. Thread safety. Customizable Job Type

roylee0704 877 Oct 20, 2021
Package tasks is an easy to use in-process scheduler for recurring tasks in Go

Tasks Package tasks is an easy to use in-process scheduler for recurring tasks in Go. Tasks is focused on high frequency tasks that run quick, and oft

Benjamin Cane 58 Oct 2, 2021
clockwork - Simple and intuitive job scheduling library in Go.

clockwork A simple and intuitive scheduling library in Go. Inspired by python's schedule and ruby's clockwork libraries. Example use package main imp

null 29 Jul 13, 2021
Simple, zero-dependency scheduling library for Go

go-quartz Simple, zero-dependency scheduling library for Go. About Inspired by the Quartz Java scheduler. Library building blocks Job interface. Any t

Eugene R. 535 Oct 18, 2021
Job scheduling made easy.

scheduler Job scheduling made easy. Scheduler allows you to schedule recurrent jobs with an easy-to-read syntax. Inspired by the article Rethinking Cr

Carles Cerezo Guzmán 371 Oct 15, 2021
A programmable, observable and distributed job orchestration system.

?? Overview Odin is a programmable, observable and distributed job orchestration system which allows for the scheduling, management and unattended bac

James McDermott 427 Oct 10, 2021
A lightweight job scheduler based on priority queue with timeout, retry, replica, context cancellation and easy semantics for job chaining. Build for golang web apps.

Table of Contents Introduction What is RIO? Concern An asynchronous job processor Easy management of these goroutines and chaining them Introduction W

Supratim Samanta 40 Oct 20, 2021
high performance distributed task scheduling system, Support multi protocol scheduling tasks

high performance distributed task scheduling system, Support multi protocol scheduling tasks

null 35 Sep 26, 2021
Machinery is an asynchronous task queue/job queue based on distributed message passing.

Machinery Machinery is an asynchronous task queue/job queue based on distributed message passing. V2 Experiment First Steps Configuration Lock Broker

Richard Knop 5.7k Oct 22, 2021
You had one job, or more then one, which can be done in steps

Leprechaun Leprechaun is tool where you can schedule your recurring tasks to be performed over and over. In Leprechaun tasks are recipes, lets observe

Strahinja 83 Oct 7, 2021
Run Jobs on a schedule, supports fixed interval, timely, and cron-expression timers; Instrument your processes and expose metrics for each job.

A simple process manager that allows you to specify a Schedule that execute a Job based on a Timer. Schedule manage the state of this job allowing you to start/stop/restart in concurrent safe way. Schedule also instrument this Job and gather metrics and optionally expose them via uber-go/tally scope.

Sherif Abdel-Naby 54 Jul 31, 2021
A simple job scheduler backed by Postgres.

A simple job scheduler backed by Postgres used in production at https://operand.ai. Setup needs two environment variables, SECRET and ENDPOINT. The se

Morgan Gallant 6 May 31, 2021
Chadburn is a scheduler alternative to cron, built on Go and designed for Docker environments.

Chadburn - a job scheduler Chadburn is a modern and low footprint job scheduler for docker environments, written in Go. Chadburn aims to be a replacem

PremoWeb Internet Services 5 Oct 11, 2021
GPU Sharing Scheduler for Kubernetes Cluster

GPU Sharing Scheduler Extender in Kubernetes Overview More and more data scientists run their Nvidia GPU based inference tasks on Kubernetes. Some of

Aliyun (Alibaba Cloud) Container Service 782 Oct 15, 2021