DigitalOcean Droplets target plugin for HashiCorp Nomad Autoscaler

Overview

Nomad DigitalOcean Droplets Autoscaler

The do-droplets target plugin allows for the scaling of the Nomad cluster clients via creating and destroying DigitalOcean Droplets.

Requirements

  • nomad autoscaler 0.3.0+
  • DigitalOcean account

Documentation

Agent Configuration Options

To use the do-droplets target plugin, the agent configuration needs to be populated with the appropriate target block. Currently, Personal Access Token (PAT) is the only method of authenticating with the API. You can manage your tokens at the DigitalOcean Control Panel Applications Page.

target "do-droplets" {
  driver = "do-droplets"
  config = {
    token = "local/token"
  }
}
  • token (string: "") - a DigitalOcean API token or a path to a file containing a token. Alternatively, this can also be specified using environment variables ordered by precedence:
    • DIGITALOCEAN_TOKEN
    • DIGITALOCEAN_ACCESS_TOKEN

Policy Configuration Options

check "hashistack-allocated-cpu" {
  # ...
  target "do-droplets" {
    name                = "hashi-worker"
    region              = "nyc1"
    size                = "s-1vcpu-1gb"
    snapshot_id         = 84589509
    user_data           = "local/hashi-worker-user-data.sh"
    tags                = "hashi-stack"
    node_class          = "hashistack"
    node_drain_deadline = "5m"
    node_purge          = "true"
  }
  # ...
}
  • name (string: <required>) - A logical name of a Droplet "group". Every managed Droplet will be tagged with this value and its name is this value with a random suffix

  • region (string: <required>) - The region to start in.

  • vpc_uuid (string: <required>) - The ID of the VPC where the Droplet will be located.

  • size (string: <required>) - The unique slug that indentifies the type of Droplet. You can find a list of available slugs on DigitalOcean API documentation.

  • snapshot_id (string: <required>) - The Droplet image ID.

  • user_data (string: "") - A string of the desired User Data for the Droplet or a path to a file containing the User Data

  • ssh_keys (string: "") - A comma-separated list of SSH fingerprints to enable

  • tags (string: "") - A comma-separated list of additional tags to be applied to the Droplets.

  • datacenter (string: "") - The Nomad client datacenter identifier used to group nodes into a pool of resource. Conflicts with node_class.

  • node_class (string: "") - The Nomad client node class identifier used to group nodes into a pool of resource. Conflicts with datacenter.

  • node_drain_deadline (duration: "15m") The Nomad drain deadline to use when performing node draining actions. Note that the default value for this setting differs from Nomad's default of 1h.

  • node_drain_ignore_system_jobs (bool: "false") A boolean flag used to control if system jobs should be stopped when performing node draining actions.

  • node_purge (bool: "false") A boolean flag to determine whether Nomad clients should be purged when performing scale in actions.

  • node_selector_strategy (string: "least_busy") The strategy to use when selecting nodes for termination. Refer to the node selector strategy documentation for more information.

You might also like...
Copy your HashiCorp Vault secrets to a file

Vault Backup ⚠️ Check the oficial way to backup your HashiCorp Vault. Create a backup file of all HashiCorp Vault kv2 secrets. ./vault-backup -help

Terraform Provider for Latest HashiCorp Product Versions

terraform-provider-hashicorpversions The purpose of this Terraform provider is to get the latest semantic version of any of the suite of HashiCorp too

Pulumi-hcp - A Pulumi provider for interacting with the Hashicorp Cloud Platform

Terraform Bridge Provider Boilerplate This repository contains boilerplate code

VaultOperator provides a CRD to interact securely and indirectly with secrets stored in Hashicorp Vault.

vault-operator The vault-operator provides several CRDs to interact securely and indirectly with secrets. Details Currently only stage 1 is implemente

Kubectl Locality Plugin - A plugin to get the locality of pods

Kubectl Locality Plugin - A plugin to get the locality of pods

Drone plugin for trigger Jenkins jobs.
Drone plugin for trigger Jenkins jobs.

drone-jenkins Drone plugin for trigger Jenkins jobs. Setup the Jenkins Server Setup the Jenkins server using the docker command: $ docker run \ --na

A kubernetes plugin which enables dynamically add or remove GPU resources for a running Pod
A kubernetes plugin which enables dynamically add or remove GPU resources for a running Pod

GPU Mounter GPU Mounter is a kubernetes plugin which enables add or remove GPU resources for running Pods. This Introduction(In Chinese) is recommende

Dothill (Seagate) AssuredSAN dynamic provisioner for Kubernetes (CSI plugin).

Dothill-csi dynamic provisioner for Kubernetes A dynamic persistent volume (PV) provisioner for Dothill AssuredSAN based storage systems. Introduction

Fast docker image distribution plugin for containerd, based on CRFS/stargz
Fast docker image distribution plugin for containerd, based on CRFS/stargz

[ ⬇️ Download] [ 📔 Browse images] [ ☸ Quick Start (Kubernetes)] [ 🤓 Quick Start (nerdctl)] Stargz Snapshotter Read also introductory blog: Startup C

Comments
  • Plugin is trying to check the cluster status after draining the nodes

    Plugin is trying to check the cluster status after draining the nodes

    Hi @jsiebens

    First of all, thank you so much for creating this plugin.

    We are testing this plugin to scale droplets in DigitalOcean. The droplet is getting scaled as per the defined policy but after scaling down and draining the node the plugin starts giving the below warning continuously. The plugin is actually trying to monitor the health of the drained droplet and also the droplet is not getting removed from DigitalOcean.

    2022-09-30T13:55:08.537Z [WARN] external_plugin.do-droplets: node pool status readiness check failed: error="node c92595ac-08f2-7686-75fc-5fb90e29f478 is ineligible" timestamp=2022-09-30T13:55:08.537Z

    Scaling policy:

    scaling "app-redis-slave" {
      enabled = true
      min     = 1
      max     = 2
    
      policy {
        cooldown            = "1m"
        evaluation_interval = "10s"
    
        check "redis_jobs_in_progess" {
          source = "prometheus"
          query  = "sum(nomad_nomad_job_summary_queued{exported_job=~\"app\",task_group=\"redis-slave\"} + nomad_nomad_job_summary_running{exported_job=~\"app\",task_group=\"redis-slave\"}) OR on() vector(0)"
    
          strategy "pass-through" {}
        }
    
        target "do-droplets" {
          name = "app-redis-slave"
          region = "xxx1"
          size = "s-1vcpu-1gb"
          snapshot_id = "XXXXXX"
          user_data = "local/do-user_data.sh"
          tags = "hashi-stack,job:app,group:redis-slave"
    
          datacenter             = "do"
          node_drain_deadline    = "15m"
          node_drain_ignore_system_jobs = true
          node_selector_strategy = "empty_ignore_system"
        }
      }
    }
    
    opened by patademahesh 0
Releases(v0.2.0)
Owner
Johan Siebens
Johan Siebens
Digitalocean-kubernetes-challenge - Deploy a GitOps CI/CD implementation

DigitalOcean Kubernetes Challenge 2021 I chose to participate in the DigitalOcean Kubernetes Challenge in order to learn more about Kubernetes and to

Engin Diri 2 Nov 9, 2022
Annotated and kubez-autoscaler-controller will maintain the HPA automatically for kubernetes resources.

Kubez-autoscaler Overview kubez-autoscaler 通过为 deployment / statefulset 添加 annotations 的方式,自动维护对应 HorizontalPodAutoscaler 的生命周期. Prerequisites 在 kuber

null 138 Jan 2, 2023
General Pod Autoscaler(GPA) is a extension for K8s HPA, which can be used not only for serving, also for game.

Introduction General Pod Autoscaler(GPA) is a extension for K8s HPA, which can be used not only for serving, also for game. Features Compatible with a

Open Cloud-native Game-application Initiative 15 Aug 19, 2022
An example of Kubernetes' Horizontal Pod Autoscaler using costume metrics.

Kubernetes Autoscaling Example In this project, I try to implement Horizontal Pod AutoscalerHPA provided by Kubernetes. The Horizontal Pod Autoscaler

Jaskeerat Singh Randhawa 9 Dec 1, 2022
A plugin for Hashicorp Vault to create ephemeral users and API tokens for Jenkins CI

vault-plugin-secrets-jenkins This is a backend plugin to be used with Hashicorp Vault. This plugin generates ephemeral Jenkins Users and API tokens. v

Caleb Lemoine 6 Dec 15, 2022
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.

HashiCorp 13k Jan 5, 2023
The Operator Pattern, in Nomad

Nomad Operator Example Repostiory to go along with my The Operator Pattern in Nomad blog post. Usage If you have tmux installed, you can run start.sh

Andy Davies 9 May 12, 2022
ArgoCD is widely used for enabling CD GitOps. ArgoCD internally builds manifest from source data in Git repository, and auto-sync it with target clusters.

ArgoCD Interlace ArgoCD is widely used for enabling CD GitOps. ArgoCD internally builds manifest from source data in Git repository, and auto-sync it

International Business Machines 62 Dec 14, 2022
Display (Namespace, Pod, Container, Primary PID) from a host PID, fails if the target process is running on host

Display (Namespace, Pod, Container, Primary PID) from a host PID, fails if the target process is running on host

K8s-school 13 Oct 17, 2022
Dependency management solution for Hashicorp Terraform modules

TERRADEP This is the module dependency solution for implementing terraform's modules dependency. Using this, users can now manage dependencies both fr

Tejaswi Kasat 1 Dec 21, 2021