Get cloud instances with your favourite software pre-loaded

Overview

cloud-provision

Get cloud instances with your favourite software pre-loaded

This Golang package can be used to provision cloud hosts using a simple CRUD-style API along with a cloud-init user-data script. It could be used to automate anything from k3s clusters, to blogs, or CI runners. We use it to create the cheapest possible hosts in the cloud with a public IP address.

provision.go

type Provisioner interface {
	Provision(BasicHost) (*ProvisionedHost, error)
	Status(id string) (*ProvisionedHost, error)
	Delete(HostDeleteRequest) error
}

Where is this package used?

Feel free to send a PR to add your project

This package is used by:

  • inletsctl - Go CLI to create/delete exit-servers and inlets/-pro tunnels
  • inlets-operator - Kubernetes operator to automate exit-servers and inlets/-pro tunnels via CRDs and Service definitions

Try an example program

The tester app takes in a cloud-config file and provisions a host with Nginx - polling until it is ready for access.

#cloud-config
packages:
  - nginx
runcmd:
  - systemctl enable nginx
  - systemctl start nginx

See the example here: Tester app

Rules for adding a new provisioner

The first rule about the provision package is that we don't do SSH. Key management and statefulness are out of scope. Cheap servers should be treated like cattle, not pets. ssh may well be enabled by default, but is out of scope for management. For instance, with DigitalOcean, you can get a root password if you need to log in. Configure as much as you can via cloud-init / user-data.

  • Use the Ubuntu 16.04 LTS image
  • Select the cheapest plan and update the README with the estimated monthly cost
  • For inlets OSS open just the required ports
  • For inlets-pro you must open all ports since the client advertises, not the server
  • This API is event-driven and is expected to use polling from the Kubernetes Operator or inletsctl CLI, not callbacks or waits
  • Do not use any wait or blocking calls, all API calls should return ideally within < 1s
  • Document how you chose any image or configuration, so that the code can be maintained, so that means links and // comments
  • All provisioning code should detect the correct "status" for the provider and set the standard known value
  • Always show your testing in PRs.

Finally please add an example to the documentation for your provider in the inlets/docs repo.

If you would like to add a provider please propose it with an Issue, to make sure that the community are happy to accept the change, and to maintain the code on an ongoing basis.

Maintainers for each provider

Comments
  • Scaleway STARDUST1-S Instance Type Support

    Scaleway STARDUST1-S Instance Type Support

    Scaleway now has a STARDUST1-S Instance Type "STARDUST1-S" 1 vCPU, 1 GB of RAM, 1 IPv4 address, 10GB storage, up-to 100Mbps Bandwidth. €0.0025/hour.

    Having support for this would be by far the lowest cost option to use with inlets for European customers. It would be nice to add support for Amsterdam regions as well.

    opened by emolitor 12
  • Add OVHcloud as new cloud provisioner

    Add OVHcloud as new cloud provisioner

    This PR introdcue OVHcloud as new cloud provisoner for inlets*

    OVH

    https://www.ovh.com/

    OVH, legally OVH Groupe SAS, is a French cloud computing company which offers VPS, dedicated servers and other web services. As of 2016 OVH owned the world's largest data center in surface area. As of 2019, it was the largest hosting provider in Europe, and the third largest in the world based on physical servers.[1]

    Signed-off-by: Engin Diri [email protected]

    [1] https://en.wikipedia.org/wiki/OVHcloud

    opened by dirien 8
  • GCE: the iam.serviceAccountUser role is unused

    GCE: the iam.serviceAccountUser role is unused

    The GCE provisioner requires a GCP service account with two roles:

    • roles/compute.admin
    • roles/iam.serviceAccountUser

    The first role makes sense, but the second one was a surprise to me. Looking at gce.go, I discovered that the VM is created with a "service account VM" (a service account that is mounted into the VM and allows for GCP API calls):

    https://github.com/inlets/cloud-provision/blob/9626406375488183630d2b3904deeaf12445531e/provision/gce.go#L94-L101

    Should we drop this mounted service account and update the documentation (see sections inlets-operator and inletsctl) to only require roles/compute.admin?

    opened by maelvls 2
  • Linode: Remove 'inlets-' prefix on label

    Linode: Remove 'inlets-' prefix on label

    The Linode provider add a prefix to the label

    https://github.com/inlets/cloud-provision/blob/f58cc468f6b080abbf246d88415aee7eefd52406/provision/linode.go#L90

    which could conflict with the max lable length of the Linode API (https://www.linode.com/docs/api/linode-instances/)

    Should be removed and to be sure trimmed down to 32 chars.

    opened by dirien 1
  • Equinix Metal Provisiner Errors

    Equinix Metal Provisiner Errors

    Hi,

    i get following errors, when using the Equinix Metal Provisioner:

    1. Default Region am1 does not exist
    Using provider: equinix-metal
    Requesting host: reverent-khayyam0 in ams1, from equinix-metal
    2021/07/27 06:31:29 [DEBUG] POST https://api.equinix.com/metal/v1/projects/xx/devices
    POST https://api.equinix.com/metal/v1/projects/xxx/devices: 422 ams1 is not a valid facility 
    
    1. You can not create an Equinix Metal Server without an SSH Key anymore:
    Using provider: equinix-metal
    Requesting host: magical-johnson8 in fr2, from equinix-metal
    2021/07/27 06:36:30 [DEBUG] POST https://api.equinix.com/metal/v1/projects/xxx/devices
    POST https://api.equinix.com/metal/v1/projects/xxx/devices: 422 must have at least one SSH key or explicitly send no_ssh_keys option 
    

    See -> https://feedback.equinixmetal.com/platform/p/allow-to-deploy-servers-without-ssh-key And new API for the no_ssh_keys option -> https://metal.equinix.com/developers/api/devices/#devices-createdevice

    For issue 2 we need to update the Go lib -> 0.17.0

    opened by dirien 1
  • Add sessionToken param to NewEC2Provisioner

    Add sessionToken param to NewEC2Provisioner

    Currently the session token field is defaulted to "". This makes it impossible to use temporary session creds in inletsctl as AWS expects key, secret & token.

    This change adds a new parameter to NewEC2Provisioner to pass the session token through which is subsequenty used in NewStaticCredentials.

    Signed-off-by: Richard Gee [email protected]

    new-contributor 
    opened by rgee0 1
  • Add delete by IP on OVHcloud provisioner

    Add delete by IP on OVHcloud provisioner

    Signed-off-by: Engin Diri [email protected]

    Valuable Feedback from @jsiebens

    @dirien @alexellis fyi, deleting an instance by IP instead of ID is missing in this new provider

    in the #10 PR.

    This PR adds the missing functionality.

    opened by dirien 0
  • Add Network and Region to instance

    Add Network and Region to instance

    To create a civo instance, you need to assign a default network to it. Otherwise, it will not create the instance.

    In addition, we need to handle the region too, as different regions have different default networks.

    This issue popped up during handling the https://github.com/inlets/inletsctl/issues/95

    Signed-off-by: Engin Diri [email protected]

    new-contributor 
    opened by dirien 0
  • Linode: Remove Startupscripts

    Linode: Remove Startupscripts

    When deleting an exit sever on linode, we should remove the startupscript too.

    If a sever can not be created, for whatever reasons we should also delete any orphan startupscripts. image

    opened by dirien 0
  • EC2, Enable authentication using default credential chain

    EC2, Enable authentication using default credential chain

    Signed-off-by: Tim Luimes [email protected]

    Enables the ability for the ec2 client session to be authenticated using methods provided by the aws SDK's default credential chain when the accessKey and secretKey aren't provided.

    https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html "When you initialize a new service client without providing any credential arguments, the SDK uses the default credential provider chain to find AWS credentials. The SDK uses the first provider in the chain that returns credentials without an error. The default provider chain looks for credentials in the following order:

    Environment variables.

    Shared credentials file.

    If your application uses an ECS task definition or RunTask API operation, IAM role for tasks.

    If your application is running on an Amazon EC2 instance, IAM role for Amazon EC2."

    new-contributor no-dco 
    opened by tluimes 1
  • [Suggestion] Remove inlets specific references from the provisioners

    [Suggestion] Remove inlets specific references from the provisioners

    At this moment, many provisioners have some specific, hard-coded references to inlets. Like tags or firewall rules. Referring to this repository: https://github.com/inlets/provision-cloudinit-go, removing those references and making the provisioners more generic, they can be used as a library by others.

    Expected Behaviour

    Most provisioners create cloud host with hard-coded inlets references, like tags.

    Current Behaviour

    The provisioners can be used by others without creating inlets tags.

    Possible Solution

    Pass the inlets references to the provisioners as arguments. I'm aware this can be a tedious task, as there are many provisioners and they all have a target cloud specific implementation.

    Steps to Reproduce (for bugs)

    N/A

    Context

    Playing around with the provisioners as a library, I've noticed that the cloud hosts are tagged with "inlets"., while I was creating other, non-inlets relared hosts.

    Your Environment

    • inlets version inlets --version N/A
    • Docker/Kubernetes version docker version / kubectl version: N/A
    • Operating System and version (e.g. Linux, Windows, MacOS): N/A
    • Link to your project or a code example to reproduce issue: https://github.com/inlets/provision-cloudinit-go
    opened by jsiebens 3
Releases(v0.6.0)
  • v0.5.5(Aug 26, 2021)

    Changelog for v0.5.5:

    Commits 8d3ba22fab5757e7bb1fe9f1bbaefe754279511b Add validation on name to exit early by @alexellis a03b98f6d0387ded5f12432cf284848698d5f252 Remove linode label prefix by @alexellis

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • v0.5.4(Aug 25, 2021)

    Changelog for v0.5.4:

    • PR #15 Update govultr sdk to v2 by @dirien

    Commits f58cc468f6b080abbf246d88415aee7eefd52406 Update govultr sdk to v2 by @dirien

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • v0.5.3(Aug 4, 2021)

    Changelog for v0.5.3:

    • PR #14 Add delete by IP on OVHcloud provisioner by @dirien

    Commits 5ffecf43cc4d6a07e8a9dfb7c7348b4aa8823355 Add delete by IP on OVHcloud provisioner by @dirien

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • v0.5.2(Jul 29, 2021)

    Changelog for v0.5.2:

    • PR #10 Add OVHcloud as new cloud provisioner by @dirien

    Commits a67b248b2507d75eeb18699c3e5afb3798e9a1c9 Add OVHCloud as new cloud provisioner by @dirien ef6c3face211607d7d261ee753cd4250da4995fa Update README.md by @alexellis

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Jul 27, 2021)

    Changelog for v0.5.1:

    • PR #13 Fix Equinix Metal Provisiner Errors by @dirien

    Commits 0d5db0ba28e3e8fa102d690001326e5c2aa9af34 Fix Equinix Metal Provisiner Errors by @dirien

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • v0.5.0-rc2(Jul 17, 2021)

  • v0.5.0-rc1(Jul 17, 2021)

    Changelog for v0.5.0-rc1:

    Commits c319b474f1165f15678fc1046bb18dbffd0a85ea Update tester cmd by @alexellis b3debaed2e0894b768a8a668e2a21bf917cae2d1 Move go module to top level of repo by @alexellis

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Jul 17, 2021)

    Changelog for v0.4.0:

    • PR #8 Add sessionToken param to NewEC2Provisioner by @rgee0
    • PR #7 fix linode and userdata tests by @jsiebens

    Commits 016f5231b09e8794300dead21ee1a36640575ad9 Add sessionToken param to NewEC2Provisioner by @rgee0 7c57c2d9d2d8b93a195745355f038499d270ad2a fix linode and userdata tests by @jsiebens

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • 0.3.2(Jul 7, 2021)

    Changelog for 0.3.2:

    • PR #5 make service account optional by @jsiebens

    Commits 93cc13af558aa9a0f9f3d97b1db5c7fb522e4a77 make service account optional by @jsiebens

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • 0.3.1(Jul 7, 2021)

    Changelog for 0.3.1:

    • PR #6 Add Network and Region to instance by @dirien

    Commits 1aa1adf1e11a6f04b8c471f46d17637cba9ea2c5 Move to use Unmarshal instead of Decode by @alexellis d4bc7c194600b1cc7933d956ced5e78beac47150 Add Network and Region to instance by @dirien

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • 0.3.0(Feb 12, 2021)

    Changelog for 0.3.0:

    Commits 9626406375488183630d2b3904deeaf12445531e Download unit file from releases by @alexellis e6888300f19ea903e16f1a86dc103a44a1cd3337 Add two files for provisioning examples by @alexellis

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
  • 0.2.0(Feb 8, 2021)

    Changelog for 0.2.0:

    • PR #1 Add systemctl daemon-reload by @frezbo

    Commits ccdeae5973dc30a2d7892ca2a9e8eccf0fbe8f01 Update userdata for inlets to use pinned version by @alexellis 7c78922fb3f6ff3c66f36b99bba4116604b88df8 Add systemctl daemon-reload by @frezbo db1d3bea7e03098e8ccd59b5f231d70488505dfa Create .DEREK.yml by @alexellis 392a2a52884ad44fef11064d5c9157ac3f831be5 Update README.md by @alexellis 2e89567515b78db72d6c3ef05ea805b097be7c07 Update README.md by @alexellis

    Generated by Derek

    Source code(tar.gz)
    Source code(zip)
Owner
inlets
The Cloud Native Tunnel
inlets
Cell is a Go package that creates new instances by string in running time.

Cell Cell is a Go package that creates new instances by string in running time. Getting Started Installing To start using CELL, install Go and run go

null 0 Dec 20, 2021
A comprehensive list of alternatives to your favorite software

alternativeto A comprehensive list of alternatives to your favorite software. Please do not edit this file directly. Instead, follow the steps outline

Brent Adamson 33 Jun 16, 2022
✔️ Get random data for your app from a third-party source.

Random Data Securely produced random data for application testing. FAQ What would i use this data for? You can use this information to test your apps

ComplexOrganizations 1 Jul 5, 2022
Easy way to get credentials of your clusters in GCP and stay connected

Easy way to get credentials of your clusters in GCP and stay connected. Don't lose time with alias or annotations with projects names and clusters. Use k8skey

Antonio Thomacelli Gomes 1 Nov 9, 2021
Project developed for the course Software Systems Analysis and Design (SSAD) at IU in F21 semester.

Go knowledge yield summary Project description Project developed for the course Software Systems Analysis and Design (SSAD) at IU in F21 semester. Eva

null 28 Sep 17, 2022
System software of computers

SSoC System software of computers BSUIR labs Project structure Client client client/components client/components/client client/components/command clie

Max Ersh 2 Jan 5, 2022
Graceful - shutdown package when a service is turned off by software function

graceful Graceful shutdown package when a service is turned off by software func

Bo-Yi Wu 38 Sep 19, 2022
this allows you to get the real link of bit.ly

check the real url from a url shortener (bit.ly) Also you can use it as an API example with deno const rawResponse = await fetch("https://anti-url-s

pai 10 Feb 19, 2022
redis-util business-friendly encapsulation of redis operations, such as the common cache set get operation

redis-util 方便业务使用的redis操作封装,比如常见的缓存set get操作, 一行代码搞定,不像开源库需要写好多行 使用方法

connor 1 Oct 22, 2021
Simple utility to get/set the PWM duty cycle and to measure the RPM for a fan connected to the 4-pin header on the CM4IO.

cm4iofan Simple utility to get/set the PWM duty cycle and to measure the RPM for a fan connected to the 4-pin header on the CM4IO. Requirements Enable

Thomas Maier 5 Mar 31, 2022
get windows system call number dynamically, it doesn't need the system call table.

GetSyscallID get windows system call number dynamically, it doesn't need the system call table. Try download go get github.com/akkuman/getsyscallid/cm

null 7 Jun 1, 2022
GoApiRandom - Api to get random numbers

GoApiRandom - Api to get random numbers

bdobrosielski 0 Jan 18, 2022
Hotswap provides a solution for reloading your go code without restarting your server, interrupting or blocking any ongoing procedure.

Hotswap provides a solution for reloading your go code without restarting your server, interrupting or blocking any ongoing procedure. Hotswap is built upon the plugin mechanism.

Edwin 115 Sep 21, 2022
gopkg is a universal utility collection for Go, it complements offerings such as Boost, Better std, Cloud tools.

gopkg is a universal utility collection for Go, it complements offerings such as Boost, Better std, Cloud tools. Table of Contents Introduction

Bytedance Inc. 847 Sep 16, 2022
流媒体NetFlix解锁检测脚本 / A script used to determine whether your network can watch native Netflix movies or not

netflix-verify 流媒体NetFlix解锁检测脚本,使用Go语言编写 在VPS网络正常的情况下,哪怕是双栈网络也可在几秒内快速完成IPv4/IPv6的解锁判断 鸣谢 感谢 @CoiaPrant 指出对于地域检测更简便的方法 感谢 @XmJwit 解决了IPV6 Only VPS无法下载脚

null 1.8k Sep 22, 2022
Visualize your Go data structures using graphviz

memviz How would you rather debug a data structure? "Pretty" printed Visual graph (*test.fib)(0xc04204a5a0)({ index: (int) 5, prev: (*test.fib)(0xc0

Bradley Kemp 1.3k Sep 17, 2022
Dynamically generated Last.fm stats for your profile readme

GitHub Readme Last.fm Stats Dynamically generated last.fm stats in your profile readme Contents Usage Options Demo Development & Deployment Issues, Re

Rafael Wiska-Ilnicki 6 Mar 17, 2022
gProfiler combines multiple sampling profilers to produce unified visualization of what your CPU

gProfiler combines multiple sampling profilers to produce unified visualization of what your CPU is spending time on, displaying stack traces of your processes across native programs1 (includes Golang), Java and Python runtimes, and kernel routines.

Granulate 548 Sep 23, 2022
this is an api that execute your deno code and send you the output

this a simple api that execute your deno code and send you the output, has not limit per request example request: in deno: const rawResponse = await f

pai 13 May 28, 2022