Environment variables substitution for Go

Overview

envsubst

GoDoc License Build status Github All Releases

Environment variables substitution for Go. see docs below

Installation:

From binaries

Latest stable envsubst prebuilt binaries for 64-bit Linux, or Mac OS X are available via Github releases.

Linux and MacOS
curl -L https://github.com/a8m/envsubst/releases/download/v1.2.0/envsubst-`uname -s`-`uname -m` -o envsubst
chmod +x envsubst
sudo mv envsubst /usr/local/bin
Windows

Download the latest prebuilt binary from releases page, or if you have curl installed:

curl -L https://github.com/a8m/envsubst/releases/download/v1.2.0/envsubst.exe
With go

You can install via go get (provided you have installed go):

go get github.com/a8m/envsubst/cmd/envsubst

Using via cli

envsubst < input.tmpl > output.text
echo 'welcome $HOME ${USER:=a8m}' | envsubst
envsubst -help

Imposing restrictions

There are three command line flags with which you can cause the substitution to stop with an error code, should the restriction associated with the flag not be met. This can be handy if you want to avoid creating e.g. configuration files with unset or empty parameters. Setting a -fail-fast flag in conjunction with either no-unset or no-empty or both will result in a faster feedback loop, this can be especially useful when running through a large file or byte array input, otherwise a list of errors is returned.

The flags and their restrictions are:

Option Meaning Type Default
-i input file ```string stdin```
-o output file ```string stdout```
-no-unset fail if a variable is not set flag false
-no-empty fail if a variable is set but empty flag false
-fail-fast fails at first occurence of an error, if -no-empty or -no-unset flags were not specified this is ignored flag false

These flags can be combined to form tighter restrictions.

Using envsubst programmatically ?

You can take a look on _example/main or see the example below.

package main

import (
	"fmt"
	"github.com/a8m/envsubst"
)

func main() {
    input := "welcom $HOME"
    str, err := envsubst.String(input)
    // ...
    buf, err := envsubst.Bytes([]byte(input))
    // ...
    buf, err := envsubst.ReadFile("filename")
}

Docs

api docs here: GoDoc

Expression Meaning
${var} Value of var (same as $var)
${var-$DEFAULT} If var not set, evaluate expression as $DEFAULT
${var:-$DEFAULT} If var not set or is empty, evaluate expression as $DEFAULT
${var=$DEFAULT} If var not set, evaluate expression as $DEFAULT
${var:=$DEFAULT} If var not set or is empty, evaluate expression as $DEFAULT
${var+$OTHER} If var set, evaluate expression as $OTHER, otherwise as empty string
${var:+$OTHER} If var set, evaluate expression as $OTHER, otherwise as empty string
$$var Escape expressions. Result will be $var.

Most of the rows in this table were taken from here

See also

  • os.ExpandEnv(s string) string - only supports $var and ${var} notations

License

MIT

Issues
  • support for restriction flags -no-unset and -no-empty

    support for restriction flags -no-unset and -no-empty

    I added two flags -no-unset and -no-empty to the envsubst command, with which you can cause the substitution to stop with an error code, should the restriction associated with the flag not be met. This can be handy if you want to avoid creating e.g. configuration files with unset or empty parameters. The flags and their restrictions are:

    |Flag | Meaning | | ------------| -------------- | |-no-unset | fail if a variable is not set |-no-empty | fail if a variable is set but empty

    These flags can be combined to form tighter restrictions.

    Sorry about the fairly large change set...

    opened by xelalexv 7
  • Add actions to test and build binaries for x86_64/arm64

    Add actions to test and build binaries for x86_64/arm64

    Hi!

    We really need to build this binary for arm64 (we're switching to m1 macs) and I figured it wouldn't be fair to ask you to put any work in so I whipped this up. This PR:

    • Tests on Go 1.16/1.17 in github actions
    • Automatically builds linux/darwin x86_64/arm64 binaries whenever you cut a release

    I'd love it if you'd merge this and cut a release :)

    opened by apenney 5
  • Feature Request: escape `$$`

    Feature Request: escape `$$`

    Thank you so much for this project, the -no-unset option was exactly what I was looking for, I've immediately replaced our usage of gettext with it.

    Now that I'm a user, would it be possible for envsubst to replace all strings $$foo with $foo?

    In my case, I have kubernetes templates for nginx configuration, which I template with some values, but which also contain nginx variables like $remote_user and $bytes_sent strings. My current choices are either to not use -no-unset, or (what I actually do) implement my own escaping with sed.

    good first issue 
    opened by nicknovitski 3
  • Release binaries by GitHub action

    Release binaries by GitHub action

    Turns out this was an upstream breakage and I had to upgrade the action (they fixed it 5 hours ago!), just terrible timing on the merge and I apologize for you wasting a bunch of time on this. Since the upgrade I can run the action in my fork just fine with no failures, so I think we're good again.

    opened by apenney 2
  • feat: add default behaviour of running through all the errors

    feat: add default behaviour of running through all the errors

    #22

    Added a flag to override the defualt - fail-fast will enable users to fail fast if required otherwise all errors are displayed in non relaxed modes

    image

    opened by dnitsch 2
  • Restrictions only restrict when variable inside braces

    Restrictions only restrict when variable inside braces

    If this is intended, apologies... I didn't see it documented.

    package main
    
    import (
    	"log"
    	"github.com/a8m/envsubst"
    )
    
    func main() {
    	process("Hello $name, how are you?")
    	process("Hello ${name}, how are you?")
    }
    
    func process(tpl string) {
    	out, err := envsubst.StringRestricted(tpl, true, true)
    	if err != nil {
    		panic(err)
    	}
    	log.Println("Shouldn't be here - " + out)
    }
    
    bug help wanted good first issue 
    opened by elliots 2
  • Need a simple way to escape dollar symbols

    Need a simple way to escape dollar symbols

    There needs to be a way to insert dollar symbols in a template Instead of creating a variable (E.G. DOLLAR=\$).

    For instance if envsubst was used in a bash script, replacing all the dollars $ with a variable will be messy.

    opened by dexterp 1
  • Add support for full error list

    Add support for full error list

    I would like to add support for returning a full list of errors and optionally stop execution on first error. This will be very handy in more complex CI/CD scenarios where config files have a large number of variables to be replaced.

    opened by dnitsch 1
  • Escaping syntax

    Escaping syntax

    Hi,

    nice little helper tool;) Although i recently stumbled over a problem i couldn't find a solution to.. How can i escape sth. like this:

    A=${static}
    

    I know about $${static} -> $static. But what if i need the {} ?

    regards, Roman

    opened by strowi 1
  • Infinite loop

    Infinite loop

    When using a default value that will have a : somewhere in the text, envsubst will go to infinite loop. For example:

    $ echo '${SOME_URL-http://localhost:8080}' | envsubst
    

    If I remove the quotes or add a double quotes it will work fine:

    $ echo ${SOME_URL-http://localhost:8080} | envsubst
    or
    $ echo "${SOME_URL-http://localhost:8080}" | envsubst
    

    However, if I want to replace the same env variable in the file it will be stuck:

    $ cat file.txt
    property=${SOME_URL-http://localhost:8080}
    
    $ envsubst <file.txt >file2.txt
    
    opened by komljen 1
  • Last line missing in output if not terminated with newline

    Last line missing in output if not terminated with newline

    When the last line in the input is not terminated with \n, then it is missing in the produced output, e.g. this input (\n added for clarity):

    bar="foo"\n
    foo="bar"
    

    will be output by envsubst as:

    bar="foo"\n
    

    Output should be:

    bar="foo"\n
    foo="bar"
    
    opened by xelalexv 1
  • Feature for selected Environments vars

    Feature for selected Environments vars

    This PR intends to add a new feature for selecting which env vars the caller wants to lookup and replace while using the cli or programmatically.

    I believe it addresses what was requested in https://github.com/a8m/envsubst/issues/25.

    Summary

    • A new -envs added for listing what env vars we want to lookup and replace.
      echo 'hi ${USER} at ${PWD}' |envsubst -envs ENV,USER
      > hi myuser at ${PWD}
      
    • Add new funcs for selecting envs programmatically:
      • StringSelectedEnvs envsubst.StringSelectedEnvs(input, []string{"ENV1"})
      • StringRestrictedSelectedEnvs envsubst.StringRestrictedSelectedEnvs(input, true, false, []string{"ENV1"})
      • BytesSelectedEnvs envsubst.BytesSelectedEnvs([]byte(input), []string{"ENV1"})
      • BytesRestrictedSelectedEnvs envsubst.BytesRestrictedSelectedEnvs([]byte(input), false, true, []string{"ENV1"})
      • ReadFileSelectedEnvs envsubst.ReadFileSelectedEnvs("filename", []string{"ENV1"})
      • ReadFileRestrictedSelectedEnvs envsubst.ReadFileRestrictedSelectedEnvs("filename", true, true, []string{"ENV1"})
    • Add more tests
    opened by renato-farias 0
  • Add option to only parse $ALL_CAPS env vars

    Add option to only parse $ALL_CAPS env vars

    Especially for nginx configs, lots of directives use $lowercase_variable, so having to escape a lot of content.

    It would be nice to have an caps-only flag to only detect env vars that are uppercase.

    opened by vladnosek 2
  • Feature Request / Possible bug: Support nested variables enclosed by raw string

    Feature Request / Possible bug: Support nested variables enclosed by raw string

    I have a weird issue where envsubst is not working as expected.

    When I add only a single a default variable to the right side of the experssion it will work similarly as a default POSIX shell. But when I add additional text to the default it start to do some strange behaviour.

    I'll provide some examples, I believe that will be the best description.

    sh use case:

    docker run --rm -it debian:buster-slim sh
    # export VAR1="shouldevaluate"
    # export EMPTY_VAR=""
    # export MAIN_VAR="${EMPTY_VAR:-$VAR1}"
    # echo $MAIN_VAR
    shouldevaluate
    #
    #
    # export MAIN_VAR="${EMPTY_VAR:-asdqwe$VAR1}"
    # echo $MAIN_VAR
    asdqweshouldevaluate
    #
    #
    # export MAIN_VAR="${EMPTY_VAR:-asdqwe $VAR1 qweasd}"
    # echo $MAIN_VAR
    asdqwe shouldevaluate qweasd
    #
    #
    # export MAIN_VAR="${EMPTY_VAR:-asdqwe${VAR1}qweasd}"
    # echo $MAIN_VAR
    asdqweshouldevaluateqweasd
    

    And this happens when I try to use the same method in envsubst.

    # cat test.template
    name = ${EMPTY_VAR:-$VAR1}
    
    name = ${EMPTY_VAR:-asdqwe$VAR1}
    
    name = ${EMPTY_VAR:-asdqwe $VAR1 qweasd}
    
    name = ${EMPTY_VAR:-asdqwe${VAR1}qweasd}
    #
    #
    # envsubst < test.template
    name = shouldevaluate
    
    name = asdqwe$VAR1
    
    name = asdqwe $VAR1 qweasd
    
    name = asdqwe${VAR1qweasd}
    
    opened by janosmiko 1
  • convert Env to an interface and implement a Functor variant.

    convert Env to an interface and implement a Functor variant.

    when dealing with a large environment (i.e. lots of environment variables), the slice based lookup becomes inperformant. os also allows variable lookups based on keys, which the Functor implementation uses for its core logic.

    opened by UiP9AV6Y 0
  • Support ignore and keep empty system environments

    Support ignore and keep empty system environments

    e.g I have prometheus config below which want only replace CONSUL_URL from system environment. but it will replace $1 left blank there which make metrics_path not correct.

      - job_name: 'consul'
        metrics_path: '/actuator/prometheus'
        consul_sd_configs:
          - server: '${CONSUL_URL:-127.0.0.1:8500}'
            services: [] #match all service
        relabel_configs:
          - source_labels: [__meta_consul_service]
            target_label: job
          - source_labels: [__meta_consul_service_metadata_management_context_path]
            regex: "(.+)"
            target_label: __metrics_path__
            replacement: '$1/prometheus'
    
    opened by MadeInChina 6
Owner
Ariel Mashraki
Ariel Mashraki
Simple lib to parse environment variables to structs

env Simple lib to parse envs to structs in Go. Example A very basic example: package main import ( "fmt" "time" // if using go modules "github.c

Carlos Alexandro Becker 2.6k Aug 7, 2022
Un-marshaling environment variables to Go structs

envcfg Un-marshaling environment variables to Go structs Getting Started Let's set a bunch of environment variables and then run your go app #!/usr/bi

Tomaž Kovačič 98 May 4, 2022
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Vincent Rischmann 222 May 18, 2022
Go helpers to manage environment variables

Envh This library is made up of two parts : Env object : it wraps your environments variables in an object and provides convenient helpers. Env tree o

Anthony HAMON 95 Apr 14, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Go Sidekick 0 May 30, 2022
A Go port of Ruby's dotenv library (Loads environment variables from `.env`.)

GoDotEnv A Go (golang) port of the Ruby dotenv project (which loads env vars from a .env file) From the original Library: Storing configuration in the

John Barton 5.2k Aug 10, 2022
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Josh Betz 212 Jul 22, 2022
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Kelsey Hightower 4.2k Aug 3, 2022
A Go library for parsing struct tags from environment variables.

Envconfig Envconfig populates struct field values based on environment variables or arbitrary lookup functions. It supports pre-setting mutations, whi

Seth Vargo 655 Aug 4, 2022
Quickly read variables from environment files

go-quick-env Quickly read variables from environment files The best way to import environment variables to your code, is by using .env files. This lib

Panos Petropoulos 3 May 11, 2021
Read files into environment variables and execute command

read-file-to-env -- Read files into environment variables and execute command Example use: read-file-to-env -one-line=HOST=/etc/hostname sh -c 'echo h

Tv 2 Nov 12, 2021
A mapper of ENVironment variables to Structure for Go

envs a mapper of ENVironment variables to a Structure for Go. This library maps the environment variables to the struct according to the fields' types

moznion 3 Dec 3, 2021
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

null 3 Feb 22, 2022
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Sergey Prokhorov 0 Dec 30, 2021
formicidate is a small tool for Go application can update the value of environment variables in a .env file with code

formicidae Update .env files in Go with code. What is fomicidae? formicidate is a small tool for Go application. You can update the value of environme

akuma 0 Jan 23, 2022
Lightweight package that makes easier and safer to deal with environment variables.

Envisage A lightweight package that makes easier and safer to deal with environment variables. Example Try it on On GoPlay https://goplay.tools/snippe

GOLang Sugar 4 Apr 11, 2022
Tmpl - A tool to apply variables from cli, env, JSON/TOML/YAML files to templates

tmpl allows to apply variables from JSON/TOML/YAML files, environment variables or CLI arguments to template files using Golang text/template and functions from the Sprig project.

krako 1 May 30, 2022
✨Clean and minimalistic environment configuration reader for Golang

Clean Env Minimalistic configuration reader Overview This is a simple configuration reading tool. It just does the following: reads and parses configu

Ilya Kaznacheev 686 Aug 8, 2022
Golang Configuration tool that support YAML, JSON, TOML, Shell Environment

Configor Golang Configuration tool that support YAML, JSON, TOML, Shell Environment (Supports Go 1.10+) Usage package main import ( "fmt" "github.c

Jinzhu 1.5k Aug 11, 2022