Golang library for managing configuration data from environment variables

Overview

envconfig

Build Status

import "github.com/kelseyhightower/envconfig"

Documentation

See godoc

Usage

Set some environment variables:

export MYAPP_DEBUG=false
export MYAPP_PORT=8080
export MYAPP_USER=Kelsey
export MYAPP_RATE="0.5"
export MYAPP_TIMEOUT="3m"
export MYAPP_USERS="rob,ken,robert"
export MYAPP_COLORCODES="red:1,green:2,blue:3"

Write some code:

package main

import (
    "fmt"
    "log"
    "time"

    "github.com/kelseyhightower/envconfig"
)

type Specification struct {
    Debug       bool
    Port        int
    User        string
    Users       []string
    Rate        float32
    Timeout     time.Duration
    ColorCodes  map[string]int
}

func main() {
    var s Specification
    err := envconfig.Process("myapp", &s)
    if err != nil {
        log.Fatal(err.Error())
    }
    format := "Debug: %v\nPort: %d\nUser: %s\nRate: %f\nTimeout: %s\n"
    _, err = fmt.Printf(format, s.Debug, s.Port, s.User, s.Rate, s.Timeout)
    if err != nil {
        log.Fatal(err.Error())
    }

    fmt.Println("Users:")
    for _, u := range s.Users {
        fmt.Printf("  %s\n", u)
    }

    fmt.Println("Color codes:")
    for k, v := range s.ColorCodes {
        fmt.Printf("  %s: %d\n", k, v)
    }
}

Results:

Debug: false
Port: 8080
User: Kelsey
Rate: 0.500000
Timeout: 3m0s
Users:
  rob
  ken
  robert
Color codes:
  red: 1
  green: 2
  blue: 3

Struct Tag Support

Envconfig supports the use of struct tags to specify alternate, default, and required environment variables.

For example, consider the following struct:

type Specification struct {
    ManualOverride1 string `envconfig:"manual_override_1"`
    DefaultVar      string `default:"foobar"`
    RequiredVar     string `required:"true"`
    IgnoredVar      string `ignored:"true"`
    AutoSplitVar    string `split_words:"true"`
    RequiredAndAutoSplitVar    string `required:"true" split_words:"true"`
}

Envconfig has automatic support for CamelCased struct elements when the split_words:"true" tag is supplied. Without this tag, AutoSplitVar above would look for an environment variable called MYAPP_AUTOSPLITVAR. With the setting applied it will look for MYAPP_AUTO_SPLIT_VAR. Note that numbers will get globbed into the previous word. If the setting does not do the right thing, you may use a manual override.

Envconfig will process value for ManualOverride1 by populating it with the value for MYAPP_MANUAL_OVERRIDE_1. Without this struct tag, it would have instead looked up MYAPP_MANUALOVERRIDE1. With the split_words:"true" tag it would have looked up MYAPP_MANUAL_OVERRIDE1.

export MYAPP_MANUAL_OVERRIDE_1="this will be the value"

# export MYAPP_MANUALOVERRIDE1="and this will not"

If envconfig can't find an environment variable value for MYAPP_DEFAULTVAR, it will populate it with "foobar" as a default value.

If envconfig can't find an environment variable value for MYAPP_REQUIREDVAR, it will return an error when asked to process the struct. If MYAPP_REQUIREDVAR is present but empty, envconfig will not return an error.

If envconfig can't find an environment variable in the form PREFIX_MYVAR, and there is a struct tag defined, it will try to populate your variable with an environment variable that directly matches the envconfig tag in your struct definition:

export SERVICE_HOST=127.0.0.1
export MYAPP_DEBUG=true
type Specification struct {
    ServiceHost string `envconfig:"SERVICE_HOST"`
    Debug       bool
}

Envconfig won't process a field with the "ignored" tag set to "true", even if a corresponding environment variable is set.

Supported Struct Field Types

envconfig supports these struct field types:

Embedded structs using these fields are also supported.

Custom Decoders

Any field whose type (or pointer-to-type) implements envconfig.Decoder can control its own deserialization:

export DNS_SERVER=8.8.8.8
type IPDecoder net.IP

func (ipd *IPDecoder) Decode(value string) error {
    *ipd = IPDecoder(net.ParseIP(value))
    return nil
}

type DNSConfig struct {
    Address IPDecoder `envconfig:"DNS_SERVER"`
}

Example for decoding the environment variables into map[string][]structName type

export SMS_PROVIDER_WITH_WEIGHT= `IND=[{"name":"SMSProvider1","weight":70},{"name":"SMSProvider2","weight":30}];US=[{"name":"SMSProvider1","weight":100}]`
type providerDetails struct {
	Name   string
	Weight int
}

type SMSProviderDecoder map[string][]providerDetails

func (sd *SMSProviderDecoder) Decode(value string) error {
	smsProvider := map[string][]providerDetails{}
	pairs := strings.Split(value, ";")
	for _, pair := range pairs {
		providerdata := []providerDetails{}
		kvpair := strings.Split(pair, "=")
		if len(kvpair) != 2 {
			return fmt.Errorf("invalid map item: %q", pair)
		}
		err := json.Unmarshal([]byte(kvpair[1]), &providerdata)
		if err != nil {
			return fmt.Errorf("invalid map json: %w", err)
		}
		smsProvider[kvpair[0]] = providerdata

	}
	*sd = SMSProviderDecoder(smsProvider)
	return nil
}

type SMSProviderConfig struct {
    ProviderWithWeight SMSProviderDecoder `envconfig:"SMS_PROVIDER_WITH_WEIGHT"`
}

Also, envconfig will use a Set(string) error method like from the flag.Value interface if implemented.

Comments
  • Primary goals for next release

    Primary goals for next release

    I've been thinking about what to try and get into the next release, and thought I'd put it here for comment.

    What I'm largely focused on is getting to rough parity with stdlib examples of the "use reflect to deserialize into interface{}" -- thinking mainly about encoding/json and database/sql.

    Nested struct support

    This will be especially useful for eliminating boilerplate when configuring option or client structs for libraries. I'd like to get this in soon.

    Pointer field support

    Particularly for nested structs above (fields of pointer-to-struct are so common), but in the general case if we support decoding into a type then it should be pretty straightforward to decode into a *type. In our case it's not as simple as it should be, there's a little refactoring necessary.

    json and sql both do this, and it can be useful for differentiating the zero value of a type from "wasn't set at all" (in which case it remains nil).

    Anything else I'm missing?

    opened by teepark 12
  • convert to a two phase approach and add usage capability

    convert to a two phase approach and add usage capability

    there is now a GatherInformation phase and then two second phases, Process to populate the structure, and Usage to output usage information.

    The GatherInformation and associated structure VarInfo are public / exposed so that additional behavior can be written outside of the core package if desired.

    opened by davidkbainbridge 9
  • Enable usage message to provide runtime documentation of configuration options

    Enable usage message to provide runtime documentation of configuration options

    Most applications today provide a -h or --help command line argument that will display the usage of the command and describe, briefly, the available command line arguments and their default values.

    If an application is configured by its environment there should be a mechanism by which a user can be provided with a usage description at runtime.

    (see discussion for more information: https://github.com/kelseyhightower/envconfig/issues/40)

    opened by davidkbainbridge 8
  • Allow map[string]string to support ':' in its keys and values

    Allow map[string]string to support ':' in its keys and values

    This is useful for urls for either keys or values. As of now, envconfig fails with -- invalid map item at https://github.com/kelseyhightower/envconfig/blob/master/envconfig.go#L274.

    opened by sethcleveland 7
  • Support the general case of nested structs

    Support the general case of nested structs

    #22 added support for embedded nested structs, this proposes supporting nested structs in named fields as well.

    There is a potential name collision issue if we don't do something about the naming convention:

    var Config = struct {
        HTTPClient http.Client
        Timeout int
    }{}
    
    func init() {
        envconfig.MustProcess("widget", &Config)
    }
    

    Does WIDGET_TIMEOUT correspond to Config.Timeout, or Config.HTTPClient.Timeout?

    So I propose we underscore-delimit the field names: WIDGET_TIMEOUT and WIDGET_HTTPCLIENT_TIMEOUT.

    envconfig tags would apply globally for nested structs as they do outer structs today (clobbering any prefix), so any field at any position can specify its full env var name. But we can't rely on struct tags alone and still support dropping in a stdlib or third-party-lib struct like the example above.

    opened by teepark 7
  • Add support for setting slice values from environment variables

    Add support for setting slice values from environment variables

    Fixes #31 by adding support for populating slices from comma separated environment variables.

    type Specification struct {
        AdminUsers                   []string
        MagicNumbers                 []int
    

    To support this, I split out the switch statement from Process() into Parse() and Set() functions which deal with deserializing values from a specific type and setting the values using reflection (respectively). This was done primarily to allow for reuse of the existing string conversion logic for each type with slices containing those types.

    Tests included

    opened by mark-adams 7
  • [Feature] Custom Decoders

    [Feature] Custom Decoders

    So I needed to implement an additional config provider for my app via env vars and stumbled across this short and sweet library, but I needed a little extra in the form of custom decoding of the env var value itself (think slices of data or even references to other env vars).

    In my particular case I had a variable number of hashes I needed to get into my app as a single struct field and couldn't find a way to do it easily with the current code base so I extended it with the concept of custom decoders.

    Effectively you register a decoder for a field name before you call Process(), you can register as many decoders as you like and you can also clear them out once you finish (as the storage mechanism is a package level var):

         type Config struct {
             Keys []string
         }
    
         envconfig.RegisterDecoder("Keys", func(value string, fieldValue, struc reflect.Value) error {
             ...logic...
         })
         defer envconfig.ClearDecoders()
    

    In this case it is registered to the Keys struct field. You are given access to the current env var value, struct field and the struct being operated on. This should allow for maximum flexibility for a custom decoder.

    The decoder func could be turned into a named type but I thought it was shorter for the caller to use an anonymous func. This can be changed though.

    Thanks, Adrian

    opened by adrianduke 6
  • Feature Suggestion — Check arbitrary keys/values

    Feature Suggestion — Check arbitrary keys/values

    I work in a team that uses docker-compose. Commonly, we have docker-compose.yml files for each of our environments. It's a fairly common issue that we define new environment variables for a service, but forget to include them in the production environment's compose file, since it doesn't tend to be used locally.

    I don't propose to make envconfig aware of docker compose, but I would like a way to 'check' an arbitrary set of keys/values against a spec. I can put together a tool to get the environment variables from the docker-compose file, but I can't currently pass those in to envconfig. Would you be open to adding a Check(prefix string, spec interface{}, kvs map[string]string) method? I'd be happy to submit a PR for it.

    opened by rossmcf 5
  • New release

    New release

    Hi.

    I see the latest tag is 12 behind the current master branch.

    The feature I'm looking for is not having to specify the prefix on envconfig.Process() in snake case format. It seems odd to specify the env prefix like my_app where MY_APP is the standard format to specify environment variables.

    I see this is fixed on master branch though, so since locking versions is always a good practice, when a new tag can be expected?

    Cheers!

    opened by ricardogama 5
  • Doesn't appear to work under macOS

    Doesn't appear to work under macOS

    macOS 10.13.3 High Sierra Golang go1.10rc1

    When I run the example app, it just gives me the zero-values for the struct, not the values from the export commands. I seem to remember I had similar problems when I used os.Getenv but I had hoped this package solved those problems.

    opened by carlca 5
  • Generate usage / help message

    Generate usage / help message

    modified the code to a "visitor" pattern implementation, converted the existing Process method to used this pattern as well as added a visitor that can can output usage information using a Go template, including table semantics. See README.md for example.

    opened by davidkbainbridge 5
  • Feature Request: Make usage available in machine processable format

    Feature Request: Make usage available in machine processable format

    Feature

    This goes into a very similar direction as #97. I would like to be able to get the usage info in form that can easily be processes by other parts of my program.

    Reasons

    I think it is a good idea to give developers more control how to inform the user about usage information. I can think of the following scenarios that currently are hard to implement:

    • Showing the usage information in some sort of graphical user interface (native or web)
    • Logging the usage information through an existing logging framework
      • This might be favorable because
        • The logging framework might apply specific formatting that is hard/tedious to replicate with a template
        • The logging output location, format, etc. might be user-configurable. Replicating the effect of these options for envconfig would most likely not make sense from a maintainability point of view

    Workaround

    Currently I use a template which outputs the usage information as JSON, which can then be parsed again. But this is a really hacky solution requiring additional code, which would be unnecessary if envconfig had a way to get this info directly.

    For anyone who might have a similar problem: Here is my code to do that:

    // Variable describes one possible environment variable
    type Variable struct {
    	Name        string `json:"name"`
    	Type        string `json:"type"`
    	Default     string `json:"default"`
    	Required    bool   `json:"required"`
    	Description string `json:"description"`
    }
    
    // usageFormat is used by getUsage to print the usage info from envconfig as a json
    const usageFormat = `{{ define "dec" }}{{ len (slice (printf "%*s" . "") 1) }}{{ end -}}
    {{$length := len . -}}
    [
    {{range $idx, $val := .}}  {
        "name": "{{usage_key $val}}",
        "type": "{{usage_type $val}}",
        "default": "{{usage_default $val}}",
        "required": {{if usage_required $val -}} true {{- else -}} false {{- end}},
        "description": "{{usage_description $val}}"
      }{{if not (eq $idx (len (slice (printf "%*s" $length "") 1)))}},{{end}}{{/* If not last element print comma */}}
    {{end}}]
    `
    
    // getUsage gets the usage information from envconfig, parses it and returns it as a array of Variables
    func getUsage(config interface{}) ([]Variable, error) {
    	var buff bytes.Buffer
    	var vars []Variable
    
    	if err := envconfig.Usagef(PREFIX, config, io.Writer(&buff), usageFormat); err != nil {
    		return vars, errors.New(err)
    	}
    
    	if err := json.Unmarshal(buff.Bytes(), &vars); err != nil {
    		return vars, errors.New(err)
    	}
    
    	return vars, nil
    }
    
    opened by neumantm 0
  • Is this project still maintained?

    Is this project still maintained?

    It looks like this project has several forks and lots of PRs ready to merge, but there's not been any feedback nor no new releases in a long time. Is this project still maintained?

    If not, maybe it would be good to state so and even possibly point to a successor fork.

    Thanks for a very useful library!

    opened by jgimenez 2
  • Support for conditional requirement of variables

    Support for conditional requirement of variables

    type Specification struct {
    	FEATURETOGGLE   bool `required:"true"`
    	FEATURESETTING  string `conditionalRequired:"{FEATURETOGGLE == true}"`
    
    }
    

    Feature description:

    • Be able to make field a requirement based on the fact if a given condition is met or not. For example, if the featuretoogle is turned on make all fields with the conditionalRequired Tag required if their condition is met.

    Feel free to close this right away if this feature is somehow present already, in case I overlooked it.

    opened by MB175 0
  • Required by default

    Required by default

    It would be nice if there was a way to make required true by default. Most of our fields are required and it's rather cumbersome to individually mark everything as required.

    opened by DeadlySurgeon 0
  • Unnecessary initialization of struct pointers fix

    Unnecessary initialization of struct pointers fix

    Here is the correct version of the removeEmptyStructs mentioned in #113 that support multiple levels of structs nesting when some levels can be pointers and some not:

    https://github.com/hexdigest/envconfig/blob/master/envconfig.go#L230:L263

    I also added tests for this func. Hope it helps to everyone who struggle from the same issue.

    @kelseyhightower not quite sure that point_up thing can be a good candidate for a PR because it's more of a cleanup rather than an actual fix for the env parser even though I don't see how it can break something.

    opened by hexdigest 0
  • Improve error for required env vars with alt name

    Improve error for required env vars with alt name

    If we have a prefix called "env_config" and the following struct tags for property "Foo", we should expect to see an error message stating that the required env var "ENV_CONFIG_BAR" is missing and not just "BAR" since that is the env var being looked up.

    type Spec struct {
      Foo string `envconfig:"bar" required:"true"`
    }
    

    Fixes https://github.com/kelseyhightower/envconfig/issues/199

    opened by wfernandes 0
Releases(v1.4.0)
  • v1.4.0(May 24, 2019)

    • improved boolean parsing
    • support for encoding.BinaryUnmarshaler
    • CheckDisallowed function to validate a spec struct against the environment
    • improvement to word splitting for uppercase words
    • go module support!
    • various error message and documentation improvements
    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(Jan 24, 2017)

    • supports a split_words struct field tag for using underscores in env var names
    • Usage* API for printing out usage information calculated from envconfig spec structs
    • now supports maps with key1:value1,key2:value2 syntax in environment variables
    Source code(tar.gz)
    Source code(zip)
  • v1.2.0(Oct 19, 2016)

    • targets support embedded structs
    • interface for custom decoding of the env var string
    • ignored field tag
    • targets support pointer fields
    • support for nested (non-embedded) structs
    • exposes underlying error within ParseError type
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Mar 4, 2016)

Owner
Kelsey Hightower
Kelsey Hightower
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

null 3 Aug 22, 2022
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Vincent Rischmann 229 Nov 3, 2022
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Josh Betz 214 Oct 15, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Go Sidekick 1 Sep 26, 2022
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Sergey Prokhorov 0 Dec 30, 2021
A Go port of Ruby's dotenv library (Loads environment variables from `.env`.)

GoDotEnv A Go (golang) port of the Ruby dotenv project (which loads env vars from a .env file) From the original Library: Storing configuration in the

John Barton 5.5k Nov 23, 2022
A Go library for parsing struct tags from environment variables.

Envconfig Envconfig populates struct field values based on environment variables or arbitrary lookup functions. It supports pre-setting mutations, whi

Seth Vargo 697 Nov 16, 2022
Simple lib to parse environment variables to structs

env Simple lib to parse envs to structs in Go. Example A very basic example: package main import ( "fmt" "time" // if using go modules "github.c

Carlos Alexandro Becker 2.9k Nov 23, 2022
Un-marshaling environment variables to Go structs

envcfg Un-marshaling environment variables to Go structs Getting Started Let's set a bunch of environment variables and then run your go app #!/usr/bi

Tomaž Kovačič 99 Sep 26, 2022
Go helpers to manage environment variables

Envh This library is made up of two parts : Env object : it wraps your environments variables in an object and provides convenient helpers. Env tree o

Anthony HAMON 96 Sep 26, 2022
Environment variables substitution for Go

envsubst Environment variables substitution for Go. see docs below Installation: From binaries Latest stable envsubst prebuilt binaries for 64-bit Lin

Ariel Mashraki 555 Nov 23, 2022
Quickly read variables from environment files

go-quick-env Quickly read variables from environment files The best way to import environment variables to your code, is by using .env files. This lib

Panos Petropoulos 3 May 11, 2021
Read files into environment variables and execute command

read-file-to-env -- Read files into environment variables and execute command Example use: read-file-to-env -one-line=HOST=/etc/hostname sh -c 'echo h

Tv 2 Nov 12, 2021
A mapper of ENVironment variables to Structure for Go

envs a mapper of ENVironment variables to a Structure for Go. This library maps the environment variables to the struct according to the fields' types

moznion 3 Dec 3, 2021
formicidate is a small tool for Go application can update the value of environment variables in a .env file with code

formicidae Update .env files in Go with code. What is fomicidae? formicidate is a small tool for Go application. You can update the value of environme

akuma 0 Jan 23, 2022
Lightweight package that makes easier and safer to deal with environment variables.

Envisage A lightweight package that makes easier and safer to deal with environment variables. Example Try it on On GoPlay https://goplay.tools/snippe

GOLang Sugar 4 Apr 11, 2022
✨Clean and minimalistic environment configuration reader for Golang

Clean Env Minimalistic configuration reader Overview This is a simple configuration reading tool. It just does the following: reads and parses configu

Ilya Kaznacheev 791 Nov 25, 2022
Golang Configuration tool that support YAML, JSON, TOML, Shell Environment

Configor Golang Configuration tool that support YAML, JSON, TOML, Shell Environment (Supports Go 1.10+) Usage package main import ( "fmt" "github.c

Jinzhu 1.5k Nov 23, 2022
Cfginterpolator is an interpolate library in golang allowing to include data from external sources in your configuration

cfginterpolator cfginterpolator is an interpolate library in golang allowing to include data from external sources in your configuration cfginterpolat

Benoit Bayszczak 0 Dec 14, 2021