Simple lib to parse environment variables to structs

Overview

env

Build Status Coverage Status

Simple lib to parse envs to structs in Go.

Example

A very basic example:

package main

import (
	"fmt"
	"time"

	// if using go modules
	"github.com/caarlos0/env/v6"

	// if using dep/others
	"github.com/caarlos0/env"
)

type config struct {
	Home         string        `env:"HOME"`
	Port         int           `env:"PORT" envDefault:"3000"`
	IsProduction bool          `env:"PRODUCTION"`
	Hosts        []string      `env:"HOSTS" envSeparator:":"`
	Duration     time.Duration `env:"DURATION"`
	TempFolder   string        `env:"TEMP_FOLDER" envDefault:"${HOME}/tmp" envExpand:"true"`
}

func main() {
	cfg := config{}
	if err := env.Parse(&cfg); err != nil {
		fmt.Printf("%+v\n", err)
	}

	fmt.Printf("%+v\n", cfg)
}

You can run it like this:

$ PRODUCTION=true HOSTS="host1:host2:host3" DURATION=1s go run main.go
{Home:/your/home Port:3000 IsProduction:true Hosts:[host1 host2 host3] Duration:1s}

Supported types and defaults

Out of the box all built-in types are supported, plus a few others that are commonly used.

Complete list:

  • string
  • bool
  • int
  • int8
  • int16
  • int32
  • int64
  • uint
  • uint8
  • uint16
  • uint32
  • uint64
  • float32
  • float64
  • string
  • time.Duration
  • encoding.TextUnmarshaler
  • url.URL

Pointers, slices and slices of pointers of those types are also supported.

You can also use/define a custom parser func for any other type you want.

If you set the envDefault tag for something, this value will be used in the case of absence of it in the environment.

By default, slice types will split the environment value on ,; you can change this behavior by setting the envSeparator tag.

If you set the envExpand tag, environment variables (either in ${var} or $var format) in the string will be replaced according with the actual value of the variable.

Unexported fields are ignored.

Custom Parser Funcs

If you have a type that is not supported out of the box by the lib, you are able to use (or define) and pass custom parsers (and their associated reflect.Type) to the env.ParseWithFuncs() function.

In addition to accepting a struct pointer (same as Parse()), this function also accepts a map[reflect.Type]env.ParserFunc.

env also ships with some pre-built custom parser funcs for common types. You can check them out here.

If you add a custom parser for, say Foo, it will also be used to parse *Foo and []Foo types.

This directory contains pre-built, custom parsers that can be used with env.ParseWithFuncs to facilitate the parsing of envs that are not basic types.

Check the example in the go doc for more info.

Required fields

The env tag option required (e.g., env:"tagKey,required") can be added to ensure that some environment variable is set. In the example above, an error is returned if the config struct is changed to:

type config struct {
    Home         string   `env:"HOME"`
    Port         int      `env:"PORT" envDefault:"3000"`
    IsProduction bool     `env:"PRODUCTION"`
    Hosts        []string `env:"HOSTS" envSeparator:":"`
    SecretKey    string   `env:"SECRET_KEY,required"`
}

From file

The env tag option file (e.g., env:"tagKey,file") can be added to in order to indicate that the value of the variable shall be loaded from a file. The path of that file is given by the environment variable associated with it Example below

package main

import (
	"fmt"
	"time"
	"github.com/caarlos0/env"
)

type config struct {
	Secret       string   `env:"SECRET,file"`
	Password     string   `env:"PASSWORD,file" envDefault:"/tmp/password"`
	Certificate  string   `env:"CERTIFICATE,file" envDefault:"${CERTIFICATE_FILE}" envExpand:"true"`
}

func main() {
	cfg := config{}
	if err := env.Parse(&cfg); err != nil {
		fmt.Printf("%+v\n", err)
	}

	fmt.Printf("%+v\n", cfg)
}
$ echo qwerty > /tmp/secret
$ echo dvorak > /tmp/password
$ echo coleman > /tmp/certificate

$ SECRET=/tmp/secret  \
	CERTIFICATE_FILE=/tmp/certificate \
	go run main.go
{Secret:qwerty Password:dvorak Certificate:coleman}

Options

Environment

By setting the Options.Environment map you can tell Parse to add those keys and values as env vars before parsing is done. These envs are stored in the map and never actually set by os.Setenv. This option effectively makes env ignore the OS environment variables: only the ones provided in the option are used.

This can make your testing scenarios a bit more clean and easy to handle.

package main

import (
	"fmt"
	"log"

	"github.com/caarlos0/env"
)

type Config struct {
	Password string `env:"PASSWORD"`
}

func main() {
	cfg := &Config{}
	opts := &env.Options{Environment: map[string]string{
		"PASSWORD": "MY_PASSWORD",
	}}

	// Load env vars.
	if err := env.Parse(cfg, opts); err != nil {
		log.Fatal(err)
	}

	// Print the loaded data.
	fmt.Printf("%+v\n", cfg.envData)
}

Changing default tag name

You can change what tag name to use for setting the env vars by setting the Options.TagName variable.

For example

package main

import (
	"fmt"
	"log"

	"github.com/caarlos0/env"
)

type Config struct {
	Password string `json:"PASSWORD"`
}

func main() {
	cfg := &Config{}
	opts := &env.Options{TagName: "json"}

	// Load env vars.
	if err := env.Parse(cfg, opts); err != nil {
		log.Fatal(err)
	}

	// Print the loaded data.
	fmt.Printf("%+v\n", cfg.envData)
}

Stargazers over time

Stargazers over time

Comments
  • Introduce support for custom parsers + url.URL

    Introduce support for custom parsers + url.URL

    Previous PR was based on an outdated fork. Woops! This should be better.

    • Added support for url.URL
    • Added support for custom parsers via ParseWithFuncs() (new func to avoid breaking backwards compat)
    • Updated documentation
    • Updated tests (100% cov for all new functionality)

    @caarlos0 can you take a peek at this?

    opened by dselans 17
  • Expand support for custom parsers to non-struct types

    Expand support for custom parsers to non-struct types

    Sometimes you end up with having types in your config that are essentially int or string. An example of this would be log levels. I'm using logrus, and would like to be able to set LOG_LEVEL=debug, and map it directly on to my config struct:

    
    type Config struct {
        LogLevel logrus.Level `env:"LOG_LEVEL" envDefault:"debug"`
    }
    
    func logLevelParser(v string) (interface{}, error) {
        return logrus.ParseLevel(v)
    }
    
    func Get() (*Config, error) {
        c := &Config{}
        parsers := env.CustomParsers{
            reflect.TypeOf(logrus.GetLevel()): logLevelParser,
        }
        if err := env.ParseWithFuncs(c, parsers); err != nil {
            return nil, err
        }
        return c, nil
    }
    

    I like the custom parser support, but I think it would be even better still if we could add parser functions for basic types (ie non-struct types). Hence the PR.

    It's a small change, but I think quite a useful addition to the package. I've added 1 test covering this case. Let me know if more tests are required.

    opened by EVODelavega 15
  • panic: reflect: reflect.Value.Set using value obtained using unexported field

    panic: reflect: reflect.Value.Set using value obtained using unexported field

    It's mandatory to use exported fields on the struct, which we need to parse. We should able to use unexported fields as well.

    Steps to reproduce

    • Use https://github.com/caarlos0/env#example
    • Rename Home field of config struct to home (Line 11)
    • run PRODUCTION=true HOSTS="host1:host2:host3" DURATION=1s go run main.go

    Expected

    • output {Home:/your/home Port:3000 IsProduction:true Hosts:[host1 host2 host3] Duration:1s}

    Actual

    panic: reflect: reflect.Value.Set using value obtained using unexported field
    
    goroutine 1 [running]:
    reflect.flag.mustBeAssignable(0x1b8)
            /usr/local/go/src/reflect/value.go:231 +0x1bd
    
    bug 
    opened by dumindu 13
  • Adding a general-purpose, optional zero-overhead per-field convenience callback:

    Adding a general-purpose, optional zero-overhead per-field convenience callback:

    --- working in a project right now where we want to log the env-vars on-service-startup, and also present them in a "service self-description API endpoint".

    Since this is already relying on your neat env package, it would be ugly to recreate your existing reflect traversals for our purposes here. With this new optional general-purpose callback OnEnvVarSet(field, value) anyone can easily achieve such (or similar, or other) custom-requirements with env =)

    The first commit in here is not pertinent, simply some minor hygiene. The second commit is the meat of this PR, as per above.

    opened by metaleap 10
  • maintainers?!

    maintainers?!

    is anyone interested in helping me maintain this repo?

    I have lots of OSS repos and this one needs more love from my part, but at the same time I don't have much free time lately... maybe someone is interested in helping?

    question 
    opened by caarlos0 9
  • Pointer type support

    Pointer type support

    I have a global var Dev, but i want to use it in submodule only when submodule's Config.Dev is not set, so i must use *bool, can you support this? Thanks!

    package main
    
    import (
    	"github.com/caarlos0/env"
    	"github.com/golang/glog"
    )
    
    type Config struct {
    	Dev *bool `env:"DEV"`
    }
    
    func main() {
    	config := new(Config)
    	err := env.Parse(config)
    	glog.Errorln(config.Dev, err)
    }
    

    output:

    <nil> Type is not supported
    
    opened by empirefox 9
  • added conditional values

    added conditional values

    Main inspiration was to get something like NODENV and similar tools that use it. Looking around I didn't find anything too similar. Basic idea is that providing cases a list of options, and using the resolved env var as key to the specified options (in the example's case, prod and dev).

    opened by jamesalbert 8
  • Breaking change with uninitialised pointer fields

    Breaking change with uninitialised pointer fields

    I was using this package to set some fields of a struct, while other fields of the same struct were not controlled using environment variables.

    This worked fine, until recently - unfortunately not sure exactly when the change was introduced, but now the following example main.go

    package main
    
    import (
    	"github.com/caarlos0/env"
    	"log"
    	"net/http"
    )
    
    type Session struct {
    	Token      string `env:"TOKEN"`
    	HttpClient *http.Client
    }
    
    func main() {
    	sess := Session{}
    	err := env.Parse(&sess)
    	if err != nil {
    		log.Fatal(err)
    	}
    	sess.HttpClient = &http.Client{}
    }
    

    produces the output

    $ go run main.go
    2017/11/21 10:57:04 Expected a pointer to a Struct
    exit status 1
    

    but used to work just fine.

    Now maybe it was coincidence that it worked just fine - maybe the package doesn't support structs that have fields that are not supposed to be handled by env vars.

    But if it is supposed to be a feature (and I think at least I would like it to be ^^), then the above is probably a bug.

    A workaround, btw, is to move the assignment before the parsing, as in

    	sess := Session{}
    	sess.HttpClient = &http.Client{}
    	err := env.Parse(&sess)
    

    This won't throw an error.

    bug 
    opened by valentin-krasontovitsch 8
  • Support UnmarshalText interface

    Support UnmarshalText interface

    Hi, I saw this library on the gophers slack and the one thing I think it's missing for me is the ability to support types that implement the TextUnmarshaler interface out of the box, like https://github.com/kelseyhightower/envconfig does.

    docs 
    opened by johanbrandhorst 7
  • Broken go.mod

    Broken go.mod

    It looks like this commit broke the go.mod file:

    https://github.com/caarlos0/env/commit/0b4f1fc4774952aad2a0e3d796bf446cc294960d

    Latest working version that I can update to and go mod tidy goes through is 5.1.1, 5.1.2 fails with following message:

    go: github.com/caarlos0/[email protected]+incompatible: go.mod has post-v5 module path "github.com/caarlos0/env/v5" at revision v5.1.2

    caarlos0/env is listed as github.com/caarlos0/env v5.1.1+incompatible in my go.mod file (upgrading from v4).

    question 
    opened by rkuska 7
  • Fix bug when parsing env variables set and empty

    Fix bug when parsing env variables set and empty

    the following bugs were found:

    • default bug fixed: when environment variable does not exists, default value will be chosen... But when environment variable exists and it is empty, that empty string will be chosen.

    • required bug fixed: when environment variable does not exists, in error will be thrown... But when environment variable exists and it is empty, no error will be throw.

    opened by ljesparis 7
  • Pointer structs not created with env prefixes

    Pointer structs not created with env prefixes

    Following the issue raised in #202, if a pointer struct is used then the default object is not created - this is different to a non-pointer object. See the following example:

    package main
    
    import (
    	"encoding/json"
    	"fmt"
    	"log"
    	"os"
    
    	"github.com/caarlos0/env/v6"
    )
    
    type Test struct {
    	Str string `env:"TEST"`
    }
    type ComplexConfig struct {
    	Foo   *Test `envPrefix:"FOO_"`
    	Bar   Test  `envPrefix:"BAR_"`
    	Clean *Test
    }
    
    func main() {
    	os.Setenv("FOO_TEST", "kek")
    	os.Setenv("BAR_TEST", "lel")
    
    	cfg := ComplexConfig{}
    	err := env.Parse(&cfg)
    	if err != nil {
    		log.Fatal(err)
    	}
    	confBytes, _ := json.Marshal(cfg)
    	fmt.Printf("%s", confBytes)
    }
    

    This results in:

    {"Foo":null,"Bar":{"Str":"lel"},"Clean":null}
    

    Notice how Foo is null, but Bar has the object created.

    opened by MrSimonEmms 1
Releases(v6.10.1)
  • v6.10.1(Sep 12, 2022)

  • v6.10.0(Aug 22, 2022)

    Changelog

    New Features

    • 69c7b5a1dc8b2c4e78a7211aef3fd102333e4e4f: feat: aggregate validation errors (#233) (@akutuev)

    Bug fixes

    • 439dbad60f367bc1995f066cf9127743d468a572: fix: RequiredIfNoDef with nested structs (#229) (@HarmlessEvil)

    Documentation updates

    • 60b5226da3e5e4ca3dcb795b8416f30c5cb2c5a9: docs: broken link (@caarlos0)
    • 4c93d81012635da80954a1b3ec2e30566aa1b9d9: docs: remove duplicate string (#232) (@dacapoday)

    Released with GoReleaser Pro!

    Source code(tar.gz)
    Source code(zip)
  • v6.9.3(May 30, 2022)

  • v6.9.2(May 12, 2022)

    Changelog

    Bug fixes

    • 302a1031fb7c5e9de120b467f7ec1c636cd923e5: fix: race condition in ParseWithFuncs (#223) (@caarlos0)

    Other work

    • 8eb2e726db15f6d74b8225095da3c70a0aadfcb1: refactor: remove unneccessary line (#212) (@gozeloglu)

    Released with GoReleaser Pro!

    Source code(tar.gz)
    Source code(zip)
  • v6.9.1(Dec 30, 2021)

  • v6.9.0(Dec 29, 2021)

  • v6.8.0(Dec 1, 2021)

    Changelog

    New Features

    • f22b8199bef9cb3530f056e959a117111f35a983: feat: rename master to main (@caarlos0)
    • 2cf397a58a36021e20b385fa64839ba446fbe4fd: feat:prefix envs (#201) (@caarlos0)

    Bug fixes

    • d8dc27c896429a7cddb9c2789e26aacc6cbb5bba: fix: prefixes on pointers (#203) (@caarlos0)

    Released with GoReleaser Pro!

    Source code(tar.gz)
    Source code(zip)
  • v6.7.2(Oct 17, 2021)

    Changelog

    db06a3f344740e4bce2a0d4e60d5b5861c2566b3: fix: gofumpt (@caarlos0) adc0f6371c9fee028bf4ecb55958c4a63e061a6d: fix: lint issues (@caarlos0) d2550013e8b1044bad96d8dd8d8f58ef70e6b369: refactor: move from io/ioutil to os package (#176) (@Juneezee)


    Released with GoReleaser Pro!

    Source code(tar.gz)
    Source code(zip)
  • v6.7.1(Sep 6, 2021)

  • v6.7.0(Aug 30, 2021)

    Changelog

    929e30e chore(deps): bump codecov/codecov-action from 1 to 2 (#186) 92ab404 feat: Add option: 'required' by default (#187) 5dd6d0c feat: go 1.17 1cd374b feat: onset hook (#185) 53ff073 fix: ineffective break d96cac5 fix: merge issues faf605d refactor: improve test

    Source code(tar.gz)
    Source code(zip)
  • v6.6.2(May 30, 2021)

  • v6.6.1(May 30, 2021)

  • v6.6.0(May 16, 2021)

    Changelog

    f2ced39 chore(deps): bump actions/cache from v2 to v2.1.4 (#156) 41425b4 chore(deps): bump actions/cache from v2 to v2.1.4 (#160) 346d77c chore(deps): bump actions/cache from v2.1.4 to v2.1.5 (#164) c33df78 chore(deps): bump actions/checkout from 2 to 2.3.4 (#168) 7c39877 chore(deps): bump codecov/codecov-action from 1 to 1.5.0 (#167) d80f61d chore(deps): bump golangci/golangci-lint-action from v2 to v2.5.2 (#161) 2174282 chore(deps): bump goreleaser/goreleaser-action from 2 to 2.5.0 (#166) 6cbd70c docs/test: document how to use custom time formats (#172) fd499bb feat: allow to unset a variable after reading it (#171) 3de7ad1 feat: go 1.16, test on mac and windows (#159) b2f4697 feat: notEmpty tag (#174) eb9cdc4 fix(ci): workflow versions 9a4342d fix(ci): workflow versions d552093 fix: linter 0397ad7 fix: some cases on windows (#170) f878cc5 tests: use require instead of assert

    Source code(tar.gz)
    Source code(zip)
  • v6.5.0(Feb 1, 2021)

    Changelog

    54c9d5e chore(ci): dependabot 7e1f128 chore: delete gosum workflow aa84092 feat: upgrade to go 1.15 (#150) 2ed4d36 fix: add check for blank env key (#153) e244b8a refactor: improve if

    Source code(tar.gz)
    Source code(zip)
  • v6.4.0(Oct 15, 2020)

    Changelog

    3ac02fe chore(deps): go mod tidy (#122) f21b8c3 feat(ci): dependabot 52290f9 feat: add options (#131) a1ac5e4 fix: support custom slice types (#130) 07535a5 refactor: make unmarshal error handling clearer

    Source code(tar.gz)
    Source code(zip)
  • v6.3.0(Jun 23, 2020)

    Changelog

    822f229 chore: funding e8c6818 feat(ci): github actions 6cc3632 feat(ci): remove travis c81891c feat(ci): test on go 1.14 as well eeded10 feat: go 1.14 on go.mod 8088497 fix: test on go1.14

    Source code(tar.gz)
    Source code(zip)
  • v6.2.2(Apr 26, 2020)

  • v6.2.1(Feb 26, 2020)

  • v6.2.0(Feb 24, 2020)

    • fix: go mod tidy
    • fix: readme fmt
    • chore(deps): bump github.com/stretchr/testify from 1.4.0 to 1.5.1 (#107)
    • feat: Support to load values from file (#105)
    • fix: lint
    • chore: go mod tidy
    • refactor: small code improvement (#101)
    • test: run tests on Go 1.13 (#100)
    Source code(tar.gz)
    Source code(zip)
  • v6.1.0(Oct 3, 2019)

  • v6.0.0(May 15, 2019)

  • v5.1.4(May 3, 2019)

  • v5.1.3(Apr 29, 2019)

  • v5.1.2(Apr 29, 2019)

  • v5.1.1(Apr 29, 2019)

  • v5.1.0(Apr 29, 2019)

  • v5.0.0(Apr 25, 2019)

  • v4.0.0(Mar 8, 2019)

    • no longer testing on go versions bellow 1.10.x
    • now testing on go 1.12.x
    • added go.mod and etc so it should work better with go modules
    • added nested struct support
    • added more native types
    Source code(tar.gz)
    Source code(zip)
  • v3.5.0(Oct 20, 2018)

  • v3.4.0(Sep 22, 2018)

    • 998eb22 feat: Add ExpandVar tag to support variable expansion (#57)
    • c4f2804 feat(parsing): Added encoding.TextUnmarshaler support (#53)
    • f8dab76 adds a general-purpose per-field callback (#51)
    • 3ce33ec ref.Field(i) is non-trivial, so one call per iter should well suffice (#51)
    Source code(tar.gz)
    Source code(zip)
Owner
Carlos Alexandro Becker
Creator @goreleaser; SRE @totvslabs; I try to delete more code than I write.
Carlos Alexandro Becker
Un-marshaling environment variables to Go structs

envcfg Un-marshaling environment variables to Go structs Getting Started Let's set a bunch of environment variables and then run your go app #!/usr/bi

Tomaž Kovačič 99 Sep 26, 2022
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Vincent Rischmann 226 Sep 16, 2022
Go helpers to manage environment variables

Envh This library is made up of two parts : Env object : it wraps your environments variables in an object and provides convenient helpers. Env tree o

Anthony HAMON 95 Apr 14, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Go Sidekick 0 May 30, 2022
A Go port of Ruby's dotenv library (Loads environment variables from `.env`.)

GoDotEnv A Go (golang) port of the Ruby dotenv project (which loads env vars from a .env file) From the original Library: Storing configuration in the

John Barton 5.3k Sep 21, 2022
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Josh Betz 215 Sep 20, 2022
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Kelsey Hightower 4.3k Sep 27, 2022
A Go library for parsing struct tags from environment variables.

Envconfig Envconfig populates struct field values based on environment variables or arbitrary lookup functions. It supports pre-setting mutations, whi

Seth Vargo 676 Sep 28, 2022
Environment variables substitution for Go

envsubst Environment variables substitution for Go. see docs below Installation: From binaries Latest stable envsubst prebuilt binaries for 64-bit Lin

Ariel Mashraki 531 Sep 21, 2022
Quickly read variables from environment files

go-quick-env Quickly read variables from environment files The best way to import environment variables to your code, is by using .env files. This lib

Panos Petropoulos 3 May 11, 2021
Read files into environment variables and execute command

read-file-to-env -- Read files into environment variables and execute command Example use: read-file-to-env -one-line=HOST=/etc/hostname sh -c 'echo h

Tv 2 Nov 12, 2021
A mapper of ENVironment variables to Structure for Go

envs a mapper of ENVironment variables to a Structure for Go. This library maps the environment variables to the struct according to the fields' types

moznion 3 Dec 3, 2021
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

null 3 Aug 22, 2022
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Sergey Prokhorov 0 Dec 30, 2021
formicidate is a small tool for Go application can update the value of environment variables in a .env file with code

formicidae Update .env files in Go with code. What is fomicidae? formicidate is a small tool for Go application. You can update the value of environme

akuma 0 Jan 23, 2022
Lightweight package that makes easier and safer to deal with environment variables.

Envisage A lightweight package that makes easier and safer to deal with environment variables. Example Try it on On GoPlay https://goplay.tools/snippe

GOLang Sugar 4 Apr 11, 2022
Tmpl - A tool to apply variables from cli, env, JSON/TOML/YAML files to templates

tmpl allows to apply variables from JSON/TOML/YAML files, environment variables or CLI arguments to template files using Golang text/template and functions from the Sprig project.

krako 1 May 30, 2022
Library for setting values to structs' fields from env, flags, files or default tag

Configuration is a library for injecting values recursively into structs - a convenient way of setting up a configuration object. Available features:

Bogdan Daragan 79 Sep 21, 2022