✨Clean and minimalistic environment configuration reader for Golang

Overview

Clean Env

Clean Env

Minimalistic configuration reader

Mentioned in Awesome Go GoDoc Go Report Card Coverage Status Build Status Release License

Overview

This is a simple configuration reading tool. It just does the following:

  • reads and parses configuration structure from the file
  • reads and overwrites configuration structure from environment variables
  • writes a detailed variable list to help output

Content

Installation

To install the package run

go get -u github.com/ilyakaznacheev/cleanenv

Usage

The package is oriented to be simple in use and explicitness.

The main idea is to use a structured configuration variable instead of any sort of dynamic set of configuration fields like some libraries does, to avoid unnecessary type conversions and move the configuration through the program as a simple structure, not as an object with complex behavior.

There are just several actions you can do with this tool and probably only things you want to do with your config if your application is not too complicated.

  • read configuration file
  • read environment variables
  • read some environment variables again

Read Configuration

You can read a configuration file and environment variables in a single function call.

import github.com/ilyakaznacheev/cleanenv

type ConfigDatabase struct {
    Port     string `yaml:"port" env:"PORT" env-default:"5432"`
    Host     string `yaml:"host" env:"HOST" env-default:"localhost"`
    Name     string `yaml:"name" env:"NAME" env-default:"postgres"`
    User     string `yaml:"user" env:"USER" env-default:"user"`
    Password string `yaml:"password" env:"PASSWORD"`
}

var cfg ConfigDatabase

err := cleanenv.ReadConfig("config.yml", &cfg)
if err != nil {
    ...
}

This will do the following:

  1. parse configuration file according to YAML format (yaml tag in this case);
  2. reads environment variables and overwrites values from the file with the values which was found in the environment (env tag);
  3. if no value was found on the first two steps, the field will be filled with the default value (env-default tag) if it is set.

Read Environment Variables Only

Sometimes you don't want to use configuration files at all, or you may want to use .env file format instead. Thus, you can limit yourself with only reading environment variables:

import github.com/ilyakaznacheev/cleanenv

type ConfigDatabase struct {
    Port     string `env:"PORT" env-default:"5432"`
    Host     string `env:"HOST" env-default:"localhost"`
    Name     string `env:"NAME" env-default:"postgres"`
    User     string `env:"USER" env-default:"user"`
    Password string `env:"PASSWORD"`
}

var cfg ConfigDatabase

err := cleanenv.ReadEnv(&cfg)
if err != nil {
    ...
}

Update Environment Variables

Some environment variables may change during the application run. To get the new values you need to mark these variables as updatable with the tag env-upd and then run the update function:

import github.com/ilyakaznacheev/cleanenv

type ConfigRemote struct {
    Port     string `env:"PORT" env-upd`
    Host     string `env:"HOST" env-upd`
    UserName string `env:"USERNAME"`
}

var cfg ConfigRemote

cleanenv.ReadEnv(&cfg)

// ... some actions in-between

err := cleanenv.UpdateEnv(&cfg)
if err != nil {
    ...
}

Here remote host and port may change in a distributed system architecture. Fields cfg.Port and cfg.Host can be updated in the runtime from corresponding environment variables. You can update them before the remote service call. Field cfg.UserName will not be changed after the initial read, though.

Description

You can get descriptions of all environment variables to use them in the help documentation.

import github.com/ilyakaznacheev/cleanenv

type ConfigServer struct {
    Port     string `env:"PORT" env-description:"server port"`
    Host     string `env:"HOST" env-description:"server host"`
}

var cfg ConfigRemote

help, err := cleanenv.GetDescription(&cfg, nil)
if err != nil {
    ...
}

You will get the following:

Environment variables:
  PORT  server port
  HOST  server host

Model Format

Library uses tags to configure the model of configuration structure. There are the following tags:

  • env="<name>" - environment variable name (e.g. env="PORT");
  • env-upd - flag to mark a field as updatable. Run UpdateEnv(&cfg) to refresh updatable variables from environment;
  • env-required - flag to mark a field as required. If set will return an error during environment parsing when the flagged as required field is empty (default Go value). Tag env-default is ignored in this case;
  • env-default="<value>" - default value. If the field wasn't filled from the environment variable default value will be used instead;
  • env-separator="<value>" - custom list and map separator. If not set, the default separator , will be used;
  • env-description="<value>" - environment variable description;
  • env-layout="<value>" - parsing layout (for types like time.Time);

Supported types

There are following supported types:

  • int (any kind);
  • float (any kind);
  • string;
  • boolean;
  • slices (of any other supported type);
  • maps (of any other supported type);
  • time.Duration;
  • time.Time (layout by default is RFC3339, may be overridden by env-layout);
  • any type implementing cleanenv.Setter interface.

Custom Functions

To enhance package abilities you can use some custom functions.

Custom Value Setter

To make custom type allows to set the value from the environment variable, you need to implement the Setter interface on the field level:

type MyField string

func (f *MyField) SetValue(s string) error  {
    if s == "" {
        return fmt.Errorf("field value can't be empty")
    }
    *f = MyField("my field is: "+ s)
    return nil
}

type Config struct {
    Field MyField `env="MY_VALUE"`
}

SetValue method should implement conversion logic from string to custom type.

Custom Value Update

You may need to execute some custom field update logic, e.g. for remote config load.

Thus, you need to implement the Updater interface on the structure level:

type Config struct {
    Field string
}

func (c *Config) Update() error {
    newField, err := SomeCustomUpdate()
    f.Field = newField
    return err
}

Supported File Formats

There are several most popular config file formats supported:

  • YAML
  • JSON
  • TOML
  • ENV
  • EDN

Integration

The package can be used with many other solutions. To make it more useful, we made some helpers.

Flag

You can use the cleanenv help together with Golang flag package.

// create some config structure
var cfg config 

// create flag set using `flag` package
fset := flag.NewFlagSet("Example", flag.ContinueOnError)

// get config usage with wrapped flag usage
fset.Usage = cleanenv.FUsage(fset.Output(), &cfg, nil, fset.Usage)

fset.Parse(os.Args[1:])

Examples

type Config struct {
    Port string `yaml:"port" env:"PORT" env-default:"8080"`
    Host string `yaml:"host" env:"HOST" env-default:"localhost"`
}

var cfg Config

err := ReadConfig("config.yml", &cfg)
if err != nil {
    ...
}

This code will try to read and parse the configuration file config.yml as the structure is described in the Config structure. Then it will overwrite fields from available environment variables (PORT, HOST).

For more details check the example directory.

Contribution

The tool is open-sourced under the MIT license.

If you will find some error, want to add something or ask a question - feel free to create an issue and/or make a pull request.

Any contribution is welcome.

Thanks

Big thanks to a project kelseyhightower/envconfig for inspiration.

The logo was made by alexchoffy.

Blog Posts

Clean Configuration Management in Golang.

Issues
  • Problem with bool variable in YAML file

    Problem with bool variable in YAML file

    yaml file

    isdebug: false
    rule:
      cachesize: 10485760000
    

    golang struct

    type AppConfig struct {
    	IsDebug bool `env:"RE_IS_DEBUG" env-required:"true"`
    	Rule struct {
    		CacheSize          int    `env:RULE_CACHE_SIZE" env-required:"true"`
            }
    

    i got an error field \"IsDebug\" is required but the value is not provided", but if set isdebug: true - all ok, config parse without any errors.

    discussion proposal 
    opened by dikkini 12
  • BUG: env-default overwrites value from yml file

    BUG: env-default overwrites value from yml file

    Wehnever env-default tag is defined, it's value overwrites value read from yml file (which is not intended behaviour, according to https://github.com/ilyakaznacheev/cleanenv#read-configuration )

    100% reproducibility, observed while running go app in vagrant box (ubuntu/bionic).

    configs/config.yml

    # server config
    server:
      host: "localhost"
      port: 3000
    
    # database config
    db:
      type: "mongodb"
      mongoConfig:
                # full endpoint: address + port
        URL: "mongodb://127.0.0.1:27017"
        ProjectsDBName: "Projects"
        SchemasDBName: "Schemas"
    

    config.go

    package common
    package common
    
    import (
      "github.com/ilyakaznacheev/cleanenv"
      "fmt"
    )
    
    const YmlConfFile string = "./configs/config.yml"
    
    type AppConfig struct {
    	ServerConfig struct {
    		PORT string `yaml:"port" env:"MYAPP_PORT" env-default:"3000" env-upd`
    		HOST string `yaml:"host" env:"MYAPP_HOST" env-default:"localhost" env-upd`
    	} `yaml:"server"`
    	DBConfig struct {
    		DBType string `yaml:"type" env:"MYAPP_DB_SRV" env-default:"mongodb"`
    		MongoConfig struct {
    			URL string			`yaml:"URL" env:"MYAPP_DB_URL" env-default:"mongodb://localhost:27017" env-upd`
    			ProjectsDB string	`yaml:"ProjectsDBName" env:"MYAPP_PROJECTS_DB" env-default:"TProjects" env-upd`
    			SchemasDB string	`yaml:"SchemasDBName"  env:"MYAPP_SCHEMAS_DB" env-default:"TSchemas" env-upd`
    		} `yaml:"mongoConfig"`
    	} `yaml:"db"`
    }
    
    func NewAppConfig(confFilePath string) *AppConfig {
    	var AppC AppConfig
        err := cleanenv.ReadConfig(confFilePath, &AppC)
        if err != nil { panic(err) }
    	fmt.Println("AppC.DBConfig.MongoConfig.URL %v",AppC.DBConfig.MongoConfig.URL)
    	return &AppC
    }
    

    main.go

    package main
    import "myappp/common"
    
    func main() {
      appConfig := common.NewAppConfig(common.YmlConfFile)
    }
    

    it prints: AppC.DBConfig.MongoConfig.URL %v mongodb://localhost:27017 (as in env-default)

    when I remove respective env-default tag with value, e.g.

    it prints

    AppC.DBConfig.MongoConfig.URL %v mongodb://127.0.0.1:27017 (as taken from file)

    Aside from report above, let me mention that's a great effort to conveniently handle different configuration sources. Thank you!

    bug discussion 
    opened by MichalRybinski 8
  • Expand environment variables when reading from file

    Expand environment variables when reading from file

    Hi,

    it would be cool if the library would support expansion of environment variables using os.ExpandEnv A use-case for this feature would be if you need to construct a configuration variable based on multiple different ones. e.g. building an URL wrong multiple environment variables

    I've tried to figure out where this must be added but couldn't find the write code part.

    Thanks

    PS: i'm thinking about something like this https://mtyurt.net/post/go-using-environment-variables-in-configuration-files.html

    discussion proposal 
    opened by eloo 7
  • Updating example app to include .env config loading

    Updating example app to include .env config loading

    The example was lacking demonstration of how environment variables can override the yaml configs. I have added a .env file and using godotenv to load them first.

    discussion 
    opened by ghost 5
  • Add url support

    Add url support

    First off, thanks for this package!

    I wanted to use it in our project, but we have in our environment a few URLs (DB connection strings in a single value, or URL for an image CDN for example), which should be valid ones. It felt to me a fairly valid case to have as supported type, so here's a PR for it!

    opened by padawin 4
  • Value containing `.` always clipped

    Value containing `.` always clipped

    Trying to read a value that contains . and it doesn't matter what I set the env-separator to. It gets clipped anyway.

    Am I using it wrong or this a bug?

    discussion 
    opened by stoffeastrom 4
  • Please fix readme

    Please fix readme

    Readme examples don't work out-of-the-box.

    Readme suggests using 'yml' tag that doesn't seem to work. Tests use 'yaml' instead.

    Setter from readme might be invalid as well. It should be defined on pointer type.

    Reproduced on go1.13 darwin/amd64

    opened by LihMeh 4
  • Use config value over default if not in env

    Use config value over default if not in env

    This is a proposed fix for https://github.com/ilyakaznacheev/cleanenv/issues/40.

    This behaves the way I expected, given this line of the documentation:

    1. if no value was found on the first two steps, the field will be filled with the default value (env-default tag) if it is set.

    in https://github.com/ilyakaznacheev/cleanenv#read-configuration.

    I'm no reflect wizard, so I suggest a careful review. It works in my testing though.

    opened by marcoderama 3
  • Not able load the config.yml file

    Not able load the config.yml file

    I am new to AWS and Multi Docker Container. Not able load config.yml and I don't know how to configure Dockerrun.aws.json to make sure the config.yml file is accessible in the docker container.

    var err = cleanenv.ReadConfig("config.yml", &cfg)
    if err != nil {
    	log.Fatal(err)
    }
    
    question 
    opened by sriramkp 2
  • add github action workflow for code coverage and notification

    add github action workflow for code coverage and notification

    Description

    This PR will provide a github action workflow for building coverage & uploading it to codecove. This will be triggered on push request to master/develop/ and any tag with release pattern. Needs CODECOV_TOKEN under github secrets for the repo.

    2nd workflow is only triggered for pull_request. On each pull request it will send a message to specified Telegram channel with head reference of branch, you can change the messages to anything.

    Need to set github secret TELEGRAM_TOKEN , this is bot token and TELEGRAM_CHANNELS, this is target telegram channel.

    Fixes # https://github.com/ilyakaznacheev/cleanenv/issues/52

    Type of change

    • [x] New feature (non-breaking change which adds functionality)
    enhancement 
    opened by kha7iq 2
  • Add default env variable binding

    Add default env variable binding

    Hi,

    i'm currently migrating from a combination of godotenv, uber-config and envconfig to your library because AFAIK your library is doing a good job integrating this loading mechanism into a single library without hastle.

    But as i have already a decent config struct based on envconfig i see thats its pretty hard to add for every property an env tag (AFAIK this is needed). So i would like to request some kind of default naming schema like envconfig has already. Further i'm missing the usage of prefixes but i've seen that there is already an issue addressing this.

    For so the default naming here is short example:

    type Config struct {
    	Server   ServerConfig
    }
    
    type ServerConfig struct {
    	Address          string
    	Port                 int        `env-default:"1328"`
            FriendlyName string
    }
    

    Currently i would to add everywhere an env tag with the name.. which is mostly the same as the property. So it would be cool if this config would be sufficient to have the env config working like this:

    SERVER_ADDRESS=cleanenv.is.great
    SERVER_PORT=1337
    SERVER_FRIENDLYNAME="Yeah cleanenv is great"
    

    What do you think? Could this be an future feature?

    Thanks

    question discussion proposal need your opinion 
    opened by eloo 2
  • Skip empty value assignment

    Skip empty value assignment

    There might be a situation when a shell contains an empty environment variable. In such a case, we get the next error for a boolean field, for example:

    strconv.ParseBool: parsing "": invalid syntax
    

    I propose to treat empty string values as absent ones.

    opened by ezh 1
  • Can't read config file in Amazon EC2 server

    Can't read config file in Amazon EC2 server

    This is how I'm simply reading the config file:

    func Read(path string) {
    	readError := cleanenv.ReadConfig(path, &App)
    	if readError != nil {
    		log.Panic("Failed to read yaml file", readError)
    	}
    }
    

    This is working on a my local machine (Mac) and in Docker container running locally. But when I run the app on EC2 (built as RPM package and ran as a service), I get this error:

    Process: 20421 ExecStart=/usr/bin/store-service -f /etc/store-service/config.yml (code=exited, status=2)
    

    The very same code runs well when I parse the config using a library like jessevdk/go-flags. Any idea?

    opened by alexTheJumbo 0
  • Can not overwrite variable

    Can not overwrite variable

    I'm having hard time overrriding a value from enviroment variable. Here is what I want. I want to read config from a enviroment file. Then read enviroment varialbes and overwrite them even If they exist in the config file. The following code does not overwrite value from enviroment varialbe as it supposed to:

    
    type ConfigDatabase struct {
    		Port     string `env:"PORT" env-default:"5432"`
    		Host     string `env:"HOST" env-default:"localhost"`
    		Name     string `env:"NAME" env-default:"postgres"`
    		User     string `env:"USER" env-default:"user"`
    		Password string `env:"PASSWORD"`
    	}
    
    	var cfg ConfigDatabase
    	err := cleanenv.ReadConfig("config.env", &cfg)
    	if err != nil {
    		log.Println(err)
    	}
    
    	err1 := cleanenv.ReadEnv(&cfg)
    	if err1 != nil {
    		log.Println(err1)
    	}
    	fmt.Println(cfg)
    
    

    Even witih cleanenv.UpdateEnv It still does not work. To my understanding, Each Read* function should overwrite the previous values.

    Also the env-update struct tag is not working either.

    
    Password string `env:"PASSWORD" env-upd`
    
    

    I get an warning in VSCode struct field tagenv:"PASSWORD" env-updnot compatible with reflect.StructTag.Get: bad syntax for struct tag pair

    opened by imraan-go 2
  • Add a custom error that would provide a meaningful feedback about the parsing issues.

    Add a custom error that would provide a meaningful feedback about the parsing issues.

    For the following example, if I provide a non-valid PORT=asd. Then in the error message, I would see "strconv.ParseInt: parsing "asd": invalid syntax". I think it would be great to have a custom error that would collect and provide some details on the type of error that happened to be able to write it to the log and give corresponding feedback to the runner of the application.

    type Config struct {
    	Web struct {
    		Port int `env:"PORT" env-default:"8000"`
    	}
    }
    
    if err := cleanenv.ReadEnv(&config); err != nil {
        fmt.Printf("Error: %v", err)
    }
    

    The message can have the following format: PORT must be a valid number or DB_HOST is a required variable.

    I'm not a Golang developer but have written some code in Go, so if you give your opinion on this and some hints I might be able to implement it and send a PR.

    opened by AM-NiceDay 0
  • Error after reading time.Duration fields in 1.2.6

    Error after reading time.Duration fields in 1.2.6

    Hello, I used v1.2.5 and alway gone all right. But problems with time fields have appeared in 1.2.6. I have this error pattern: line N: cannot unmarshal !!int `M` into time.Duration. I have config same as:

    type Config {
    ...
    Timeout time.Duration `yaml:"timeout"`
    ...
    } 
    
    opened by olya-varentsova 1
  • read config from io.Reader

    read config from io.Reader

    I want to use embed config files. But ReadConfig() only use file path. If I can use io.Reader for parseFile() like below, it would be nice. What do you think?

    Or, If there is any way to use embed files, please let me know.

    func ReadConfigFromReader(r io.Reader, cfg interface{}) {
    	err := parseFile(r, cfg)
    	if err != nil {
    		return err
    	}
    
    	return readEnvVars(cfg, false)
    }
    
    func ReadConfig(path string, cfg interface{}) {
    	f, err := os.OpenFile(path, os.O_RDONLY|os.O_SYNC, 0)
    	if err != nil {
    		return err
    	}
    	defer f.Close()
    
    	err := parseFile(f, cfg)
    	if err != nil {
    		return err
    	}
    
    	return readEnvVars(cfg, false)
    }
    
    
    opened by dlsrb6342 0
Releases(v1.3.0)
Owner
Ilya Kaznacheev
Backend SWE. Founder of Golang Voronezh, SAP Community Voronezh, active speaker & community member.
Ilya Kaznacheev
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

null 3 Feb 22, 2022
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Go Sidekick 0 May 30, 2022
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Kelsey Hightower 4.1k Jun 22, 2022
Golang Configuration tool that support YAML, JSON, TOML, Shell Environment

Configor Golang Configuration tool that support YAML, JSON, TOML, Shell Environment (Supports Go 1.10+) Usage package main import ( "fmt" "github.c

Jinzhu 1.4k Jun 27, 2022
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Josh Betz 210 Apr 10, 2022
Small library to read your configuration from environment variables

envconfig envconfig is a library which allows you to parse your configuration from environment variables and fill an arbitrary struct. See the example

Vincent Rischmann 222 May 18, 2022
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Sergey Prokhorov 0 Dec 30, 2021
A Go (golang) environment loader (which loads env vars from a .env file)

A Go (golang) environment loader (which loads env vars from a .env file)

Bendt Indonesia 0 Feb 8, 2022
Read files into environment variables and execute command

read-file-to-env -- Read files into environment variables and execute command Example use: read-file-to-env -one-line=HOST=/etc/hostname sh -c 'echo h

Tv 2 Nov 12, 2021
Go-config - Config parser for go that supports environment vars and multiple yaml files

go-multiconfig This package is able to parse yaml config files. It supports gett

Dimitris Tassopoulos 1 Jun 23, 2022
Lightweight package that makes easier and safer to deal with environment variables.

Envisage A lightweight package that makes easier and safer to deal with environment variables. Example Try it on On GoPlay https://goplay.tools/snippe

GOLang Sugar 4 Apr 11, 2022
Simple lib to parse environment variables to structs

env Simple lib to parse envs to structs in Go. Example A very basic example: package main import ( "fmt" "time" // if using go modules "github.c

Carlos Alexandro Becker 2.5k Jun 26, 2022
Un-marshaling environment variables to Go structs

envcfg Un-marshaling environment variables to Go structs Getting Started Let's set a bunch of environment variables and then run your go app #!/usr/bi

Tomaž Kovačič 98 May 4, 2022
Go helpers to manage environment variables

Envh This library is made up of two parts : Env object : it wraps your environments variables in an object and provides convenient helpers. Env tree o

Anthony HAMON 95 Apr 14, 2022
A Go port of Ruby's dotenv library (Loads environment variables from `.env`.)

GoDotEnv A Go (golang) port of the Ruby dotenv project (which loads env vars from a .env file) From the original Library: Storing configuration in the

John Barton 5k Jun 27, 2022
A Go library for parsing struct tags from environment variables.

Envconfig Envconfig populates struct field values based on environment variables or arbitrary lookup functions. It supports pre-setting mutations, whi

Seth Vargo 622 Jun 24, 2022
Environment variables substitution for Go

envsubst Environment variables substitution for Go. see docs below Installation: From binaries Latest stable envsubst prebuilt binaries for 64-bit Lin

Ariel Mashraki 506 Jun 19, 2022
Quickly read variables from environment files

go-quick-env Quickly read variables from environment files The best way to import environment variables to your code, is by using .env files. This lib

Panos Petropoulos 3 May 11, 2021
A mapper of ENVironment variables to Structure for Go

envs a mapper of ENVironment variables to a Structure for Go. This library maps the environment variables to the struct according to the fields' types

moznion 3 Dec 3, 2021