Manage local application configuration files using templates and data from etcd or consul

Related tags

Configuration confd
Overview

confd

Build Status

confd is a lightweight configuration management tool focused on:

Community

Building

Go 1.10 is required to build confd, which uses the new vendor directory.

$ mkdir -p $GOPATH/src/github.com/kelseyhightower
$ git clone https://github.com/kelseyhightower/confd.git $GOPATH/src/github.com/kelseyhightower/confd
$ cd $GOPATH/src/github.com/kelseyhightower/confd
$ make

You should now have confd in your bin/ directory:

$ ls bin/
confd

Getting Started

Before we begin be sure to download and install confd.

Next steps

Check out the docs directory for more docs.

Issues
  • support consul service health checks

    support consul service health checks

    Loads Consul service data into /_consul/service namespace. (https://github.com/kelseyhightower/confd/issues/100)

    Format for keys is:

    /_consul/service/<serviceName>[/<tagName>]/<nodeIndex>
    

    Consul datacenters are not supported yet. tagName is optional. Both sets of keys are generated. Consul data is loaded as a JSON marshalled String as the Value in the keystore. Using data in template requires unmarshalling the data using the new json or jsonArray functions bounds to the template.

    Additionally, added some convenience methods parent and sibling to the template for easier navigation around the KV store.

    Note: this is my 1st attempt at Go, so feel free to criticize style/naming/etc. I tried to follow standard where it was apparent.

    Supporting change at: https://github.com/kelseyhightower/memkv/pull/1

    opened by johnrengelman 69
  • dns srv function is not perfect

    dns srv function is not perfect

    Firstly, -srv-domain doesn't support _etcd-client-ssl._tcp.example.com which is etcd dns srv with ssl. Even if I use -scheme https

    Secondly, when I use -srv-record _etcd-client-ssl._tcp.example.com -scheme https to replace, it shows

    2017-08-30T15:27:13+08:00 linux confd[9653]: INFO SRV record set to _etcd-client-ssl._tcp.example.com
    2017-08-30T15:27:13+08:00 linux confd[9653]: INFO Backend set to etcd
    2017-08-30T15:27:13+08:00 linux confd[9653]: INFO Starting confd
    2017-08-30T15:27:13+08:00 linux confd[9653]: INFO Backend nodes set to 02.example.com:4758, 01.example.com:4758, 03.example.com:4758
    2017-08-30T15:27:13+08:00 linux confd[9653]: ERROR client: etcd cluster is unavailable or misconfigured; error #0: unsupported protocol scheme "01.example.com"
    ; error #1: unsupported protocol scheme "03.example.com"
    ; error #2: unsupported protocol scheme "02.example.com"
    
    2017-08-30T15:27:13+08:00 linux confd[9653]: FATAL client: etcd cluster is unavailable or misconfigured; error #0: unsupported protocol scheme "01.example.com"
    ; error #1: unsupported protocol scheme "03.example.com"
    ; error #2: unsupported protocol scheme "02.example.com"
    

    It seems that -scheme doesn't work.

    Finally, I use -node and it connects successfully. But it tells me that key doesn't exist, because confd prepends / to my key automatically, even if I set prefix = "" in toml.

    bug 
    opened by zyf0330 27
  • Confd giving a 501 peers not reachable single node etcd

    Confd giving a 501 peers not reachable single node etcd

    I am running confd in a docker container, with the docker host running etcd. I am able to get values from etcdctl as well as curl, but confd gives the 501: All the given peers are not reachable. Bash script (for testing) running inside the container:

    #!/bin/bash
    /usr/global/apps/etcdctl --no-sync -peers http://172.17.42.1:4001 ls system/zookeeper
    echo "curl test: " $(curl -X GET http://172.17.42.1:4001/v2/keys/system/zookeeper?consistent=true&recursive=false&sorted=false)
    /usr/global/apps/confd -onetime=true -verbose=true -debug=true -node="172.17.42.1:4001" -confdir=/usr/global/config/shared/zookeeper/confd/
    

    And here is the output I am getting:

    /system/zookeeper/zookeeper-1
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100   294    0   294    0     0   169k      0 --:--:-- --:--:-- --:--:--  287k
    test:  {"action":"get","node":{"key":"/system/zookeeper","dir":true,"nodes":[{"key":"/system/zookeeper/zookeeper-1","value":"{\"id\":\"1\",\"ip\":\"10.1.51.252\",\"cport\":\"49153\",\"pport\":\"49154\",\"lport\":\"49155\"}","modifiedIndex":30,"createdIndex":30}],"modifiedIndex":29,"createdIndex":29}}
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: WARNING Skipping confd config file.
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: NOTICE Backend set to etcd
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: NOTICE Starting confd
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: NOTICE Backend nodes set to 172.17.42.1:4001
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: DEBUG Loading template resources from confdir /usr/global/config/shared/zookeeper/confd/
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: DEBUG Processing template resource /usr/global/config/shared/zookeeper/confd/conf.d/zookeeperconf.toml
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: DEBUG Loading template resource from /usr/global/config/shared/zookeeper/confd/conf.d/zookeeperconf.toml
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: ERROR 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: DEBUG Retrieving keys from store
    2014-10-15T12:54:51-04:00 78d09c06c9fc /usr/global/apps/confd[16]: DEBUG Key prefix set to /
    

    The top two lines show the correct ip and port are being used (unlike https://github.com/kelseyhightower/confd/issues/130) versions: etcd version 0.4.5 etcdctl version 0.4.5 confd 0.6.3

    opened by nickfleming 27
  • Doesn't appear to handle removed keys

    Doesn't appear to handle removed keys

    Hi,

    I'm trying to use this to configure haproxy on a 1 node etcd cluster for testing I can add keys to Etcd and Confd builds my config and reloads haproxy fine, but if I remove keys, the config file it makes matches the old one. That is until I restart Confd then it builds the file properly.

    I've noticed this using -watch and -interval watch: http://pastebin.com/F0uBNbEi interval: http://pastebin.com/HcaJRmbt

    Cheers

    opened by stuart-warren 23
  • confd 0.6.x: getting an arbitrary part of the key path

    confd 0.6.x: getting an arbitrary part of the key path

    We have paths in etcd as follows:

    $ etcdctl ls /deis/services/inward-overalls
    /deis/services/inward-overalls/inward-overalls_v2.web.1
    /deis/services/inward-overalls/inward-overalls_v2.web.3
    /deis/services/inward-overalls/inward-overalls_v2.web.4
    

    I'm trying to template this with confd. It seems like I can match these keys as follows:

    {{ gets "/deis/services/*/*" }} 
    

    However, I need to use the elements of the path in my template logic. In this case, I need to write out inward-overalls in my template. I wish I could do something like this:

    {{ range gets "/deis/services/*/*" }} 
    foo {{ Path.split(.Key)[2] }}
    {{ end }} 
    

    ...but I'm not sure how arbitrary function calls work in golang templates. This just looks wrong.

    Could we define additional helper methods to use for this? It seems that something like that would be useful - paths are important. I would love to have the ability to match/enumerate paths, not just keys. For example:

    {{ gets "/deis/services/*" }}
    

    Then I could reference the children of this path that aren't necessarily keys. I think this was an advantage of having Nodes in the old confd.

    What is the recommended way to do this now?

    opened by carmstrong 22
  • Documentation for Installing from Source

    Documentation for Installing from Source

    Though the documentation says just run go build, the following packages are still telling me I need to use go get prior to the build. Am I doing something wrong here or is this in fact the necessary steps?

    [/repositories/git/opensource/confd]$ go build
    config.go:14:2: cannot find package "github.com/BurntSushi/toml" in any of:
        /usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
        /repositories/git/opensource/confd/src/github.com/BurntSushi/toml (from $GOPATH)
    confd.go:10:2: cannot find package "github.com/kelseyhightower/confd/backends" in any of:
        /usr/local/go/src/github.com/kelseyhightower/confd/backends (from $GOROOT)
        /repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/backends (from $GOPATH)
    confd.go:11:2: cannot find package "github.com/kelseyhightower/confd/log" in any of:
        /usr/local/go/src/github.com/kelseyhightower/confd/log (from $GOROOT)
        /repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/log (from $GOPATH)
    confd.go:12:2: cannot find package "github.com/kelseyhightower/confd/resource/template" in any of:
        /usr/local/go/src/github.com/kelseyhightower/confd/resource/template (from $GOROOT)
        /repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/resource/template (from $GOPATH)
    
    [/repositories/git/opensource/confd]$ ./build
    Building confd...
    config.go:14:2: cannot find package "github.com/BurntSushi/toml" in any of:
        /usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
        /repositories/git/opensource/confd/src/github.com/BurntSushi/toml (from $GOPATH)
    confd.go:10:2: cannot find package "github.com/kelseyhightower/confd/backends" in any of:
        /usr/local/go/src/github.com/kelseyhightower/confd/backends (from $GOROOT)
        /repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/backends (from $GOPATH)
    confd.go:11:2: cannot find package "github.com/kelseyhightower/confd/log" in any of:
        /usr/local/go/src/github.com/kelseyhightower/confd/log (from $GOROOT)
        /repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/log (from $GOPATH)
    confd.go:12:2: cannot find package "github.com/kelseyhightower/confd/resource/template" in any of:
        /usr/local/go/src/github.com/kelseyhightower/confd/resource/template (from $GOROOT)
        /repositories/git/opensource/confd/src/github.com/kelseyhightower/confd/resource/template (from $GOPATH)
    

    If this is in fact the correct approach, could we add it to the documentation, the ./build script and/or perhaps the new Vendor subdirectory? I know its pretty obvious on how to get the packages updated, but not having it in the build from source documentation lead me to think I was doing something wrong at first....might help others avoid the same suspicion (unless I am doing something wrong!).

    opened by jbkc85 21
  • Key Not Found Stops Template

    Key Not Found Stops Template

    When a key is missing or not found, the entire template stops processing. I saw there was an old issue about this and it was decided to continue generating the template - was that abandoned?

    I am doing:

    {{range $server := gets "/servers/*"}}

    If the /servers/ directory is empty, I get:

    executing "haproxy.cfg.tmpl" at <gets "/servers/...>: error calling gets: no keys match

    confd then stops processing.

    I would expect this to continue executing and throw the error or have a way for me to check that keys exists in there before running the gets. exists does not work because I will not know what would or would not exist in that servers dir.

    resource/template 
    opened by mpchlets 20
  • "if exists" doesn't work for directories in etcd

    Im running confd to monitor etcd for changes to update my HAProxy config, and I noticed that when I try using {{if exists /some/dir}}} it seems to fail even though the directory exists.

    Here's a fuller example:

    values in etcd

    /apps/webapp/node-1
    /apps/webapp/node-2
    

    /etc/confd/conf.d/haproxy.toml

    keys = [
        "/apps"
    ]
    

    /etcd/confd/templates/haproxy.cfg.toml

    {{if exists "/apps/webapp"}}
        {{range getvs "/apps/webapp/*"}}
            {{.}}
        {{end}}
    {{end}}
    

    So what Im wondering is, should if exists work on etcd directories, or is it limited to keys only? If it doesn't' currently work for directories, do you think it would make sense to add support for it? With this given use case, apps could register themselves under the /apps directory, making it so you don't have to update the list of keys. It also makes confd less prone to crashing when a specified key doesnt exist since everything will be nested under /app, rather than explicitly specifying a key/path for each app.

    resource/template 
    opened by seanmcgary 19
  • Integrate etcd and consul watch

    Integrate etcd and consul watch

    I took a quick stab at integrating etcd and consul's watch features per issue #56. This update basically wraps the go-etcd Watch command and consul's WatchList so they both have the same interface (go-etcd uses goroutines/channels while consul is a long-poll).

    It adds a -watch flag that takes precedence over any intervals: confd -watch -node 'http://127.0.0.1:4001' -confdir ~/confd

    opened by ryankee 19
  • I wish there was a `getv` equivalent that accepted a default value

    I wish there was a `getv` equivalent that accepted a default value

    My templates often end up being far too verbose because of things like this:

    option = {{if exists "/example/app/option"}}{{getv "/example/app/option"}}{{else}}default_value{{end}}
    

    That makes it quite hard to visually parse the config files. Much easier would be something like:

    option = {{getv "/example/app/option" "default_value"}}
    
    resource/template 
    opened by larsks 18
  • Owner and Group Always Set to 0 (

    Owner and Group Always Set to 0 ("root")

    I'm running confd v0.5.0 on an Ubuntu 14.04 LTS host and am having trouble with altering the owner and group for a template...no matter what I do, it changes things to be owned by the root user and root group, which is not desirable.

    /etc/confd/conf.d/example-json.toml

    [template]
    src = "example.json.tmpl"
    dest = "/etc/example.json"
    owner = "elasticdog"
    group = "elasticdog"
    mode = "0644"
    keys = [
      "/jdbc_url",
    ]
    reload_cmd = "/sbin/reload example-server"
    

    ...and it's creating the file with the desired contents, but the permissions are always set back to 0:

    $ ls -l /etc/example.json
    -rw-r--r-- 1 root root 1011 Aug  7 04:54 /etc/example.json
    $ id elasticdog
    uid=999(elasticdog) gid=999(elasticdog) groups=999(elasticdog)
    $ sudo chmod elasticdog:elasticdog /etc/example.json
    $ sudo reload confd
    

    /var/log/upstart/confd.log

    2014-08-07T04:54:21Z ip-10-193-17-174 /usr/local/bin/confd[2404]: INFO /etc/example.json has UID 999 should be 0
    2014-08-07T04:54:21Z ip-10-193-17-174 /usr/local/bin/confd[2404]: INFO /etc/example.json has GID 999 should be 0
    
    opened by elasticdog 18
  • how do you build?

    how do you build?

    This is the 1st time I try to build confd.

    go version
    go version go1.18.3 linux/amd64
    export GOPATH=/data
    

    I follow the instruction:

    $ mkdir -p $GOPATH/src/github.com/kelseyhightower
    $ git clone https://github.com/kelseyhightower/confd.git $GOPATH/src/github.com/kelseyhightower/confd
    $ cd $GOPATH/src/github.com/kelseyhightower/confd
    $ make
    

    then I got error:

    Building confd...
    go: cannot find main module, but found Gopkg.lock in /data/src/github.com/kelseyhightower/confd
            to create a module there, run:
            go mod init
    make: *** [build] Error 1
    

    If I run go mod init, I will get:

    go mod init
    go: creating new go.mod: module github.com/kelseyhightower/confd
    go: copying requirements from Gopkg.lock
    go: to add module requirements and sums:
            go mod tidy
    

    If I run go mod tidy, I will get:

    ……
    ……
    ……
    go: found github.com/armon/go-radix in github.com/armon/go-radix v1.0.0
    go: found github.com/hashicorp/go-memdb in github.com/hashicorp/go-memdb v1.3.3
    go: found github.com/hashicorp/golang-lru in github.com/hashicorp/golang-lru v0.5.4
    go: found github.com/keybase/go-crypto/openpgp in github.com/keybase/go-crypto v0.0.0-20200123153347-de78d2cb44f4
    go: found github.com/keybase/go-crypto/openpgp/packet in github.com/keybase/go-crypto v0.0.0-20200123153347-de78d2cb44f4                                                                                                                     
    go: found github.com/mitchellh/go-testing-interface in github.com/mitchellh/go-testing-interface v1.14.1
    go: found github.com/patrickmn/go-cache in github.com/patrickmn/go-cache v2.1.0+incompatible
    go: found github.com/ghodss/yaml in github.com/ghodss/yaml v1.0.0
    go: found github.com/grpc-ecosystem/grpc-gateway/runtime in github.com/grpc-ecosystem/grpc-gateway v1.16.0
    go: found github.com/tmc/grpc-websocket-proxy/wsproxy in github.com/tmc/grpc-websocket-proxy v0.0.0-20220101234140-673ab2c3ae75                                                                                                              
    go: found github.com/prometheus/client_golang/prometheus in github.com/prometheus/client_golang v1.12.2
    go: found github.com/jonboulle/clockwork in github.com/jonboulle/clockwork v0.3.0
    go: found golang.org/x/time/rate in golang.org/x/time v0.0.0-20220609170525-579cf78fd858
    go: found github.com/xiang90/probing in github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2
    go: found golang.org/x/sync/errgroup in golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f
    go: found github.com/golang/glog in github.com/golang/glog v1.0.0
    go: found github.com/coreos/bbolt in github.com/coreos/bbolt v1.3.6
    go: found github.com/hashicorp/go-plugin in github.com/hashicorp/go-plugin v1.4.4
    go: found github.com/hashicorp/go-version in github.com/hashicorp/go-version v1.5.0
    go: found github.com/grpc-ecosystem/grpc-gateway/utilities in github.com/grpc-ecosystem/grpc-gateway v1.16.0
    go: found github.com/google/btree in github.com/google/btree v1.1.1
    go: found github.com/golang/groupcache/lru in github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
    go: found github.com/hashicorp/yamux in github.com/hashicorp/yamux v0.0.0-20211028200310-0bc27b27de87
    go: found github.com/duosecurity/duo_api_golang in github.com/duosecurity/duo_api_golang v0.0.0-20220428205559-fa137a8ef05b                                                                                                                  
    go: found github.com/duosecurity/duo_api_golang/authapi in github.com/duosecurity/duo_api_golang v0.0.0-20220428205559-fa137a8ef05b                                                                                                          
    go: found github.com/gocql/gocql in github.com/gocql/gocql v1.1.0
    go: found github.com/SAP/go-hdb/driver in github.com/SAP/go-hdb v0.106.1
    go: found gopkg.in/mgo.v2 in gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22
    go: found github.com/denisenkom/go-mssqldb in github.com/denisenkom/go-mssqldb v0.12.2
    go: found github.com/go-sql-driver/mysql in github.com/go-sql-driver/mysql v1.6.0
    go: found github.com/lib/pq in github.com/lib/pq v1.10.6
    go: github.com/kelseyhightower/confd/backends/etcdv3 imports
            github.com/coreos/etcd/clientv3 tested by
            github.com/coreos/etcd/clientv3.test imports
            github.com/coreos/etcd/auth imports
            github.com/coreos/etcd/mvcc/backend imports
            github.com/coreos/bbolt: github.com/coreos/[email protected]: parsing go.mod:
            module declares its path as: go.etcd.io/bbolt
                    but was required as: github.com/coreos/bbolt
    

    If I run make AGAIN, I'll get this error:

    make
    Building confd...
    go: inconsistent vendoring in /data/src/github.com/kelseyhightower/confd:
            github.com/BurntSushi/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/aws/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/coreos/[email protected]+incompatible: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/coreos/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/fsnotify/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/garyburd/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/go-ini/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/gogo/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/golang/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/golang/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/hashicorp/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/jmespath/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/kelseyhightower/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/mitchellh/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/mitchellh/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/ryanuber/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/samuel/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/sethgrid/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/sirupsen/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/ugorji/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            github.com/xordataexchange/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            golang.org/x/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            golang.org/x/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            golang.org/x/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            golang.org/x/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            google.golang.org/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            google.golang.org/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
            gopkg.in/[email protected]: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt
    
            To ignore the vendor directory, use -mod=readonly or -mod=mod.
            To sync the vendor directory, run:
                    go mod vendor
    make: *** [build] Error 1
    

    any hint? please help.

    opened by xin-at-marathon 0
  • confd not able to retrieve credentials with IMDSv2

    confd not able to retrieve credentials with IMDSv2

    Hello. I am unable to retrieve aws credentials associated with my EC2 instance Role. I believe this is due to confd using an http GET request of the instance metadata. IMDSv2 requires an http PUT request. Is there a solution for using cond with IMDSv2?

    opened by bellackj 3
  • Confd can't authenticate with ssm backend

    Confd can't authenticate with ssm backend

    Hi,

    I'm running confd in in a container in eks. The pod has a service account which is associated with an iam role. https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html

    The AWS environment variables in the pod look like this. I've also tested without AWS_SDK_LOAD_CONFIG set with same results.

    AWS_SDK_LOAD_CONFIG=1
    AWS_REGION=us-east-2
    AWS_DEFAULT_REGION=us-east-2
    AWS_ROLE_ARN=<my role arn>
    AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
    

    I'm able to see valid aws configuration

    # aws configure list
          Name                    Value             Type    Location
          ----                    -----             ----    --------
       profile                <not set>             None    None
    access_key     ****************G3Z6 assume-role-with-web-identity    
    secret_key     ****************Uy6T assume-role-with-web-identity    
        region                us-east-2              env    AWS_DEFAULT_REGION
    

    I'm also able to access the ssm parameters through the aws cli

    # aws ssm get-parameter --name <my prefix>/<my ssm parameter name>
    {
        "Parameter": {
            "Name": "<my ssm parameter>",
            "Type": "SecureString",
            "Value": "<my value>",
            "Version": 1,
            "LastModifiedDate": 1626799800.065,
            "ARN": "<my arn>",
            "DataType": "text"
        }
    }
    

    However trying to access with confd ssm parameter backend returns an error

    confd -onetime -backend ssm -prefix /<my prefix> -log-level debug
    2022-02-02T19:39:08Z indigo-web-m-76cdb94fc8-jnx99 confd[39]: INFO Backend set to ssm
    2022-02-02T19:39:08Z indigo-web-m-76cdb94fc8-jnx99 confd[39]: INFO Starting confd
    2022-02-02T19:39:08Z indigo-web-m-76cdb94fc8-jnx99 confd[39]: INFO Backend source(s) set to 
    2022-02-02T19:39:08Z indigo-web-m-76cdb94fc8-jnx99 confd[39]: FATAL NoCredentialProviders: no valid providers in chain. Deprecated.
    	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
    

    I'm using confd version confd-0.16.0-linux-amd64 and awscli version aws-cli/1.22.46 Python/3.7.3 Linux/5.4.162-86.275.amzn2.x86_64 botocore/1.23.46. The container os is x86_64 GNU/Linux.

    My toml file looks like this

    [template]
    src  = "<my file tmpl>"
    dest = "<my file yaml>"
    mode = "0640"
    uid  = 1000
    gid  = 1000
    

    <my file tmpl> looks like this

    :ENV:
      <MY PARAM>: {{getv "/<my prefix>/<my ssm parameter name>" ""}}
    
    opened by hgibsonqb 2
  • x86_64 platform confd package needed

    x86_64 platform confd package needed

    I want to spinup the container with confd and some other packages. kindly provide me the link where i can download the confd package for x86_64

    Is confd package has removed from alpine linux dowload repository ???? if not removed kindly share the repository link and name of file ?

    opened by surendarkaniops 0
  • Can we have complex jsonArrays, like (Arr(Arr))

    Can we have complex jsonArrays, like (Arr(Arr))

    I had a requirement to parse this kind of structure: { "animals": [ { "type": "dog", "name": "Fido", "abc": [ "a", "b" ] }, { "type": "cat", "name": "Misse", "abc": [ "a", "b" ] } ] }

    I am trying to write a jsonArray inside a jsonArray but ending in errors like:

    at <.abc>: wrong type for value; expected string; got []interface {}

    My template looks like this: {{range jsonArray (getv "/animals")}} name: {{.name}} type: {{.type}} {{$abc := jsonArray .abc}} {{end}}

    opened by mudit-97 0
Releases(v0.16.0)
Owner
Kelsey Hightower
Kelsey Hightower
shops is a simple command-line tool written in Go that helps you simplify the way you manage configuration across a set of machines.

shops is a simple command-line tool written in Go that helps you simplify the way you manage configuration across a set of machines. shops is your configuration management tool of choice when Chef, Puppet, Ansible are all too complicated and all you really want to do is run a bunch of regular shell against a set of hosts.

James Mills 16 Jul 5, 2021
🛠 A configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP

config A small configuration library for Go that parses environment variables, JSON files, and reloads automatically on SIGHUP. Example func main() {

Josh Betz 210 Apr 10, 2022
Golang library for reading properties from configuration files in JSON and YAML format or from environment variables.

go-config Golang library for reading properties from configuration files in JSON and YAML format or from environment variables. Usage Create config in

null 3 Feb 22, 2022
Noel 2 Feb 13, 2022
A golang package for parsing ini-style configuration files

Mini Mini is a simple ini configuration file parser. The ini syntax supported includes: The standard name=value Comments on new lines starting with #

Stephen Asbury 30 Apr 2, 2022
Generate initial configuration files for Go.

Generate initial configuration files for Go.

ryota kise 3 Mar 26, 2022
parser for configuration files

config-loader config-loader supports to load config files and convert to map values. Supported format json Usage import ( "fmt" "github.com/tharun2

Tharun Rajendran 0 Nov 27, 2021
Tinyini - Bare-bones Go library for reading INI-like configuration files

tinyini tinyini is a minimalistic library for parsing INI-like configuration files. example configuration file globalkey = globalvalue [section] key

null 0 Jan 10, 2022
Tis module used as base fo configuration apps.By default, it expands into the inside of the application.

Tis module used as base fo configuration apps.By default, it expands into the inside of the application. Also, module c reads a dictionary of secrets from the application directory by its AppName and extension json.

LordTor 0 Dec 7, 2021
Golang library for managing configuration data from environment variables

envconfig import "github.com/kelseyhightower/envconfig" Documentation See godoc Usage Set some environment variables: export MYAPP_DEBUG=false export

Kelsey Hightower 4.2k Jul 4, 2022
YML2FSTAB - Convert from yml data to /etc/fstab configuration

YML2FSTAB - Convert from yml data to /etc/fstab configuration

null 0 Nov 1, 2021
Cfginterpolator is an interpolate library in golang allowing to include data from external sources in your configuration

cfginterpolator cfginterpolator is an interpolate library in golang allowing to include data from external sources in your configuration cfginterpolat

Benoit Bayszczak 0 Dec 14, 2021
goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configuration file.

goconfig goconfig uses a struct as input and populates the fields of this struct with parameters from command line, environment variables and configur

Go Sidekick 0 May 30, 2022
Go helpers to manage environment variables

Envh This library is made up of two parts : Env object : it wraps your environments variables in an object and provides convenient helpers. Env tree o

Anthony HAMON 95 Apr 14, 2022
An awesome command-line tool to manage Wireguard configurations.

wg-manage A command line tool to centrally manage Wireguard configuration files - all config options are stored in one YAML file that is then used to

null 31 Feb 1, 2022
✨Clean and minimalistic environment configuration reader for Golang

Clean Env Minimalistic configuration reader Overview This is a simple configuration reading tool. It just does the following: reads and parses configu

Ilya Kaznacheev 625 Jun 24, 2022
An opinionated configuration loading framework for Containerized and Cloud-Native applications.

Opinionated configuration loading framework for Containerized and 12-Factor compliant applications. Read configurations from Environment Variables, an

Sherif Abdel-Naby 81 May 2, 2022
go-up! A simple configuration library with recursive placeholders resolution and no magic.

go-up! A simple configuration library with placeholders resolution and no magic. go-up provides a simple way to configure an application from multiple

Francesco 37 Mar 23, 2022