Go package that interfaces with AWS System Manager

Overview

Build Status codecov Go Report Card GoDoc

go-aws-ssm

Go package that interfaces with AWS System Manager.

Why to use go-aws-ssm and not the aws-sdk-go?

This package is wrapping the aws-sdk-go and hides the complexity dealing with the not so Go friendly AWS SDK. Perfect use case for this package is when secure parameters for an application are stored to AWS Parameter Store using a path hierarchy. During application startup you can use this package to fetch them and use them in your application.

Install

go get github.com/PaddleHQ/go-aws-ssm

Examples

Basic Usage

        //Assuming you have the parameters in the following format:
    	//my-service/dev/param-1  -> with value `a`
    	//my-service/dev/param-2  -> with value `b`
    	pmstore, err := awsssm.NewParameterStore()
    	if err != nil {
    		return err
    	}
    	//Requesting the base path
    	params, err := pmstore.GetAllParametersByPath("/my-service/dev/", true)
    	if err!=nil{
    		return err
    	}
    	
    	//And getting a specific value
    	value:=params.GetValueByName("param-1")
    	//value should be `a`
    	
    	

Integrates easily with viper

        //Assuming you have the parameters in the following format:
     	//my-service/dev/param-1  -> with value `a`
     	//my-service/dev/param-2  -> with value `b`
     	pmstore, err := awsssm.NewParameterStore()
     	if err != nil {
     		return err
     	}
     	//Requesting the base path
     	params, err := pmstore.GetAllParametersByPath("/my-service/dev/", true)
     	if err!=nil{
     		return err
     	}
    
    	//Configure viper to handle it as json document, nothing special here!
    	v := viper.New()
    	v.SetConfigType(`json`)
    	//params object implements the io.Reader interface that is required
    	err = v.ReadConfig(params)
    	if err != nil {
    		return err
    	}
    	value := v.Get(`param-1`)
    	//value should be `a`
Comments
  • PutSecureParameter and recursive GetAllParametersByPath

    PutSecureParameter and recursive GetAllParametersByPath

    • Add easier way to set secure SSM parameters
    • recursively get parameters for a given path. By recursive, I mean that it gets more than the limit of 10 parameters per invocation, which is an AWS API limit.
    opened by ingshtrom 6
  • New session with options

    New session with options

    Hi, i tried the basic usage, but it does not get the default region unfortunately. So, i added the region as;

    pmstore, err := awsssm.NewParameterStore(aws.NewConfig().WithRegion("eu-west-1"))

    Like this, it works. But i don't want to pass region in code. And way to do it, we need create NewSessionWithOptions and enable the SharedConfig, as;

    func NewParameterStore(ssmConfig ...*aws.Config) (*ParameterStore, error) {

    sessionAWS, err := session.NewSessionWithOptions(session.Options{
    	SharedConfigState: session.SharedConfigEnable,
    })
    if err != nil {
    	return nil, err
    }
    svc := ssm.New(sessionAWS)
    return &ParameterStore{ssm: svc}, nil 
    

    }

    Is there any way to do it without manipulating this function ?

    opened by icy95 1
  • Trying to read parameter store value greater than 512 bytes into viper returns 'While parsing config: unexpected end of JSON input` error

    Trying to read parameter store value greater than 512 bytes into viper returns 'While parsing config: unexpected end of JSON input` error

    Hi,

    I'm storing a relatively large value in AWS Param Store that is greater than 512 bytes. What I'm seeing is that when I try to fetch that value from Param Store, it gets the full value, but when I try to feed that value into viper using viper.ReadConfig(params) it only feeds the first 512 bytes of the param value.

    Code:

    params, err = ssm.GetAllParametersByPath(ssmPath, true)
    if err != nil {
        return
    }
    
    // inspecting the individual params here shows that the full value has been captured
    
    err = viper.ReadConfig(params)
    // err returned: While parsing config: unexpected end of JSON input
    

    I've pinpointed where the error is coming from: In buffer.go, the minimum size of the buffer initialized in buffer.ReadFrom(r io.Reader) is 512 bytes. It is suppose to automatically increase the size of the buffer when it gets to that predefined size and doesn't find an EOF char during r.Read(...):

    // Buffer.ReadFrom. As long as the Buffer has at least MinRead bytes beyond
    // what is required to hold the contents of r, ReadFrom will not grow the
    // underlying buffer.
    const MinRead = 512
    
    // ReadFrom reads data from r until EOF and appends it to the buffer, growing
    // the buffer as needed. The return value n is the number of bytes read. Any
    // error except io.EOF encountered during the read is also returned. If the
    // buffer becomes too large, ReadFrom will panic with ErrTooLarge.
    func (b *Buffer) ReadFrom(r io.Reader) (n int64, err error) {
    	b.lastRead = opInvalid
    	for {
    		i := b.grow(MinRead)
    		b.buf = b.buf[:i]
    		m, e := r.Read(b.buf[i:cap(b.buf)])
    		if m < 0 {
    			panic(errNegativeRead)
    		}
    
    		b.buf = b.buf[:i+m]
    		n += int64(m)
    		if e == io.EOF {
    			return n, nil // e is EOF, so return nil explicitly
    		}
    		if e != nil {
    			return n, e
    		}
    	}
    }
    

    parameter.go implements the io.Reader's Read like so:

    func (p *Parameters) Read(des []byte) (n int, err error) {
    	bytesJSON, err := json.Marshal(p.getKeyValueMap())
    	if err != nil {
    		return 0, err
    	}
    	return copy(des, bytesJSON), io.EOF
    }
    

    This means that we are returning an io.EOF regardless of if we've read the full param value.

    So we return the first 512 bytes back to buffer.ReadFrom(...) along with the io.EOF, which buffer.ReadFrom(..) interprets as being finished with the full message, so it doesn't try to enlarge the buffer and read more of the value, thus chopping off the end of the value (including the end JSON }), which is why we end up with the unexpected end of JSON content error.

    The limit on the value as described by AWS for Param Store entries is 8Kb for advanced keys or 4Kb for normal keys, so I felt I should bring this issue to the repo's owners attention.

    Hope you are having a good day!

    opened by mjevans93308 1
  • Issue with copy in method Read.

    Issue with copy in method Read.

    Got issue here because in my case bytesJSON has 790 bytes but viper uses default bytes.Buffer with MinRead = 512 and io.EOF on line 44 means that my JSON will be corrupted. So looks like Read must be implemented in better way...

    opened by cn007b 1
  • Refactor get parameters by path call to not have 10 params limit

    Refactor get parameters by path call to not have 10 params limit

    • Refactor get parameters by path call to not have 10 params limit
    • Refactor to use GetParametersByPathPagesthat does the calls recursivly for us

    Closes #28

    opened by geototti21 0
  • Update Parameter Store Costructors

    Update Parameter Store Costructors

    Update constructors to not be very Paddle specific. Breaking Public API by removing exported methods that are not used by anyone internally (lets break things before we open-source it) 😬 Closes #10

    gotham 
    opened by geototti21 0
Releases(v0.8.0)
Owner
Paddle
Paddle
Using Eww Widgets to create a replacement for my bumblebee-status bars in i3 Window Manager.

Eww Config What is it Using Eww Widgets to create a replacement for my bumblebee-status bars in i3 Window Manager. Eww allows you to create widgets an

Owen Rumney 0 Nov 18, 2021
Butler CMS (Configuration Management System)

Butler CMS Butler CMS (Configuration Management System) Overview The Butler CMS (butler) tool is designed to grab any configuration files, defined in

Adobe, Inc. 13 Nov 17, 2021
ezd is an easy to configure docker-based task runner system

ezd - eZ Docker Task Runner ezd is an easy to configure docker-based task runner system. Getting started Create an ezd.yml file in your project root:

Daniel Christian Kupczak 4 Feb 11, 2022
A lightweight yet powerful config package for Go projects

Config GoLobby Config is a lightweight yet powerful config package for Go projects. It takes advantage of env files and OS variables alongside config

GoLobby 293 Sep 28, 2022
Configure is a Go package that gives you easy configuration of your project through redundancy

Configure Configure is a Go package that gives you easy configuration of your project through redundancy. It has an API inspired by negroni and the fl

Harrison Shoebridge 56 Sep 21, 2022
Package ini provides INI file read and write functionality in Go.

INI Package ini provides INI file read and write functionality in Go. Features Load from multiple data sources(file, []byte, io.Reader and io.ReadClos

INI 3.1k Sep 22, 2022
A golang package for parsing ini-style configuration files

Mini Mini is a simple ini configuration file parser. The ini syntax supported includes: The standard name=value Comments on new lines starting with #

Stephen Asbury 31 Jul 7, 2022
A cross platform package that follows the XDG Standard

XDG A cross platform package that tries to follow XDG Standard when possible. Since XDG is linux specific, I am only able to follow standards to the T

null 71 Sep 26, 2022
Flags-first package for configuration

ff stands for flags-first, and provides an opinionated way to populate a flag.FlagSet with configuration data from the environment.

Peter Bourgon 1.1k Sep 18, 2022
Environment variables configuration package for Go microservices.

gocfg Environment variables configuration package for Go microservices. It helps validate environment variable values and set default values if needed

Sergey Prokhorov 0 Dec 30, 2021
SmartYAML - Go package to handle YAML

SmartYAML - Go package to handle YAML The smartyaml is a go package to handle parsed YAML files more confortable. This package is not a parser, it use

Péter Deák 0 Feb 25, 2022
Yet another config package

cfg Yet another config package Features Read from file Read from environment variable Hot reload of the file Usage package main import ( "context"

infinite loop 3 Jan 16, 2022
Lightweight package that makes easier and safer to deal with environment variables.

Envisage A lightweight package that makes easier and safer to deal with environment variables. Example Try it on On GoPlay https://goplay.tools/snippe

GOLang Sugar 4 Apr 11, 2022
Generate TypeScript interfaces from Go structs/interfaces - useful for JSON RPC

bel Generate TypeScript interfaces from Go structs/interfaces - useful for JSON RPC bel is used in production in https://gitpod.io. Getting started be

Christian Weichel 27 Sep 3, 2022
Golang-Devnet-Interfaces - Quick example of using Scrapligo and TextFSM to parse interfaces from Devnet Sandbox

Scrapligo & TextFSM This is a simple example of using the Scrapligo library deve

null 4 Mar 7, 2022
Aws-secretsmanager-caching-extension - Cache server for AWS Secrets Manager

AWS Lambda Extension / Sidecar Container Cache Server The cache server is writte

CustomerGauge 6 Aug 12, 2022
Go Package Manager (gopm) is a package manager and build tool for Go.

?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? In favor of Go Modules Proxy since Go 1.11, this pr

Go Package Manager 2.5k Sep 6, 2022
A thin go client that interfaces with AWS SSM

go-ssm-aws A thin go client that interfaces with AWS SSM. Why this package? This

Mayank Thakur 1 May 14, 2022
Go package providing simple database and server interfaces for the CSV files produced by the sfomuseum/go-libraryofcongress package

go-libraryofcongress-database Go package providing simple database and server interfaces for the CSV files produced by the sfomuseum/go-libraryofcongr

San Francisco International Airport Museum 1 Oct 29, 2021
A package for access aws service using AWS SDK for Golang

goaws ?? A package for access aws service using AWS SDK for Golang Advantage with goaws package Example for get user list IAM with AWS SDK for Golang

Muhammad Ichsanul Fadhil 1 Nov 25, 2021