Automatically update your Go tests

Overview

autogold - automatically update your Go tests Hexops logo

Go Reference

Go CI codecov Go Report Card

autogold makes go test -update automatically update your Go tests (golden files and Go values in e.g. foo_test.go.

~5m introduction available on YouTube:

"It's 2021: you shouldn't have to update Go tests manually"

Automatic golden files

Write in a Go test:

import "github.com/hexops/autogold"
...
autogold.Equal(t, got)

go test -update will now create/update a testdata/.golden file for you automatically.

Automatic inline test updating

Write in a Go test:

want := autogold.Want("my_test", nil)
want.Equal(t, got)

go test -update will automatically update the autogold.Want("my_test", ...) call with the Go syntax for whatever value your test got (complex Go struct, slices, strings, etc.)

Diffs

Anytime your test produces a result that is unexpected, you'll get very nice diffs showing exactly what changed. It does this by converting values at runtime directly to a formatted Go AST, and using the same diffing library the Go language server uses:

--- FAIL: TestEqual (0.08s)
    autogold.go:91: mismatch (-want +got):
        --- want
        +++ got
        @@ -1 +1 @@
        +&example.Baz{Name: "Jane", Age: 31}

Subtesting

Use table-driven Go subtests? autogold.Want and go test -update will automatically find and replace the nil values for you:

func TestTime(t *testing.T) {
	testCases := []struct {
		gmt  string
		loc  string
		want autogold.Value
	}{
		{"12:31", "Europe/Zuri", autogold.Want("Europe", nil)},
		{"12:31", "America/New_York", autogold.Want("America", nil)},
		{"08:08", "Australia/Sydney", autogold.Want("Australia", nil)},
	}
	for _, tc := range testCases {
		t.Run(tc.want.Name(), func(t *testing.T) {
			loc, err := time.LoadLocation(tc.loc)
			if err != nil {
				t.Fatal("could not load location")
			}
			gmt, _ := time.Parse("15:04", tc.gmt)
			got := gmt.In(loc).Format("15:04")
			tc.want.Equal(t, got)
		})
	}
}

It works by finding the relevant autogold.Want("", ...) call below the named TestTime function, and then replacing the nil parameter (or anything that was there.)

What are golden files, when should they be used?

Golden files are used by the Go authors for testing the standard library, the gofmt tool, etc. and are a common pattern in the Go community for snapshot testing. See also "Testing with golden files in Go" - Chris Reeves

Golden files make the most sense when you'd otherwise have to write a complex multi-line string or large Go structure inline in your test, making it hard to read.

In most cases, you should prefer inline snapshots, subtest golden values, or traditional Go tests.

Custom formatting

valast is used to produce Go syntax at runtime for the Go value you provide. If the default output is not to your liking, you have options:

  • Pass a string to autogold: It will be formatted as a Go string for you in the resulting .golden file / in Go tests.
  • Use your own formatting (JSON, etc.): Make your got value of type autogold.Raw("foobar"), and it will be used as-is for .golden files (not allowed with inline tests.)
  • Exclude unexported fields: autogold.Equal(t, got, autogold.ExportedOnly())

Backwards compatibility

  • As is the case with gofmt, different Go versions may produce different formattings (although rare.)
  • Minor versions of autogold (e.g. v1.0, v1.1) may alter the formatting of .golden files, although we will be mindful of such changes.
  • Major versions of autogold (e.g. v1, v2) will be used for any major changes in output that would be difficult to review (we expect this will be rare in practice.)

Alternatives comparison

The following are alternatives to autogold, making note of the differences we found that let us to create autogold:

Issues
  • Print in color

    Print in color

    This highlights the output in red/green and strips the -/+ characters.

    (the -/+ in this screenshot are actually part of the testdata - the -/+ from autogold have been stripped)

    CleanShot 2021-10-29 at 22 27 38@2x

    Resolves #15

    opened by chrismwendt 3
  • inline-update of autogold.Value with populated *string inside got inserts invalid Go code

    inline-update of autogold.Value with populated *string inside got inserts invalid Go code

    Hello! I've got a got-struct which contains string-pointers. Using inline-update, it adds the following lines to the struct:

    MobileCountryCode: &"xxx",
    MobileNetworkCode: &"xx",
    

    But that is not valid golang code, so it fails to build. Can this be fixed somehow? Thank you! I'm thinking maybe add autogold.String which returns a *string of the passed-in string?

    A complete repro is the following. go test -update inserts the line {name: "good", want: autogold.Want("good", &A{A: &"abc", B: "def"})},, which is no valid Go code.

    func TestGet(t *testing.T) {
    	tests := []struct {
    		name string
    		want autogold.Value
    	}{
    		{name: "good", want: autogold.Want("good", nil)},
    	}
    	for _, tt := range tests {
    		t.Run(tt.name, func(t *testing.T) {
    			got := Get()
    			tt.want.Equal(t, got)
    		})
    	}
    }
    
    type A struct {
    	A *string
    	B string
    }
    
    func Get() *A {
    	str := "abc"
    
    	return &A{
    		A: &str,
    		B: "def",
    	}
    }
    
    opened by fabstu 3
  • Issue with time.Time in golden struct

    Issue with time.Time in golden struct

    This works quite nicely...but I have an issue with structs using time.Time. Namely time.Time has a bunch of private internal variables, which naturally I cannot setup.

    • Can I ignore some fields in the struct?
    • Is there some special handling for time.Time?

    The time is stable since it's coming from the database.

    opened by nmiculinic 2
  • Infinite loop

    Infinite loop

    I've come across a possible bug where autogold gets stuck in an infinite loop, and I don't know why.

    Here's a simple repo that should be reproducible on your end: https://github.com/randallmlough/autogold-infinite-loop-issue

    In the ast package, uncomment the last two tests. Both those get stuck in an infinite loop.

    opened by randallmlough 2
  • want-data is partly dynamic and has to be adapated to got before equality-check

    want-data is partly dynamic and has to be adapated to got before equality-check

    I have want-data for which part of got is dynamic.

    My example is this. This is the want-data:

        - root:
            items:
            - download:
                ilias: Algos 2
                name: Algos1
                options:
                  download_folder: ""
            path: 02-ss2021
        path: %s
    

    I read this data, do want := fmt.Sprintf(wantRaw, path) and then compare with assert.Equal:

        - root:
            items:
            - download:
                ilias: Algos 2
                name: Algos1
                options:
                  download_folder: ""
            path: 02-ss2021
        path: /var/folders/rw/35q09zqj5yv5pz4wwg3yjfkm0000gn/T/TestSetup_basic278973801/001
    

    I can't see how to do this with autogold.Equal at the moment. Any ideas? I guess generics would be an easy solution but that's still a long time out. :-)

    opened by fabstu 2
  • add AUTOGOLD_PROFILE for profiling performance

    add AUTOGOLD_PROFILE for profiling performance

    e.g. for cases like https://github.com/sourcegraph/sourcegraph/pull/18189 this helps to debug where time is being spent:

    $ AUTOGOLD_PROFILE=t go test -v -run='TestUnionMerge/#02' ./cmd/frontend/graphqlbackend/
    === RUN   TestUnionMerge
    === RUN   TestUnionMerge/#02
    autogold: profile:
      getPackageNameAndPath: 289.5897ms
      stringify (want):      1.267376868s
      stringify (got):       1.377507419s
      diffing   (got):       56.162µs
      acquire path lock:     0s
      rewrite autogold.Want: 0s
    --- PASS: TestUnionMerge (2.93s)
        --- PASS: TestUnionMerge/#02 (2.93s)
    PASS
    ok  	github.com/sourcegraph/sourcegraph/cmd/frontend/graphqlbackend	3.380s
    

    Signed-off-by: Stephen Gutekanst [email protected]

    opened by slimsag 1
  • make Equal use directories for sub-tests

    make Equal use directories for sub-tests

    In practice, having sub-tests go into sub-directories i.e.:

    testdata/TestFoo/bar.golden
    

    Is much nicer, especially as most codebases with sub-tests have a large number of them.

    Signed-off-by: Stephen Gutekanst [email protected]

    opened by slimsag 0
  • Add inline code updating capability & Raw string type

    Add inline code updating capability & Raw string type

    This brings autogold closer to its intended glory, with the ability to update your foo_test.go files directly in addition to the original .golden file updating capability. See the updated README.md for details.

    It also adds a Raw string type, for use with outputting e.g. custom formats / raw strings to .golden files.

    opened by slimsag 0
  • Missing import path when inlining a value from a different package

    Missing import path when inlining a value from a different package

    Hey there, it seems like inline snapshots don't work when using types from other packages. In my case parser and parser_test.

    I've got the following package setup:

    parser/
      parser.go
      parser_test.go
      ast.go
    

    My AST is defined in ast.go:

    package parser
    
    type AST struct { ... }
    

    I'd like to use AST in parser_test:

    package parser_test
    
    func TestParse(t *testing.T) {
      state, err := parser.Parse("...")
      want := autogold.Want(`TestParse`, nil)
      want.Equal(t, state)
    }
    

    Unfortunately when I run go test ./parser_test.go -update, it prints the following:

    package parser_test
    
    import "parser"
    
    func TestParse(t *testing.T) {
      state, err := parser.Parse("...")
      want := autogold.Want(`TestParse`, &AST{ ... })
      want.Equal(t, state)
    }
    

    When it should be:

    package parser_test
    
    import "parser"
    
    func TestParse(t *testing.T) {
      state, err := parser.Parse("...")
      want := autogold.Want(`TestParse`, &parser.AST{ ... })
      want.Equal(t, state)
    }
    
    opened by matthewmueller 0
  • go test -update doesn't work when /tmp and workdir are different devices (?)

    go test -update doesn't work when /tmp and workdir are different devices (?)

    Hi! When running go test -update on a freshly created tests I get:

    ❯ go test -update                                                                                                                                                         2021-03-16 15:19:44
    --- FAIL: Test_WorstScores_nilResult (0.00s)
        autogold.go:82: rename testdata/nil.golden /tmp/go-golden-074934610/nil.golden: invalid cross-device link
    FAIL
    exit status 1
    FAIL    <pkg name redacted>  0.006s
    
    
    service/dashboard/score on master #.‍  
    ❯                                                                                                                                                                         2021-03-16 15:19:54
    

    relevant mounts:

    tmpfs on /tmp type tmpfs (rw,nosuid,nodev,nr_inodes=409600,inode64)
    /dev/mapper/mainroot on /home type btrfs (rw,relatime,compress=zstd:3,ssd,space_cache,subvolid=258,subvol=/@home)
    

    This is weird, I assume that most people have /tmp mounted as tmpfs, so someone else should already encounter this error.

    opened by utrack 1
  • Remove Redundant Type

    Remove Redundant Type

    Description

    One of the built-in Goland linter gives warning about the redundant type. It would be nice if the generated result doesn't contain redundant fields. Help the code become cleaner.

    Problem

    Redundant type 
     Inspection info: Reports redundant types in composite literals.
    For example:
        [][]int{[]int{1}, []int{2}}
    can be simplified to:
        [][]int {{1}, {2}}
    .
    

    Example in Goland

    The grey-out fields are redundant types. Screenshot_2021-01-31_17-57-23

    opened by rizalgowandy 1
  • [WIP] remove the need for passing names to autogold.Want

    [WIP] remove the need for passing names to autogold.Want

    This is a WIP PR which currently allows autogold.Want("name", ...) calls anywhere in the file but requires the name be unique within the scope of the entire file (rather than just in the TestFoo function).

    I will update this PR soon to remove the name parameter entirely, see #11

    opened by slimsag 0
  • Make it possible to use go-cmp with autogold.Want inline updates if so desired

    Make it possible to use go-cmp with autogold.Want inline updates if so desired

    Not unreasonable that someone may have a fair amount of go-cmp tests and want to integrate autogold slowly. We could allow this by having autogold.Want(...).Value() return the interface() value it got, and allowing the user to signal if the test needs updating or not. e.g. they call .Value(), pass it to go-cmp, and invoke .Fail() or something like that.

    v2 
    opened by slimsag 0
Owner
Hexops
Experiment everywhere
Hexops
Record and replay your HTTP interactions for fast, deterministic and accurate tests

go-vcr go-vcr simplifies testing by recording your HTTP interactions and replaying them in future runs in order to provide fast, deterministic and acc

Marin Atanasov Nikolov 893 Jul 27, 2022
A next-generation testing tool. Orion provides a powerful DSL to write and automate your acceptance tests

Orion is born to change the way we implement our acceptance tests. It takes advantage of HCL from Hashicorp t o provide a simple DSL to write the acceptance tests.

Wesovi Labs 43 Jul 16, 2022
Terratest is a Go library that makes it easier to write automated tests for your infrastructure code.

Terratest is a Go library that makes it easier to write automated tests for your infrastructure code. It provides a variety of helper functions and patterns for common infrastructure testing tasks,

Gruntwork 6.2k Aug 4, 2022
Automatically generate Go test boilerplate from your source code.

gotests gotests makes writing Go tests easy. It's a Golang commandline tool that generates table driven tests based on its target source files' functi

Charles Weill 4.1k Aug 4, 2022
CLI tool to mock TCP connections. You can use it with Detox, Cypress or any other framework to automatically mock your backend or database.

Falso It is a CLI that allows you to mock requests/responses between you and any server without any configuration or previous knowledge about how it w

Sorin 3 Mar 15, 2022
Package for comparing Go values in tests

Package for equality of Go values This package is intended to be a more powerful and safer alternative to reflect.DeepEqual for comparing whether two

Google 3k Jul 28, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Maxime Soulé 312 Aug 4, 2022
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

SmartyStreets 7.3k Aug 1, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Esko Luontola 112 Apr 5, 2022
Ruby on Rails like test fixtures for Go. Write tests against a real database

testfixtures Warning: this package will wipe the database data before loading the fixtures! It is supposed to be used on a test database. Please, doub

null 826 Aug 1, 2022
A simple `fs.FS` implementation to be used inside tests.

testfs A simple fs.FS which is contained in a test (using testing.TB's TempDir()) and with a few helper methods. PS: This lib only works on Go 1.16+.

Carlos Alexandro Becker 31 Mar 3, 2022
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

SmartyStreets 7.4k Aug 7, 2022
gostub is a library to make stubbing in unit tests easy

gostub gostub is a library to make stubbing in unit tests easy. Getting started Import the following package: github.com/prashantv/gostub Click here t

Prashant Varanasi 243 Jul 29, 2022
Golang application focused on tests

Maceio Golang application that listens for webhook events coming from Github, runs tests previously defined in a configuration file and returns the ou

Leonardo Damasceno 5 Sep 8, 2021
Robust framework for running complex workload scenarios in isolation, using Go; for integration, e2e tests, benchmarks and more! 💪

e2e Go Module providing robust framework for running complex workload scenarios in isolation, using Go and Docker. For integration, e2e tests, benchma

null 105 Jul 20, 2022
gomonkey is a library to make monkey patching in unit tests easy

gomonkey is a library to make monkey patching in unit tests easy, and the core idea of monkey patching comes from Bouke, you can read this blogpost for an explanation on how it works.

Zhang Xiaolong 1.2k Aug 1, 2022
A simple and expressive HTTP server mocking library for end-to-end tests in Go.

mockhttp A simple and expressive HTTP server mocking library for end-to-end tests in Go. Installation go get -d github.com/americanas-go/mockhttp Exa

Americanas Go 6 Dec 19, 2021
How we can run unit tests in parallel mode with failpoint injection taking effect and without injection race

This is a simple demo to show how we can run unit tests in parallel mode with failpoint injection taking effect and without injection race. The basic

amyangfei 1 Oct 31, 2021
Package has tool to generate workload for vegeta based kube-api stress tests.

Package has tool to generate workload for vegeta based kube-api stress tests.

Mikhail Sakhnov 0 Nov 22, 2021