Simple Go snapshot testing

Overview

Mascot
Build Status Coverage Status Go Report Card GoDoc

Incredibly simple Go snapshot testing: cupaloy takes a snapshot of your test output and compares it to a snapshot committed alongside your tests. If the values don't match then the test will be failed.

There's no need to manually manage snapshot files: just use the cupaloy.SnapshotT(t, value) function in your tests and cupaloy will automatically find the relevant snapshot file (based on the test name) and compare it with the given value.

Usage

Write a test

Firstly, write a test case generating some output and pass this output to cupaloy.SnapshotT:

func TestParsing(t *testing.T) {
    ast := ParseFile("test_input")

    // check that the result is the same as the last time the snapshot was updated
    // if the result has changed (e.g. because the behaviour of the parser has changed)
    // then the test will be failed with an error containing a diff of the changes
    cupaloy.SnapshotT(t, ast)
}

The first time this test is run, a snapshot will be automatically created (using the github.com/davecgh/go-spew package).

Update a snapshot

When the behaviour of your software changes causing the snapshot to change, this test will begin to fail with an error showing the difference between the old and new snapshots. Once you are happy that the new snapshot is correct (and hasn't just changed unexpectedly), you can save the new snapshot by setting the UPDATE_SNAPSHOTS environment and re-running your tests:

UPDATE_SNAPSHOTS=true go test ./...

This will fail all tests where the snapshot was updated (to stop you accidentally updating snapshots in CI) but your snapshot files will now have been updated to reflect the current output of your code.

Supported formats

Snapshots of test output are generated using the github.com/davecgh/go-spew package which uses reflection to deep pretty-print your test result and so will support almost all the basic types (from simple strings, slices, and maps to deeply nested structs) without issue. The only types whose contents cannot be fully pretty-printed are functions and channels.

The most important property of your test output is that it is deterministic: if your output contains timestamps or other fields which will change on every run, then cupaloy will detect this as a change and so fail the test.

Further Examples

Table driven tests

var testCases = map[string][]string{
    "TestCaseOne": []string{......},
    "AnotherTestCase": []string{......},
    ....
}

func TestCases(t *testing.T) {
    for testName, args := range testCases {
        t.Run(testName, func(t *testing.T) {
            result := functionUnderTest(args...)
            cupaloy.SnapshotT(t, result)
        })
    }
}

Changing output directory

func TestSubdirectory(t *testing.T) {
    result := someFunction()
    snapshotter := cupaloy.New(cupaloy.SnapshotSubdirectory("testdata"))
    err := snapshotter.Snapshot(result)
    if err != nil {
        t.Fatalf("error: %s", err)
    }
}

For further usage examples see basic_test.go and advanced_test.go in the examples/ directory which are both kept up to date and run on CI.

Comments
  • fix: replace `\r\n` with `\n` on windows

    fix: replace `\r\n` with `\n` on windows

    Depending on how you setup git on windows, it might convert between line separators. For Github actions it seems to be the case that \n is replaced by \r\n, as we have a constant failure here: https://github.com/ory/x/runs/3937200972 but the content is identical when copied and compared via diff. In contrast, a run with this fix as a replacement succeeds: https://github.com/ory/x/runs/3937395472

    opened by zepatrik 7
  • Add SnapshotMultiT function

    Add SnapshotMultiT function

    I would like to have multiple snapshots from tests cases in table driven tests. For that, I would need a combination of SnapshotMulti and SnapshotT. Would it make sense to have an additional SnapshotTMulti or SnapshotMultiT functions?

    var testCases = map[string][]string{
        "TestCaseOne": []string{......},
        "AnotherTestCase": []string{......},
        ....
    }
    
    for testName, args := range testCases {
            t.Run(testName, func(t *testing.T) {
                    myMock := MyMock{
                             MockFunc: function (input string) {
                                    cupaloy.SnapshotTMulti(t, "input", input)
                             },
                    }
                    result := functionUnderTest(myMock, args...)
                    cupaloy.SnapshotTMulti(t, "result", result)
            })
    }
    

    Another possibility would be that the *testing.T can be passed into the constructor of Cupaloy (This is how https://github.com/matryer/is works):

    var testCases = map[string][]string{
        "TestCaseOne": []string{......},
        "AnotherTestCase": []string{......},
        ....
    }
    
    for testName, args := range testCases {
            t.Run(testName, func(t *testing.T) {
                    cupaloy := cupaloy.New(cupaloy.T(t))
    
                    myMock := MyMock{
                             MockFunc: function (input string) {
                                    cupaloy.SnapshotMulti("input", input)
                             },
                    }
                    result := functionUnderTest(myMock, args...)
                    cupaloy.SnapshotMulti("result", result)
            })
    }
    
    opened by cloudlena 5
  • Sort map keys when spewing

    Sort map keys when spewing

    I'm trying to snapshot a struct whose one of its fields is a map. Sometimes the test passes, at other times it doesn't. Go maps are randomly sorted by default. However, go-spew has a SortKeys configuration that solves this problem, but it doesn't seem to be used here.

    I'm sorry if I'm missing something here, I didn't read the source code so I'm guessing it's a bug and which bug it is, but I might have missed some docs.

    Thanks nonetheless!

    bug 
    opened by gbrlsnchs 4
  • Allow ignoring contents of specific structure fields

    Allow ignoring contents of specific structure fields

    Some struct fields might be known and intended to not remain constant between test runs. It would then be required to ignore these fields (perhaps just by overwriting with a constant value) Arguably tests should be deterministic though.

    enhancement 
    opened by bradleyjkemp 4
  • Update cupaloy.go

    Update cupaloy.go

    cupaloy.Global = cupaloy.Global.WithOptions(
    		cupaloy.CreateNewAutomatically(false),
    		cupaloy.EnvVariableName(updateSnapshotsEnv),
    		cupaloy.FailOnUpdate(false),
    		cupaloy.ShouldUpdate(func() bool { return updateSnapshot}),
    		cupaloy.SnapshotSubdirectory(updateSnapshotsDir),
    	)
    

    Sorry missed this earlier. This is the set of config I'm using. When snapshots don't exist and I'm running in bazel setting it is not failing as expected.

    opened by byang223 3
  • can't install

    can't install

    docs here https://pkg.go.dev/github.com/pyros2097/cupaloy#section-readme say use

    go get -u github.com/bradleyjkemp/cupaloy
    

    README on here doesn't even mention how to install. Not even sure if same package.

    I get

    go get: module github.com/bradleykemp/cupaloy: git ls-remote -q origin in /Users/kevin/.go/pkg/mod/cache/vcs/80d9f6d7d4a4bdada3cd1ba5dfc1c3543d1449b3cb2ccd046ae294400d51c447: exit status 128:
    	fatal: could not read Username for 'https://github.com': terminal prompts disabled
    Confirm the import path was entered correctly.
    

    No other package in my current project is having this problem.

    However, I got a clue from pkg.go.dev docs

    go get github.com/pyros2097/cupaloy
    

    which worked.

    opened by kswope 2
  • Add an option to disable usage of stringer methods.

    Add an option to disable usage of stringer methods.

    This is especially annoying for snapshotting custom errors that contain some payload. The payload would arbitrary be dismissed which is confusing.

    fixes #45

    opened by gilbsgilbs 2
  • Use of internal package not allowed

    Use of internal package not allowed

    After installing this package using govendor running the tests it fails with the error

    package command-line-arguments (test)
            imports github.com/bradleyjkemp/cupaloy
            imports github.com/bradleyjkemp/cupaloy/v2/internal: use of internal package github.com/bradleyjkemp/cupaloy/internal not allowed
    FAIL    command-line-arguments [setup failed]
    
    opened by cyberxander90 2
  • Make SnapshotT() a no-op for failed tests

    Make SnapshotT() a no-op for failed tests

    Skips a cupaloy.SnapshotT() invocation when the test has already failed. This avoids saving test snapshots containing known-bad data on a test's first invocation.

    cc #31

    opened by wgoodall01 2
  • Add negative testing of SnapshotT

    Add negative testing of SnapshotT

    Add some tests for when SnapshotT is called on a changed value and ensure that this would fail the test.

    Slightly non-trivial because we need to mock testing.T in order to check that this would fail the user's test but without failing our test but should be pretty simple with something like github.com/matryer/moq

    testing 
    opened by bradleyjkemp 2
  • Use a backwards compatible snapshot format

    Use a backwards compatible snapshot format

    Currently we use a raw text snapshot format. If we moved to something like JSON then we could version this to allow for breaking changes to the snapshot format without breaking everyone's tests.

    opened by bradleyjkemp 2
  • Bump github.com/stretchr/testify from 1.6.1 to 1.8.1

    Bump github.com/stretchr/testify from 1.6.1 to 1.8.1

    Bumps github.com/stretchr/testify from 1.6.1 to 1.8.1.

    Release notes

    Sourced from github.com/stretchr/testify's releases.

    Minor improvements and bug fixes

    Minor feature improvements and bug fixes

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • Snapshot filename special character scrub needed - semicolon

    Snapshot filename special character scrub needed - semicolon

    A recent commit https://github.com/jf-tech/omniparser/commit/1b4997b548aabc2579a388a00964f808aa479582 introduced a snapshot filename containing ;. While itself is fine, i.e. running tests is all good, we ran into issues when the module being imported by 3rd party, hitting an error:

    extensions/omniv21/fileformat/flatfile/fixedlength/.snapshots/TestReadAndMatchHeaderFooterBasedEnvelope-empty_buf;_single_line_header-footer_match: malformed file path "extensions/omniv21/fileformat/flatfile/fixedlength/.snapshots/TestReadAndMatchHeaderFooterBasedEnvelope-empty_buf;_single_line_header-footer_match": invalid char ';'
    

    Turns out ; isn't allowed in module filepath: https://cs.opensource.google/go/x/mod/+/master:module/module.go;l=285;drc=02c991387e35f9e4c0a9b7ce137717055f8d716d

    Your library already does many of the special character replacement during snapshot file creation. Wondering if ; semicolon can be added?

    Cheers!

    opened by jf-tech 0
  • Support for JSON Snapshots

    Support for JSON Snapshots

    Currently the diffing tool used by Cupaloy treats strings as just a sequence of characters. Which is reasonable, but limited.

    In particular, it would be fantastic if there were a way to tell it to treat the two strings as JSON. Which would mean:

    • Whitespace is ignored
    • Object keys are unordered

    That way, any changes to the output that are changes to the raw string but are not meaningful changes to the JSON structure are ignored and will not cause the tests to fail.

    In particular, I've had an issue where VSCode auto-formats the JSON within a snapshot file when I'm making changes to it, which then means that everything breaks. Often it actually means it's quicker to delete the file and regenerate it rather than just fixing up one value that's changed because of a code change.

    opened by sazzer 1
  • Write failed snapshot (#70)

    Write failed snapshot (#70)

    Hey, first of all, I started to learn go couple month ago, and I used this lib already in a couple projects. I really like to use it but I also found it a bit hard to read bigger diffs sometimes. So I gave it a shot to implement this. Please let me know if something is wrong or could be done better. I'm looking forward to your feedback.

    P.S.: this is my first open-source contribution ever, so please be nice. ;)

    opened by ninogresenz 0
  • Feature request: support Property Matchers like Jest

    Feature request: support Property Matchers like Jest

    Jest has a great feature for matching specific types for fields: https://jestjs.io/docs/snapshot-testing#property-matchers

    It would be great if this library also support this.

    opened by cdignam-segment 1
  • Add an option to write failed/changed snapshots to disk

    Add an option to write failed/changed snapshots to disk

    Debugging large snapshot changes in the terminal is difficult. #64 could help but you might still want to view the two files side by side.

    Suggested solution is adding a new flag/option that writes failing snapshots to a file with an extra suffix e.g. .failed

    Requested by @thecodejunkie: https://twitter.com/TheCodeJunkie/status/1388797594521022464

    enhancement 
    opened by bradleyjkemp 0
Releases(v2.8.0)
  • v2.8.0(Sep 14, 2022)

    What's Changed

    • Add option to specify custom snapshot differ by @smitt04 in https://github.com/bradleyjkemp/cupaloy/pull/64

    New Contributors

    • @smitt04 made their first contribution in https://github.com/bradleyjkemp/cupaloy/pull/64

    Full Changelog: https://github.com/bradleyjkemp/cupaloy/compare/v2.7.0...v2.8.0

    Source code(tar.gz)
    Source code(zip)
  • v2.5.0(Mar 24, 2019)

  • v2.4.0(Jan 19, 2019)

  • v2.3.0(Oct 21, 2018)

    Add new options to control:

    • Snapshot file extension
    • Whether tests fail immediately on snapshot mismatch
    • Whether tests should fail on update/create of snapshots
    Source code(tar.gz)
    Source code(zip)
Owner
Bradley Kemp
Bradley Kemp
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Re 296 Dec 12, 2022
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Design it, Run it 1 Nov 24, 2022
🚀🌏 Orbital is a simple end-to-end testing framework for Go

Orbital is a test framework which enables a developer to write end to end tests just like one would writing unit tests. We do this by effectively copying the testing.T API and registering tests to be run periodically on a configured schedule.

Segment 79 Nov 18, 2022
Simple Golang Load testing app built on top of vegeta

LOVE AND WAR : Give Your App Love By Unleashing War Simple load testing app to test your http services Installation Build docker image: docker build -

Raymond Gitonga 1 Oct 26, 2021
Simple auth service for testing purposes

Auth mock service Simple auth service for testing purposes. Based on https://github.com/stefanprodan/podinfo Credentials: { "username": "username-ro

Kajetan 0 Oct 5, 2021
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Fortio (Φορτίο) 2.8k Jan 2, 2023
:exclamation:Basic Assertion Library used along side native go testing, with building blocks for custom assertions

Package assert Package assert is a Basic Assertion library used along side native go testing Installation Use go get. go get github.com/go-playground/

Go Playgound 52 Jan 6, 2023
Expressive end-to-end HTTP API testing made easy in Go

baloo Expressive and versatile end-to-end HTTP API testing made easy in Go (golang), built on top of gentleman HTTP client toolkit. Take a look to the

Tom 748 Dec 13, 2022
Clean database for testing, inspired by database_cleaner for Ruby

DbCleaner Clean database for testing, inspired by database_cleaner for Ruby. It uses flock syscall under the hood to make sure the test can runs in pa

Scott Le 146 Nov 17, 2022
Golang HTTP client testing framework

flute Golang HTTP client testing framework Presentation https://speakerdeck.com/szksh/flute-golang-http-client-testing-framework Overview flute is the

Shunsuke Suzuki 17 Sep 27, 2022
API testing framework inspired by frisby-js

frisby REST API testing framework inspired by frisby-js, written in Go Proposals I'm starting to work on frisby again with the following ideas: Read s

_Hofstadter 274 Sep 27, 2022
Mutation testing for Go source code

go-mutesting go-mutesting is a framework for performing mutation testing on Go source code. Its main purpose is to find source code, which is not cove

Markus Zimmermann 568 Dec 28, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Maxime Soulé 334 Dec 22, 2022
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

null 869 Dec 25, 2022
A Comprehensive Coverage Testing System for The Go Programming Language

goc 中文页 | goc is a comprehensive coverage testing system for The Go Programming Language, especially for some complex scenarios, like system testing c

Qiniu Cloud 613 Jan 8, 2023
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

SmartyStreets 7.6k Dec 30, 2022
Testing API Handler written in Golang.

Gofight API Handler Testing for Golang Web framework. Support Framework Http Handler Golang package http provides HTTP client and server implementatio

Bo-Yi Wu 428 Dec 16, 2022
Library created for testing JSON against patterns.

Gomatch Library created for testing JSON against patterns. The goal was to be able to validate JSON focusing only on parts essential in given test cas

null 44 Oct 28, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Esko Luontola 113 Nov 28, 2022