A collection of packages to augment the go testing package and support common patterns.

Overview

gotest.tools

A collection of packages to augment testing and support common patterns.

GoDoc CircleCI Go Reportcard

Usage

With Go modules enabled (go1.11+)

$ go get gotest.tools/v3
import "gotest.tools/v3/assert"

To use gotest.tools with an older version of Go that does not understand Go module paths pin to version v2.3.0.

Packages

  • assert - compare values and fail the test when a comparison fails
  • env - test code which uses environment variables
  • fs - create temporary files and compare a filesystem tree to an expected value
  • golden - compare large multi-line strings against values frozen in golden files
  • icmd - execute binaries and test the output
  • poll - test asynchronous code by polling until a desired state is reached
  • skip - skip a test and print the source code of the condition used to skip the test

Related

Contributing

See CONTRIBUTING.md.

Comments
  • unable to compile tests due to

    unable to compile tests due to "expects import"

    I am not sure what happened, but the tests worked when I pushed them to our repository, now when I try to run go test I get these errors (changed directory names):

    $ go test
    # ~/git/repo
    package ~/git/repo (test)
    	imports gotest.tools/assert
    	imports gotest.tools/v3/assert/cmp: code in directory ~/go/src/gotest.tools/assert/cmp expects import "gotest.tools/assert/cmp"
    FAIL	~/git/repo [setup failed]
    

    This is how I import gotest.tools:

    import (
    	"gotest.tools/assert"
    	"net/http"
    	"testing"
    )
    

    How does this happen all of a sudden? I tried reinstalling everything, even cleared my GOPATH completely, but the error remains.

    question 
    opened by mankinskin 14
  • Errors from Golden could provide a hint about updating

    Errors from Golden could provide a hint about updating

    Passing -test.update-golden to update the golden files works well, but if you're not already familiar with the library, then it's not especially discoverable.

    Could the assertion failure include some help text which tells the user that this flag exists?

    opened by glenjamin 6
  • TestDeepEqualFailure fails w/ go 1.12.2

    TestDeepEqualFailure fails w/ go 1.12.2

    gotest.tools 2.3.0 w/ Go 1.12.2 on Fedora Rawhide, x86_64:

    Testing    in: /builddir/build/BUILD/gotest.tools-2.3.0/_build/src
             PATH: /builddir/build/BUILD/gotest.tools-2.3.0/_build/bin:/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
           GOPATH: /builddir/build/BUILD/gotest.tools-2.3.0/_build:/usr/share/gocode
      GO111MODULE: off
          command: go test -buildmode pie -compiler gc -ldflags "-X gotest.tools/version=2.3.0 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '"
          testing: gotest.tools
    gotest.tools/assert
    --- FAIL: TestDeepEqualFailure (0.00s)
        assert_test.go:305: should have failure message "assertion failed: \n--- actual\n+++ expected\n{assert.stub}.b:\n\t-: 1\n\t+: 2\n", got "assertion failed: \n--- actual\n+++ expected\n\u00a0\u00a0assert.stub{\n\u00a0\u00a0\ta: \"ok\",\n-\u00a0\tb: 1,\n+\u00a0\tb: 2,\n\u00a0\u00a0}\n"
    FAIL
    exit status 1
    FAIL	gotest.tools/assert	0.028s
    
    opened by eclipseo 6
  • Add MatchExtraFilesGlob PathOp

    Add MatchExtraFilesGlob PathOp

    This PR adds MatchExtraFilesGlob PathOp. As discuss in #133, this is a starting point.

    @dnephin regarding the function

     func MatchExtraFilesGlob(glob string, ops ...PathOp) PathOp 
    

    What should I do with the ops parameter? should I apply it to the directoryPath in the AddGlob function (as it is done for other directoryPath functions)?

    Signed-off-by: glefloch [email protected]

    opened by glefloch 6
  • Upgrade to go-cmp v0.4.0

    Upgrade to go-cmp v0.4.0

    and adjust assert/cmp/compare_test.go accordingly.

    In go-cmp v0.4.0, for user convenience, full name of the type is printed in the panic message when accessing an unexported field, see google/go-cmp#171

    Many thanks!

    Anthony

    opened by anthonyfok 5
  • icmd: replace all usages of os/exec with golang.org/x/sys/execabs

    icmd: replace all usages of os/exec with golang.org/x/sys/execabs

    this is a follow-up to https://github.com/gotestyourself/gotest.tools/pull/217, opening as draft because the first 3 commits are from that pull request; I thought it was good to open this change as a separate PR for visibility.

    go.mod: golang.org/x/tools v0.1.0

    Project now started to do releases; this one was related to a security issue on Windows, replacing all usages of os/exec with golang.org/x/sys/execabs.

    icmd: replace all usages of os/exec with golang.org/x/sys/execabs

    Following the changes in Go, and golang.org/x/tools themselves, this change ensures that packages using exec.LookPath or exec.Command to find or run binaries do not accidentally run programs from the current directory when they mean to run programs from the system PATH instead.

    opened by thaJeztah 4
  • go.mod: update dependencies

    go.mod: update dependencies

    go.mod: github.com/pkg/errors v0.9.1

    This version provides compatibility with Go 1.3+ native error wrapping

    go.mod: github.com/google/go-cmp v0.5.5

    Version v0.5.x provides various improvements to the reporter output; https://github.com/google/go-cmp/releases

    full diff: https://github.com/google/go-cmp/compare/v0.4.0...v0.5.5

    opened by thaJeztah 4
  • Typed nil errors should not pass

    Typed nil errors should not pass "NilError"

    I don't think the fix for #167 (#171) was entirely correct. A typed nil error is not a nil error when compared against nil in a normal context.

    package baderr
    
    import (
    	"testing"
    
    	"gotest.tools/v3/assert"
    )
    
    type myEvilError struct{}
    
    func (m *myEvilError) Error() string {
    	return "this is an error"
    }
    
    func doSomething() error {
    	var err *myEvilError
    	return err // NOT nil, this has a type.
    }
    
    // Should not pass.
    func TestErrorNil(t *testing.T) {
    	assert.NilError(t, doSomething()) // This says it's nil.
    }
    
    func TestErrorNotNil(t *testing.T) {
    	assert.Assert(t, doSomething() != nil) // An actual comparison says it's not nil.
    }
    
    opened by zikaeroh 4
  • Update import alias for major version

    Update import alias for major version

    In order to make vendoring work with go modules we need the alias to be as the major version, otherwise, the vendor files are in gotest.tools/v3/assert but the import says gotest.tools/assert which makes go fail to build without go modules.

    Takes over #162 Closes #162

    Signed-off-by: Vincent Demeester [email protected]

    opened by vdemeester 4
  • Issue when checking for nil custom error

    Issue when checking for nil custom error

    Create a custom error by implementing the error interface (e.g. Error() function)

    type MyCustomError struct {
        msg string
    }
    func (ce *MyCustomError) Error() string {
        return ce.string
    }
    

    In a test, checking for a nil MyCustomError reference using the NilError function will lead to unexpected behavior, such as:

    var myErr *MyCustomError 
    assert.NilError(t, myErr)
    

    in this case, you receive a memory address error, because the above custom Error() function will get invoked, even though ce is nil.

    If you "fix" your Error() function, such as:

    func (ce *MyCustomError) Error() string {
        if ce != nil {
            return ce.string
        }
        return ""
    }
    

    then you get rid of the null pointer memory error, but the assertion will fail.

    This happens because, in the assert.go file, at the type switch switch check := comparison.(type), it resolves to case error branch.

    Workaround: use Assert(t, myErr == nil) instead.

    bug 
    opened by eduardboamba 4
  • Go 1.11: example_test.go:22: Wrapf format %s reads arg #1, but call has 0 args

    Go 1.11: example_test.go:22: Wrapf format %s reads arg #1, but call has 0 args

    The poll_test unit test is calling a broken example file

    + LDFLAGS='-X gotest.tools/version=2.1.0'
    + GO_TEST_FLAGS='-buildmode pie -compiler gc'
    + GO_TEST_EXT_LD_FLAGS='-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '
    + go-rpm-integration check -i gotest.tools -b /builddir/build/BUILD/gotest.tools-2.1.0/_build/bin -s /builddir/build/BUILD/gotest.tools-2.1.0/_build -V 2.1.0-1.0.fc30.llt -p /builddir/build/BUILDROOT/golang-gotest-tools-2.1.0-1.0.fc30.llt.x86_64 -g /usr/share/gocode -r '.*example.*'
    Testing in: /builddir/build/BUILD/gotest.tools-2.1.0/_build/src
          PATH: /builddir/build/BUILD/gotest.tools-2.1.0/_build/bin:/builddir/.local/bin:/builddir/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
        GOPATH: /builddir/build/BUILD/gotest.tools-2.1.0/_build:/usr/share/gocode
       command: go test -buildmode pie -compiler gc -ldflags "-X gotest.tools/version=2.1.0 -extldflags '-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '"
       testing: gotest.tools
    gotest.tools/assert
    PASS
    ok  	gotest.tools/assert	0.028s
    gotest.tools/assert/cmd/gty-migrate-from-testify
    PASS
    ok  	gotest.tools/assert/cmd/gty-migrate-from-testify	5.330s
    gotest.tools/assert/cmp
    PASS
    ok  	gotest.tools/assert/cmp	0.009s
    gotest.tools/assert/opt
    PASS
    ok  	gotest.tools/assert/opt	0.008s
    gotest.tools/env
    PASS
    ok  	gotest.tools/env	0.008s
    gotest.tools/fs
    PASS
    ok  	gotest.tools/fs	0.014s
    gotest.tools/golden
    PASS
    ok  	gotest.tools/golden	0.009s
    gotest.tools/icmd
    PASS
    ok  	gotest.tools/icmd	0.369s
    gotest.tools/poll
    # gotest.tools/poll_test
    ./example_test.go:22: Wrapf format %s reads arg #1, but call has 0 args
    FAIL	gotest.tools/poll [build failed]
    
    opened by nim-nim 4
  • Generics in comparison functions?

    Generics in comparison functions?

    Hello!

    I was wondering if you'd considered whether it's worth it to use generics in the comparison functions, and what a migration path to that model might look like?

    enhancement 
    opened by glenjamin 3
  • Feature request: Add a flag for updating golden files interactively

    Feature request: Add a flag for updating golden files interactively

    It would be nice if there was a way to run the tests, and for each one that fails, prompt the user to decide whether to update the diff or not. Think a UX similar to git add -p

    enhancement 
    opened by andremarianiello 1
  • Allow comparing zero for DurationWithThreshold

    Allow comparing zero for DurationWithThreshold

    I found DurationWithThreshold returns false if both of the values are zero.

    https://github.com/gotestyourself/gotest.tools/pull/62/files#diff-d952cf5be65334da619358823c8ff4669961b0d9a30e85d542caad125beac4c6R20

    I guess if both of them are zero, we should return true to compare all test cases in the consistent way.

    What do you think?

    opened by ha1f 3
  • skip: deprecate skip.If and replace it with assert.SkipIf

    skip: deprecate skip.If and replace it with assert.SkipIf

    Originally we thought there might be more stuff to include in the skip package. It has been a few years, and we haven't added anything.

    This PR deprecates the skip package, and moves the one function to assert.SkipIf. Instead of a very small page it should be fine to move this one exported function to the assert package.

    Also improves the godoc a bit, and documents msgAndArgs on assert.Assert as well.

    opened by dnephin 2
  • Helper for limiting a test's duration?

    Helper for limiting a test's duration?

    I've run across a few cases recently where I've wanted to set a deadline on a test running. Generally when doing potentially complicated things that could deadlock, and as mistake could accidentally leave the test hanging.

    Would you be interesting in accepting a new function/package for this functionality?

    There's some related disucssion in https://github.com/golang/go/issues/10529 where it was deemed not suitable for core.

    enhancement 
    opened by glenjamin 4
  • Allow specifying default deep-equality comparison options?

    Allow specifying default deep-equality comparison options?

    I originally tried pitching this to go-cmp, and they said no because they quite resonably want all comparison to be stateless.

    https://github.com/google/go-cmp/issues/241#issue-736205887

    The context from that issue is:

    After upgrading to the most recent version of protobuf, we are finding all of our comparisons are failing due to some new unexported fields.

    There is a cmp.Option implementation provided by the protobuf module in godoc.org/google.golang.org/protobuf/testing/protocmp#Transform - but I now need to go and add this option to every single compare.

    This particular check is generic and safe to add on every compare - would it be reasonable to provide a way to install such global comparison options?

    This feels like it might be slightly safer for a test assertion tool, as each "global" would be scoped to the test binary.

    opened by glenjamin 2
Releases(v3.4.0)
  • v3.4.0(Oct 7, 2022)

    What's Changed

    • Test against Go1.19, and remove a dependency by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/242
    • assert: allow updating expected vars/consts inside functions by @motemen in https://github.com/gotestyourself/gotest.tools/pull/244
    • drop go1.16 from test-matrix, replace deprecated io/ioutil, and update golangci-lint to v1.49.0 by @thaJeztah in https://github.com/gotestyourself/gotest.tools/pull/245

    New Contributors

    • @motemen made their first contribution in https://github.com/gotestyourself/gotest.tools/pull/244

    Full Changelog: https://github.com/gotestyourself/gotest.tools/compare/v3.3.0...v3.4.0

    Source code(tar.gz)
    Source code(zip)
  • v3.3.0(Jun 18, 2022)

    What's Changed

    • golden: accept -update for updating files by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/235
    • Update README links by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/236
    • Update docs about the -update flag by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/238
    • assert: golden variables by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/237

    Full Changelog: https://github.com/gotestyourself/gotest.tools/compare/v3.2.0...v3.3.0

    Source code(tar.gz)
    Source code(zip)
  • v3.2.0(Apr 23, 2022)

    What's Changed

    • Update gty-migrate-from-testify by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/228
    • ci: add go1.18 by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/227
    • assert: improve godoc and failure messages by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/229
    • remove pkg/errors dependency by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/230
    • Ci update linters by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/231
    • assert: document that filename and line number are included by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/233
    • Small godoc improvement by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/234

    Full Changelog: https://github.com/gotestyourself/gotest.tools/compare/v3.1.0...v3.2.0

    Source code(tar.gz)
    Source code(zip)
  • v3.1.0(Jan 15, 2022)

    What's Changed

    • Add github action for merging main branch by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/211
    • ci: add go1.16 by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/213
    • golden: only create dir if update flag is set by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/215
    • CircleCI: update to GA version of Go 1.16 by @thaJeztah in https://github.com/gotestyourself/gotest.tools/pull/216
    • go.mod: update dependencies by @thaJeztah in https://github.com/gotestyourself/gotest.tools/pull/217
    • ci: set path for go install by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/219
    • icmd: replace all usages of os/exec with golang.org/x/sys/execabs by @thaJeztah in https://github.com/gotestyourself/gotest.tools/pull/218
    • ci: add go1.17, remove go1.13 by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/221
    • assert: ErrorIs by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/208
    • fs: add DirFromPath by @dnephin in https://github.com/gotestyourself/gotest.tools/pull/222
    • Stop creating directory outside of testdata by @seriousben in https://github.com/gotestyourself/gotest.tools/pull/223
    • fs: Fix #141 comparing symlink permissions by @DominicLavery in https://github.com/gotestyourself/gotest.tools/pull/224

    New Contributors

    • @DominicLavery made their first contribution in https://github.com/gotestyourself/gotest.tools/pull/224

    Full Changelog: https://github.com/gotestyourself/gotest.tools/compare/v3.0.3...v3.1.0

    Source code(tar.gz)
    Source code(zip)
  • v3.0.3(Oct 8, 2020)

    • assert: fixes a bug that would cause a panic if there were any function calls before assert.Check on the same line
    • golden: create the directory if it does not exist, when run with -test.update-golden
    Source code(tar.gz)
    Source code(zip)
  • v3.0.2(Feb 9, 2020)

  • v3.0.1(Feb 5, 2020)

  • v3.0.0(Feb 9, 2020)

  • v2.3.0(Dec 23, 2018)

    This version includes the following changes:

    • a go.mod and go.sum were added to support consuming the package using go modules #136
    • [fs] the Apply function was added #130
    • [poll] add common check function implementations #132
    • [fs] add more PathOp functions #133 #135
    • [skip] support a message from the check function #137

    Thanks @glefloch, and @icholy for your contributions to this release!

    Source code(tar.gz)
    Source code(zip)
  • v2.2.0(Nov 11, 2018)

    This version includes the following changes:

    • [fs] FromDir now handles symlinks properly
    • [fs] Support carriage return insensitive comparisons for file content
    • [assert] Improve failures message of Error
    • [assert] Fix a panic when an assertion was used in a defer
    • [env] Fix a panic in ToMap when used with an empty variable
    • [fs] Fix the filename used by NewFile and NewDir when the name included a path separator
    • Additional test coverage
    • [icmd] Add some common CmpOp
    • [assert] add Regexp assertion

    Thanks @vdemeester @kolyshkin @silvin-lubecki for the contributions!

    Source code(tar.gz)
    Source code(zip)
  • v1.4.0(Apr 30, 2018)

    • fix some windows bug, CI now includes windows
    • [assert] add Error convenience functions
    • [assert] add a gty-migrate-from-testify tool for automated migration from testify
    • [x/subtest] add an experimental subtest package
    • [fs] add assertions/comparison of a manifest to a filesystem
    Source code(tar.gz)
    Source code(zip)
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Re 290 Nov 26, 2022
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Design it, Run it 1 Nov 24, 2022
A presentable test report generator for go packages

go-tprof Overview Prerequisites 1. Go version >= 1.12 2. Node.js version >= 8.0.0 (for building the UI) 3. Yarn 4. GOPATH and local bin setup (`expor

Gokul T P 34 Feb 7, 2022
A toolkit with common assertions and mocks that plays nicely with the standard library

Testify - Thou Shalt Write Tests ℹ️ We are working on testify v2 and would love to hear what you'd like to see in it, have your say here: https://cutt

Stretchr, Inc. 18.3k Nov 28, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Maxime Soulé 329 Nov 26, 2022
A mock of Go's net package for unit/integration testing

netmock: Simulate Go network connections netmock is a Go package for simulating net connections, including delays and disconnects. This is work in pro

Lucas Wolf 1 Oct 27, 2021
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Fortio (Φορτίο) 2.8k Nov 28, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Esko Luontola 114 Sep 27, 2022
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

null 868 Nov 23, 2022
End-to-end HTTP and REST API testing for Go.

httpexpect Concise, declarative, and easy to use end-to-end HTTP and REST API testing for Go (golang). Basically, httpexpect is a set of chainable bui

Victor Gaydov 2k Nov 24, 2022
HTTP traffic mocking and testing made easy in Go ༼ʘ̚ل͜ʘ̚༽

gock Versatile HTTP mocking made easy in Go that works with any net/http based stdlib implementation. Heavily inspired by nock. There is also its Pyth

Tom 1.7k Nov 23, 2022
HTTP mock for Golang: record and replay HTTP/HTTPS interactions for offline testing

govcr A Word Of Warning I'm in the process of partly rewriting govcr to offer better support for cassette mutations. This is necessary because when I

Seb C 137 Nov 18, 2022
A next-generation testing tool. Orion provides a powerful DSL to write and automate your acceptance tests

Orion is born to change the way we implement our acceptance tests. It takes advantage of HCL from Hashicorp t o provide a simple DSL to write the acceptance tests.

Wesovi Labs 45 Aug 31, 2022
Quick and Easy server testing/validation

Goss - Quick and Easy server validation Goss in 45 seconds Note: For an even faster way of doing this, see: autoadd Note: For testing docker container

Simon Bäumer 26 Oct 7, 2022
Stress testing and benchmarking tool for the NEAR EVM

evm-bully --- stress testing and benchmarking tool for the NEAR EVM

Project Aurora 28 May 30, 2022
An always-on framework that performs end-to-end functional network testing for reachability, latency, and packet loss

Arachne Arachne is a packet loss detection system and an underperforming path detection system. It provides fast and easy active end-to-end functional

Uber Open Source 370 Nov 13, 2022
HTTP load testing tool and library. It's over 9000!

Vegeta Vegeta is a versatile HTTP load testing tool built out of a need to drill HTTP services with a constant request rate. It can be used both as a

Tomás Senart 20.5k Nov 23, 2022
A WebDriver client and acceptance testing library for Go

Agouti Agouti is a library for writing browser-based acceptance tests in Google Go. It provides Gomega matchers and plays nicely with Ginkgo or Spec.

Stephen Levine 804 Nov 16, 2022
Full-featured test framework for Go! Assertions, mocking, input testing, output capturing, and much more! 🍕

testza ?? Testza is like pizza for Go - you could life without it, but why should you? Get The Module | Documentation | Contributing | Code of Conduct

Marvin Wendt 408 Nov 15, 2022