Fuzz testing for go.

Overview

gofuzz

gofuzz is a library for populating go objects with random values.

GoDoc Travis

This is useful for testing:

  • Do your project's objects really serialize/unserialize correctly in all cases?
  • Is there an incorrectly formatted object that will cause your project to panic?

Import with import "github.com/google/gofuzz"

You can use it on single variables:

f := fuzz.New()
var myInt int
f.Fuzz(&myInt) // myInt gets a random value.

You can use it on maps:

f := fuzz.New().NilChance(0).NumElements(1, 1)
var myMap map[ComplexKeyType]string
f.Fuzz(&myMap) // myMap will have exactly one element.

Customize the chance of getting a nil pointer:

f := fuzz.New().NilChance(.5)
var fancyStruct struct {
  A, B, C, D *string
}
f.Fuzz(&fancyStruct) // About half the pointers should be set.

You can even customize the randomization completely if needed:

type MyEnum string
const (
        A MyEnum = "A"
        B MyEnum = "B"
)
type MyInfo struct {
        Type MyEnum
        AInfo *string
        BInfo *string
}

f := fuzz.New().NilChance(0).Funcs(
        func(e *MyInfo, c fuzz.Continue) {
                switch c.Intn(2) {
                case 0:
                        e.Type = A
                        c.Fuzz(&e.AInfo)
                case 1:
                        e.Type = B
                        c.Fuzz(&e.BInfo)
                }
        },
)

var myObject MyInfo
f.Fuzz(&myObject) // Type will correspond to whether A or B info is set.

See more examples in example_test.go.

You can use this library for easier go-fuzzing. go-fuzz provides the user a byte-slice, which should be converted to different inputs for the tested function. This library can help convert the byte slice. Consider for example a fuzz test for a the function mypackage.MyFunc that takes an int arguments:

// +build gofuzz
package mypackage

import fuzz "github.com/google/gofuzz"

func Fuzz(data []byte) int {
        var i int
        fuzz.NewFromGoFuzz(data).Fuzz(&i)
        MyFunc(i)
        return 0
}

Happy testing!

Issues
  • Add support to fuzz from go-fuzz input

    Add support to fuzz from go-fuzz input

    Add a helper function that enables using gofuzz (this project) with go-fuzz for continuose fuzzing. Essentially, it enables translating the fuzzing bytes from go-fuzz to any Go object using this library.

    This change will enable using this project with fuzzing websites such as fuzzit.dev or fuzzbuzz.io.

    The underlying implementation was an idea of @lavalamp, by which a random source is created that generates random numbers from the input bytes slice. In this way changes in the source result in logical changes in the random data, which enables the fuzzer to efficiently search the space of inputs.

    Fixes #33

    cla: yes 
    opened by posener 15
  • Add a FuzzNoCustom() function

    Add a FuzzNoCustom() function

    This skips custom fuzz functions. The use case is for custom-fuzzing a subset of fields of a larger struct. The custom function can call this on the struct (to avoid a recursive call) and then patch-up fields it needs to handle specially.

    Better solution to #10.

    Fixes #10

    opened by thockin 9
  • Optimize randBool() and randString()

    Optimize randBool() and randString()

    I love this library, small but gets the job done! :)

    I have tinkered a bit with the functions to generate bools and strings and made them a bit faster. Hopefully it will save someone some clocks when running tests.

    The code speeds up bool generation 3-fold. I've ran some benchmarks on GCP VM's and my local MacBook:

    My MacBook Pro:

    BenchmarkRandBool-8 75312272 14.5 ns/op BenchmarkRandBoolNew-8 235223112 5.06 ns/op BenchmarkRandStringStock-8 1439635 841 ns/op BenchmarkRandStringBuilder-8 1541736 775 ns/op

    8 core GCP E2:

    BenchmarkRandBool-8 70003590 16.6 ns/op BenchmarkRandBoolNew-8 236062108 5.08 ns/op BenchmarkRandStringStock-8 1370116 874 ns/op BenchmarkRandStringBuilder-8 1484086 807 ns/op

    8 core GCP N2 (Intel Cascade Lake):

    BenchmarkRandBool-8 91276062 13.1 ns/op BenchmarkRandBoolNew-8 301435986 3.98 ns/op BenchmarkRandStringStock-8 1743852 688 ns/op BenchmarkRandStringBuilder-8 1883102 638 ns/op

    8 core GCP N1 (Intel Skylake):

    BenchmarkRandBool-8 71896552 16.5 ns/op BenchmarkRandBoolNew-8 239149131 5.03 ns/op BenchmarkRandStringStock-8 1375860 873 ns/op BenchmarkRandStringBuilder-8 1484378 808 ns/op

    randString() will be 5-7% due to less data being moved around (thanks to strings.Builder)

    randString() memory usage change:

    BenchmarkRandStringStock-8 1543179 772 ns/op 71 B/op 1 allocs/op BenchmarkRandStringBuilder-8 1639436 735 ns/op 48 B/op 1 allocs/op

    cla: yes 
    opened by jake-ciolek 8
  • Allow skipping certain struct fields while fuzzing (based on a specified prefix)

    Allow skipping certain struct fields while fuzzing (based on a specified prefix)

    Protobuf generated code have certain exported fields (XXX_...) which cannot be set as that corrupts the serialization. This change allows specifying certain prefix for fields which are ignored while fuzzing a struct.

    cla: yes 
    opened by samirkut 8
  • Set the character generation mode to generate character that conform to Unicode encoding

    Set the character generation mode to generate character that conform to Unicode encoding

    You can use f.WithStringGenMode(mode) to set generation mode to generate different character, like ASCII, MultiByte, CommonCJK, Chinese, Japanese, Number, Lower-Case, Upper-Case , default is random mode(each rune will be one of above character).

    cla: yes 
    opened by kwongtailau 7
  • Support fuzzing unexported fields

    Support fuzzing unexported fields

    The current gofuzz only support fuzzing the exported fields (i.e. the fields that start with an uppercase letter), other fields are simply filtered out according to this line.

    However, a lot of struct also contains un-exported fields (i.e. fields start with lowercase letters) and sometimes people want to do fuzzing on those structs. In this pr I fixed this problem by adding allowUnexportedFields to the fuzzer, which allows user to decide whether they want to include unexported fields while fuzzing. It is set to false by default. But it can be turned on with AllowUnexportedFields function.

    Tests are also added.

    cla: yes 
    opened by yanzhoupan 5
  • Fix charRange to allow picking the last item

    Fix charRange to allow picking the last item

    Before this change, choose would never pick the last character of the set. This is not the expected behavior: if the range is a-z, we expect a,b,...,y,z to be picked in a uniform way.

    cla: yes 
    opened by punkeel 5
  • Allow Continue.Fuzz to accept a reflect.Value

    Allow Continue.Fuzz to accept a reflect.Value

    This is a small optimization, so I completely understand if you would rather keep the existing interface and close this PR without merging. I made the change before I realized I could use reflect.Value.Addr() as the argument. I thought I would open this PR anyway, in case there is interest in the small optimization. I imagine in some larger test suites it might save a little time by avoiding the need to reflect again.

    Continue.Fuzz (and FuzzNoCustom) immediately convert the interface{} into a reflect.Value. In some cases the caller may already have a reflect.Value, so accepting that type makes it easier to write custom fuzz functions.

    One use case for this is ignoring a field on a struct. The SkipFieldsWithPattern option is a good choice when the field name should always be ignored. In other cases a field name may exist on multiple structs, and using a regex pattern would result in the field being skipped in all cases. To ignore a field by name on only a single struct a custom fuzz function can use reflect to iterate over the fields, and call c.Fuzz() on all fields except for the ignored field.

    cla: yes 
    opened by dnephin 4
  • Update import statement to reference correct library name

    Update import statement to reference correct library name

    The example in the documentation references the go-fuzz library rather than gofuzz, which appears to be a mistake. This just tweaks that so that the example should hopefully work when copied.

    cla: yes 
    opened by colinnewell 4
  • Fix testing methodology; passes accumulate now.

    Fix testing methodology; passes accumulate now.

    go test -count=10000 . passes now.

    I spot checked by introducing errors and the new tests continue to catch bugs. Running 10k tests as above caught the one place I messed up the fail/pass logic.

    cla: yes 
    opened by lavalamp 3
  • Reduce the chance a test can flake

    Reduce the chance a test can flake

    This commit adjust the values used in tests to reduce the probability of a flake.

    For tryFuzz, increase the attempts from 20 to 30. For checkFailed, increase the attempts to 20, and only fail if 19/20 iteration received a zero value. Previously it would fail if 9/10 received zero values. Also adds t.Helper() to the two test helpers.

    I was testing this with go test -count=100 and I saw much fewer flakes.

    cla: yes 
    opened by dnephin 3
  • NewFromGoFuzz doesn't work well with go-fuzz

    NewFromGoFuzz doesn't work well with go-fuzz

    I've come across projects using go-fuzz with NewFromGoFuzz where fuzzing didn't catch obvious bugs. I've distilled these down into very simple, unstructured cases (which you'd probably not use gofuzz for) which demonstrate the problems. In the projects in questions, I've used JSON and protobuf unmarshalling of go-fuzz provided data into structs with better results. Not sure if there's anything obvious to do with this information except to document the downsides?

    It's probably easiest to describe the problem with examples:

    // +build gofuzz
    package mypackage
    
    import fuzz "github.com/google/gofuzz"
    
    func MyFunc(i int) {
        if i == 1337 {
            panic("found correct i")
        }
    }
    
    func Fuzz(data []byte) int {
        var i int
        fuzz.NewFromGoFuzz(data).Fuzz(&i)
        MyFunc(i)
        return 0
    }
    

    The above (an extension of the README example) would have to run for a very long time to find a crasher. Go-fuzz has its sonar to guide it to new coverage gated by comparisons e.g. the following finds crashers almost immediately:

    // +build gofuzz
    
    package mypackage
    
    import "encoding/binary"
    
    func MyFunc(i int) {
        if i == 1337 {
            panic("found correct i")
        }
    }
    
    func Fuzz(data []byte) int {
        i := binary.BigEndian.Uint64(data)
        MyFunc(int(i))
        return 0
    }
    

    Decoding the data from go-fuzz as little-endian, hex encoded strings or ascii will also result in the crasher being found quickly as the number "1337" is used in the mutator in all of these forms.

    I found that applying this patch

    diff --git a/bytesource/bytesource.go b/bytesource/bytesource.go
    index 5bb3659..efd6a65 100644
    --- a/bytesource/bytesource.go
    +++ b/bytesource/bytesource.go
    @@ -40,7 +40,9 @@ func New(input []byte) *ByteSource {
                    fallback: rand.NewSource(0),
            }
            if len(input) > 0 {
    -               s.fallback = rand.NewSource(int64(s.consumeUint64()))
    +               seed := make([]byte, 8)
    +               copy(seed, input)
    +               s.fallback = rand.NewSource(int64(binary.BigEndian.Uint64(seed)))
            }
            return s
     }
    

    allowed go-fuzz to work a little bit better for this scenario. The improvement comes because the first 8 bytes of the go-fuzz input are not lost to the random number generator seed.

    The following demonstrates how inputs from go-fuzz with string literals are unrecognisable after passing through NewFromGoFuzz.Fuzz:

    package main
    
    import (
        "fmt"
    
        fuzz "github.com/google/gofuzz"
    )
    
    func main() {
        data := []byte("-----BEGIN RSA PRIVATE KEY-----")
        var str string
        fuzz.NewFromGoFuzz(data).Fuzz(&str)
        fmt.Printf("str = %s\n", str)
    }
    

    outputs

    str = Ȱ#ǻŐ蒉
    

    Here, randString will consume at least 8*(n+1) bytes of the go-fuzz input (n is the length chosen for the string) in such a way as to "slice and dice" the input leaving it with no recognisable fragments. This prevents the fuzzer finding crashers for the following:

    // +build gofuzz
    package mypackage
    
    import fuzz "github.com/google/gofuzz"
    
    func MyFunc(str string) {
        if str == "1337" {
            panic("found correct")
        }
    }
    
    func Fuzz(data []byte) int {
        var str string
        fuzz.NewFromGoFuzz(data).Fuzz(&i)
        MyFunc(str)
        return 0
    }
    

    An equivalent fuzzer without gofuzz

    // +build gofuzz
    package mypackage
    
    import fuzz "github.com/google/gofuzz"
    
    func MyFunc(str string) {
        if str == "1337" {
            panic("found correct string")
        }
    }
    
    func Fuzz(data []byte) int {
        MyFunc([]byte(str))
        return 0
    }
    

    immediately finds a crasher. It should also be noted that strings from gofuzz will be limited in length and well-formed utf-8 which may be at odds with the aims for fuzzing.

    opened by stevenjohnstone 1
  • Added Support for Linux on Power

    Added Support for Linux on Power

    Hi, I had added ppc64le(Linux on Power) support on travis-ci in the branch and looks like its been successfully added. I believe it is ready for the final review and merge. The travis ci build logs can be verified from the link below.

    https://travis-ci.com/github/ujjwalsh/gofuzz/builds/191587932 Please have a look.

    Regards, ujjwal

    cla: yes 
    opened by ujjwalsh 0
  • Include Type in panic message

    Include Type in panic message

    The switch is looking at type, not the value. It should be easier to understand these failures when the message includes the type. I ran into this problem while using gofuzz.

    Previous the message was:

    Can't handle <nil>
    

    After this change:

    Can't handle type interface {} with value <nil>
    
    cla: yes 
    opened by dnephin 0
  • Add note to docs that private fields of structs cannot be fuzzed using this package

    Add note to docs that private fields of structs cannot be fuzzed using this package

    I tried to use this package (through go-fuzz) for the first time when I was fuzzing a struct that has only private fields. It took me a while to figure out that this package doesn't put any values into private fields...

    Now, I'm not terribly familiar with reflection in Go but from what I can tell based on the reflection docs and some testing is that it would be impossible for this package to set the value of private fields. Am i correct?

    If the above statement is correct, I think it would be useful if there was a note in the README (or somewhere else in the docs) stating that private fields cannot be populated by this package. Thoughts?

    opened by ericcornelissen 2
  • issue building with go 1.11.3 and previous

    issue building with go 1.11.3 and previous

    9532c33cb8812a3a439858750fb792cb88d35495 is causing versions prior to go 1.11.4 to have issues requiring the module.

    e.g. semaphore v1 has go 1.11.0 as latest version and builds fail with:

    go build github.com/google/gofuzz: module requires Go 1.12
    

    note: travis-ci hasn't tested the last 2 commits on master.

    opened by dnozay 0
Releases(v1.1.0)
Owner
Google
Google ❤️ Open Source
Google
Fuzz Go defer/panic/recover

deferfuzz deferfuzz is a fuzzer for Go defer/panic/recover. Caveat: I wrote this in a couple hours, and I'm more of a compiler engineer than a fuzzer

Matthew Dempsky 26 Mar 3, 2022
siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

siusiu (suite-suite harmonics) a suite used to manage the suite, designed to free penetration testing engineers from learning and using various security tools, reducing the time and effort spent by penetration testing engineers on installing tools, remembering how to use tools.

Re 261 Jul 24, 2022
A yaml data-driven testing format together with golang testing library

Specimen Yaml-based data-driven testing Specimen is a yaml data format for data-driven testing. This enforces separation between feature being tested

Design it, Run it 0 Jan 31, 2022
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Fortio (Φορτίο) 2.6k Aug 9, 2022
:exclamation:Basic Assertion Library used along side native go testing, with building blocks for custom assertions

Package assert Package assert is a Basic Assertion library used along side native go testing Installation Use go get. go get github.com/go-playground/

Go Playgound 41 Aug 4, 2022
Expressive end-to-end HTTP API testing made easy in Go

baloo Expressive and versatile end-to-end HTTP API testing made easy in Go (golang), built on top of gentleman HTTP client toolkit. Take a look to the

Tom 727 Jul 26, 2022
Simple Go snapshot testing

Incredibly simple Go snapshot testing: cupaloy takes a snapshot of your test output and compares it to a snapshot committed alongside your tests. If t

Bradley Kemp 221 Aug 4, 2022
Clean database for testing, inspired by database_cleaner for Ruby

DbCleaner Clean database for testing, inspired by database_cleaner for Ruby. It uses flock syscall under the hood to make sure the test can runs in pa

Scott Le 139 Jul 16, 2022
Golang HTTP client testing framework

flute Golang HTTP client testing framework Presentation https://speakerdeck.com/szksh/flute-golang-http-client-testing-framework Overview flute is the

Shunsuke Suzuki 17 Apr 19, 2022
API testing framework inspired by frisby-js

frisby REST API testing framework inspired by frisby-js, written in Go Proposals I'm starting to work on frisby again with the following ideas: Read s

_Hofstadter 272 Jun 13, 2022
Mutation testing for Go source code

go-mutesting go-mutesting is a framework for performing mutation testing on Go source code. Its main purpose is to find source code, which is not cove

Markus Zimmermann 539 Jul 22, 2022
Extremely flexible golang deep comparison, extends the go testing package and tests HTTP APIs

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Maxime Soulé 312 Aug 4, 2022
Minimal and Beautiful Go testing framework

Goblin A Mocha like BDD testing framework written in Go that requires no additional dependencies. Requires no extensive documentation nor complicated

null 860 Aug 3, 2022
A Comprehensive Coverage Testing System for The Go Programming Language

goc 中文页 | goc is a comprehensive coverage testing system for The Go Programming Language, especially for some complex scenarios, like system testing c

Qiniu Cloud 547 Jul 28, 2022
Go testing in the browser. Integrates with `go test`. Write behavioral tests in Go.

GoConvey is awesome Go testing Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser accordi

SmartyStreets 7.3k Aug 1, 2022
Testing API Handler written in Golang.

Gofight API Handler Testing for Golang Web framework. Support Framework Http Handler Golang package http provides HTTP client and server implementatio

Bo-Yi Wu 423 Jul 27, 2022
Library created for testing JSON against patterns.

Gomatch Library created for testing JSON against patterns. The goal was to be able to validate JSON focusing only on parts essential in given test cas

null 42 Jul 24, 2022
Testing framework for Go. Allows writing self-documenting tests/specifications, and executes them concurrently and safely isolated. [UNMAINTAINED]

GoSpec GoSpec is a BDD-style testing framework for the Go programming language. It allows writing self-documenting tests/specs, and executes them in p

Esko Luontola 112 Apr 5, 2022
A collection of packages to augment the go testing package and support common patterns.

gotest.tools A collection of packages to augment testing and support common patterns. Usage With Go modules enabled (go1.11+) $ go get gotest.tools/v3

null 333 Jul 30, 2022