Goroutine Leak Detector

Overview

Leaktest Build Status codecov Sourcegraph Documentation

Refactored, tested variant of the goroutine leak detector found in both net/http tests and the cockroachdb source tree.

Takes a snapshot of running goroutines at the start of a test, and at the end - compares the two and voila. Ignores runtime/sys goroutines. Doesn't play nice with t.Parallel() right now, but there are plans to do so.

Installation

Go 1.7+

go get -u github.com/fortytw2/leaktest

Go 1.5/1.6 need to use the tag v1.0.0, as newer versions depend on context.Context.

Example

These tests fail, because they leak a goroutine

// Default "Check" will poll for 5 seconds to check that all
// goroutines are cleaned up
func TestPool(t *testing.T) {
    defer leaktest.Check(t)()

    go func() {
        for {
            time.Sleep(time.Second)
        }
    }()
}

// Helper function to timeout after X duration
func TestPoolTimeout(t *testing.T) {
    defer leaktest.CheckTimeout(t, time.Second)()

    go func() {
        for {
            time.Sleep(time.Second)
        }
    }()
}

// Use Go 1.7+ context.Context for cancellation
func TestPoolContext(t *testing.T) {
    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
    defer cancel()
    defer leaktest.CheckContext(ctx, t)()

    go func() {
        for {
            time.Sleep(time.Second)
        }
    }()
}

LICENSE

Same BSD-style as Go, see LICENSE

Issues
  • Does not work with go 1.7

    Does not work with go 1.7

    With Go 1.7, there is always a leaked Gorouting reported, even for empty tests:

    func TestStoppedSlotReporter(t *testing.T) { defer leaktest.Check(t)() }

    opened by ghaering 10
  • Timeouts happen immediately

    Timeouts happen immediately

    It appears that since the context.WithTimeout is called at the beginning of the test, tests which take longer than timeout to complete automatically fail. This is because the Done channel on the context is already closed by time the test completes. This is problematic for tests which take an non-deterministic amount of time to complete.

    I would have expected the timeout value to be the amount of time leaktest waits when the deferred function is executed, not from the moment Check is called.

    Currently we are getting around the issue by wrapping leaktest functionality:

    func CheckTimeout(t *testing.T, dur time.Duration) func() {
        ctx, cancel := context.WithCancel(context.Background())
        fn := leaktest.CheckContext(ctx, t)
        return func() {
            time.AfterFunc(dur, cancel)
    	fn()
        }
    }
    
    opened by seglberg 3
  • Tests hang in simple for loop

    Tests hang in simple for loop

    Hello! I've create a minial step to create a condition of a leaked goroutine. In this case, the test never finishes.

    I think I've narrowed the bug down to this line where it seems to block.

    My guess is, because this goroutine never blocks and runtime.Stack() is waiting for the goroutine to block?

    package hub
    
    import (
    	"testing"
    	"time"
    
    	"github.com/fortytw2/leaktest"
    )
    
    func TestInfiniteLoop(t *testing.T) {
    	defer leaktest.CheckTimeout(t, time.Second)()
    	go func() {
    		i := 0
    		for {
    			i++
    		}
    	}()
    }
    

    System details

    go version go1.9 linux/amd64
    GOARCH="amd64"
    GOBIN=""
    GOEXE=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GOOS="linux"
    GOPATH="/home/anthony/go"
    GORACE=""
    GOROOT="/usr/local/go"
    GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
    GCCGO="gccgo"
    CC="gcc"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build343347876=/tmp/go-build -gno-record-gcc-switches"
    CXX="g++"
    CGO_ENABLED="1"
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOROOT/bin/go version: go version go1.9 linux/amd64
    GOROOT/bin/go tool compile -V: compile version go1.9
    uname -sr: Linux 4.10.0-40-generic
    Distributor ID:	Ubuntu
    Description:	Ubuntu 16.04.3 LTS
    Release:	16.04
    Codename:	xenial
    /lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Ubuntu GLIBC 2.23-0ubuntu9) stable release version 2.23, by Roland McGrath et al.
    gdb --version: GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
    
    opened by while-loop 3
  • Tests fail with

    Tests fail with "unexpected fault address 0x117160" for aarch64

    Architecture: aarch64 Golang version: 1.8.3 OS: Fedora 26

    Same tests fail for other architectures randomly. Could not reproduce for those other arches though.

    go test -compiler gc -ldflags '' github.com/fortytw2/leaktest unexpected fault address 0x117160 fatal error: fault [signal SIGSEGV: segmentation violation code=0x2 addr=0x117160 pc=0x117160] goroutine 18 [running]: runtime.throw(0x147168, 0x5) /usr/lib/golang/src/runtime/panic.go:596 +0x70 fp=0x4420040eb0 sp=0x4420040e90 runtime.sigpanic() /usr/lib/golang/src/runtime/signal_unix.go:297 +0x224 fp=0x4420040f00 sp=0x4420040eb0 github.com/fortytw2/leaktest.CheckTimeout.func1() /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest.go:75 +0x38 fp=0x4420040f20 sp=0x4420040f10 github.com/fortytw2/leaktest.TestCheck(0x4420078270) /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:73 +0xb0 fp=0x4420040fa0 sp=0x4420040f20 testing.tRunner(0x4420078270, 0x150138) /usr/lib/golang/src/testing/testing.go:657 +0x84 fp=0x4420040fc0 sp=0x4420040fa0 runtime.goexit() /usr/lib/golang/src/runtime/asm_arm64.s:981 +0x4 fp=0x4420040fc0 sp=0x4420040fc0 created by testing.(*T).Run /usr/lib/golang/src/testing/testing.go:697 +0x240 goroutine 1 [chan receive]: testing.(*T).Run(0x44200781a0, 0x147b6e, 0x9, 0x150138, 0x60) /usr/lib/golang/src/testing/testing.go:698 +0x260 testing.runTests.func1(0x44200781a0) /usr/lib/golang/src/testing/testing.go:882 +0x58 testing.tRunner(0x44200781a0, 0x442003fde0) /usr/lib/golang/src/testing/testing.go:657 +0x84 testing.runTests(0x4420086980, 0x1f1480, 0x2, 0x2, 0x14b00a) /usr/lib/golang/src/testing/testing.go:888 +0x260 testing.(*M).Run(0x442003ff28, 0x442003ff28) /usr/lib/golang/src/testing/testing.go:822 +0xe0 main.main() github.com/fortytw2/leaktest/_test/_testmain.go:44 +0xe0 goroutine 20 [sleep]: time.Sleep(0x3b9aca00) /usr/lib/golang/src/runtime/time.go:59 +0xe8 github.com/fortytw2/leaktest.glob..func1() /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:25 +0x20 created by github.com/fortytw2/leaktest.TestCheck /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:71 +0xa4 goroutine 37 [chan receive]: github.com/fortytw2/leaktest.glob..func2() /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:32 +0x48 created by github.com/fortytw2/leaktest.TestCheck /builddir/build/BUILDROOT/golang-github-fortytw2-leaktest-1.1.0-1.fc26.aarch64/usr/share/gocode/src/github.com/fortytw2/leaktest/leaktest_test.go:71 +0xa4 FAIL github.com/fortytw2/leaktest 2.078s

    opened by athos-ribeiro 3
  • Add CheckTimeout function

    Add CheckTimeout function

    Allows for a configurable timeout without changing the previously exposed API :)

    closes https://github.com/fortytw2/leaktest/issues/5

    cc/ @lukechampine @bradleyfalzon

    opened by fortytw2 3
  • Memory Leak Not Detected When Leaking Goroutine

    Memory Leak Not Detected When Leaking Goroutine

    Given that golang is garbage-collected, I'm trying to understand instances when it would make sense to use this package. As such, I tried to simulate a memory leak.

    
    func TestGoRoutineLeak(t *testing.T) {
    
    	leaktest.Check(t)()
    	c := make (chan int,0)
    
    	go func () {
    		for _ = range c{
    
    		}
    
    	}()
    }
    

    This doesn't throw any errors? I was expecting it to throw an error since we're leaking a goroutine in plain sight. I'd love a pointer or two in the right direction here.

    Thanks :)

    opened by NavyaZaveri 1
  • Tracks interesting goroutines by ID instead of stacktrace

    Tracks interesting goroutines by ID instead of stacktrace

    Previously the leak detection compared the goroutine stacks from the starting of a function with their stacks at the end to find new goroutines. But, when there are preexisting goroutines whose stacks change during the function, they are incorrectly detected as a leaked goroutine. This update compare goroutine IDs instead.

    opened by bsdavidson 1
  • Prevent broken usage

    Prevent broken usage

    defer leaktest.Check(t) is surprisingly incorrect and does nothing. Lets investigate ways to detect when this happens and inform the caller, otherwise they may think that their code is leak free and dandy when it really isn't

    opened by fortytw2 1
  • Test timeouts

    Test timeouts

    leaktest does not play well with go test -timeout=5s. Regardless, waiting 5 seconds for goroutines to shut down seems excessive. My expectation was that goroutines would be checked as soon as the deferred function was called. I suggest adding an optional timeout argument to Check, or providing some other way of delaying the check. Note that if Check did not delay by default, you could implement this behavior yourself:

    func TestFoo(t testing.T) {
        checkFn := leaktest.Check(t)
        defer func() {
            time.Sleep(5 * time.Second)
            checkFn()
        }()
    }
    

    Furthermore, this would give you more fine-grained control over when the check executes. For example, you could wait on a channel before checking. The only reason I propose adding an optional timeout argument is because it seems like a common case.

    opened by lukechampine 1
  • README example is missing a ()

    README example is missing a ()

    Example in README should be

    func TestPool(t *testing.T) {
        defer leaktest.Check(t)()
    
        go func() {
            for {
                time.Sleep(time.Second)
            }
        }()
    }
    

    Notice the () after the } at the end of the go function -- Just a typo I am certain.

    opened by pschlump 1
  • Ignore HTTP Keep-alives

    Ignore HTTP Keep-alives

    As per the change in Cockroach leaktest I'd like to ignore HTTP keep-alives in this utility.

    leaktest: ignore HTTP keep alives Tests that performed an HTTP connection using the default client had to have a special instruction to poke http.DefaultClient to disable keep alives to prevent the leaktester from failing. An upcoming change performs an HTTP connection with its own unconfigurable client, which would have required a bunch of special code just to make the leak tester pass. Instead, ignore stacks that are included during a connection in keep alive. This matches the code in net/http/main_test.go#afterTest.

    https://github.com/cockroachdb/cockroach/commit/e5e6dde6a58d551631640261f9e08c74c87ae917#diff-fdc7f75e12e2453349457c6742ff532f

    opened by lawrencegripper 0
  • refactor: extract 'LeakCheckConfiguration'

    refactor: extract 'LeakCheckConfiguration'

    👋 Hello—thank you for making this package, I was just about to start copying stuff out of net/http until I found it!

    One thing which would be convenient for me is the ability to specify a certain goroutine is "not interesting"—e.g. sometimes my code might be using some other package which starts some internal goroutine which by design runs forever. It would be nice if I could basically add it to the ignore list in the context of that particular test.

    This PR attempts to achieve that by factoring out the slice of 'known disinteresting' values to a configuration struct, which can then be customised arbitrarily. The existing surface of the library doesn't change, as the package level functions now delegate to a package global default configuration.

    opened by kjgorman 0
  • feat: add skip options in checks

    feat: add skip options in checks

    This PR introduces options to skip some goroutines based on their stack.

    Use case: I know something I depend on is leaking, or it is ok that something is leaking, and I want only to error on a precise leak.

    opened by affo 6
  • Error parsing stack

    Error parsing stack

    I'm seeing a mysterious issue when I use leaktest in combination with a high -count flag, as in:

    go test -run . -failfast -count 100 --tags=unit ./...
        leaktest.go:83: leaktest: error parsing stack: "goro"
    FAIL
    

    The reported line number in leaktest.go is always the same. But the exact string it's failing to parse changes from run to run. e.g.

    leaktest.go:83: leaktest: error parsing stack: "gorouti"
    leaktest.go:83: leaktest: error parsing stack: "goroutine 99673 [select]:"
    

    It seems to happen more predictably when I run with high -count values.

    Any ideas what could be causing this?

    Here is my go build env:

    GO111MODULE=""
    GOARCH="amd64"
    GOBIN=""
    GOCACHE="/home/marshall/.cache/go-build"
    GOENV="/home/marshall/.config/go/env"
    GOEXE=""
    GOFLAGS=""
    GOHOSTARCH="amd64"
    GOHOSTOS="linux"
    GONOPROXY=""
    GONOSUMDB=""
    GOOS="linux"
    GOPATH="/home/marshall/code/go"
    GOPRIVATE=""
    GOPROXY="https://proxy.golang.org,direct"
    GOROOT="/usr/lib/go"
    GOSUMDB="sum.golang.org"
    GOTMPDIR=""
    GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
    GCCGO="gccgo"
    AR="ar"
    CC="x86_64-pc-linux-gnu-gcc"
    CXX="x86_64-pc-linux-gnu-g++"
    CGO_ENABLED="1"
    GOMOD=""
    CGO_CFLAGS="-g -O2"
    CGO_CPPFLAGS=""
    CGO_CXXFLAGS="-g -O2"
    CGO_FFLAGS="-g -O2"
    CGO_LDFLAGS="-g -O2"
    PKG_CONFIG="pkg-config"
    GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build261227009=/tmp/go-build -gno-record-gcc-switches"
    
    
    opened by marshall-mcmullen 4
  • Plans for supporting t.Parallel

    Plans for supporting t.Parallel

    Hello!

    Thanks for this library, it'll definitely come in handy. Would you be able to document the reasons why t.Parallel isn't supported and what ideas you have for addressing the problem?

    Thanks

    opened by tsenart 7
Releases(v1.3.0)
  • v1.3.0(Nov 9, 2018)

  • v1.2.0(Mar 20, 2018)

  • v1.1.0(May 17, 2017)

  • v1.0.0(May 13, 2017)

    I'm tagging v1.0.0 from current master prior to merging of https://github.com/fortytw2/leaktest/pull/12 so that anyone stuck on Go 1.5 / 1.6 can still use leaktest by pinning to this tag

    Source code(tar.gz)
    Source code(zip)
Owner
Ian
building good things
Ian
🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants 是一个高性能且低损耗的 goroutine 池。

A goroutine pool for Go English | ???? 中文 ?? Introduction Library ants implements a goroutine pool with fixed capacity, managing and recycling a massi

Andy Pan 8.6k Jun 26, 2022
🐝 A Highly Performant and easy to use goroutine pool for Go

gohive Package gohive implements a simple and easy to use goroutine pool for Go Features Pool can be created with a specific size as per the requireme

Lovelesh 36 Jun 21, 2022
golang worker pool , Concurrency limiting goroutine pool

golang worker pool 中文说明 Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks su

xxj 398 Jun 21, 2022
Lightweight Goroutine pool

grpool Lightweight Goroutine pool Clients can submit jobs. Dispatcher takes job, and sends it to first available worker. When worker is done with proc

Ivan Pusic 694 Jun 16, 2022
errgroup with goroutine worker limits

neilotoole/errgroup neilotoole/errgroup is a drop-in alternative to Go's wonderful sync/errgroup but limited to N goroutines. This is useful for inter

Neil O'Toole 126 Jun 21, 2022
Minimalistic and High-performance goroutine worker pool written in Go

pond Minimalistic and High-performance goroutine worker pool written in Go Motivation This library is meant to provide a simple way to limit concurren

Alejandro Durante 557 Jun 22, 2022
A goroutine pool for Go

Tunny is a Golang library for spawning and managing a goroutine pool, allowing you to limit work coming from any number of goroutines with a synchrono

Ashley Jeffs 3.1k Jun 22, 2022
Concurrency limiting goroutine pool

workerpool Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting task

Andrew Gillis 841 Jun 24, 2022
Waiting group for collecting goroutine information.

在go语言waitGroup和errGroup都是用来控制goroutine的并发的方式,前者只能等待所有goroutine执行完成之后再执行Wait()函数后面的代码并且不

Jarvib Ding 111 Dec 3, 2021
A simple and useful goroutine concurrent library.

Taskgroup A simple and useful goroutine concurrent library. Installation go get github.com/anthhub/taskgroup

Tangqy 4 May 19, 2021
Provides some convenient API, includes Goid(), AllGoid(), and LocalStorage, which is a goroutine's local storage, just like ThreadLocal in other languages.

routine 中文版 routine encapsulates and provides some easy-to-use, high-performance goroutine context access interfaces, which can help you access corout

null 58 Jun 27, 2022
Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Bo-Yi Wu 189 Jun 26, 2022
Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

golang-queue 188 Jun 19, 2022
A universal mechanism to manage goroutine lifecycles

A universal mechanism to manage goroutine lifecycles

OK Log 1.2k Jun 21, 2022
goroutine pool in golang

goroutine pool in golang

wksw 1 Nov 1, 2021
A cross goroutine storage tool with very simple implementation and function.

Simple-goroutine-local is a cross goroutine storage tool with very simple implementation and function (the concept is similar to Java ThreadLocal). Ge

null 0 Jan 13, 2022
A lib for monitoring runtime goroutine stack

Overview A lib for monitoring runtime goroutine stack. Such as wait for goroutines to exit, leak detection, etc. Features context.Context first design

Yad Smood 33 Apr 30, 2022
Goroutine Leak Detector

Leaktest Refactored, tested variant of the goroutine leak detector found in both net/http tests and the cockroachdb source tree. Takes a snapshot of r

Ian 953 Jun 19, 2022
:speedboat: a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation

Package pool Package pool implements a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation. Features

Go Playgound 687 Jun 20, 2022
🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants 是一个高性能且低损耗的 goroutine 池。

A goroutine pool for Go English | ???? 中文 ?? Introduction Library ants implements a goroutine pool with fixed capacity, managing and recycling a massi

Andy Pan 8.6k Jun 26, 2022
Friends don't let friends leak secrets on their terminal window

senv - safer env Friends don't let friends leak secrets in terminal windows. ?? Print your environment to the terminal without worry.

null 99 Jun 16, 2022
Tracking down a Memory Leak in Go/SQLite

Tracking down a Memory Leak in Go/SQLite run make test - WARNING: long running - several minutes on my workstation OSs supported: Windows_NT => memory

Stefan Thiel 1 Feb 28, 2022
A toy deadlock detector written in Go.

Toy Deadlock Detector This package aims to provide a DSL to represent processes as finite state machines and their concurrent composition. A detector

Yuto Takahashi 27 Feb 2, 2022
Chronos - A static race detector for the go language

Chronos Chronos is a static race detector for the Go language written in Go. Quick Start: Download the package go get -v github.com/amit-davidson/Chro

Amit Davidson 367 Jun 20, 2022
A faster file programming language detector

Programming language detector and toolbox to ignore binary or vendored files. enry, started as a port to Go of the original Linguist Ruby library, that has an improved 2x performance.

go-enry 284 Jun 28, 2022
TProx is a fast reverse proxy path traversal detector and directory bruteforcer.

TProx is a fast reverse proxy path traversal detector and directory bruteforcer Install • Usage • Examples • Join Discord Install Options From Source

Krypt0mux 27 Jun 13, 2022
A detector for the Trojan Source and other unicode-based vulnerabilities.

Trojan Source Detector This application detects Trojan Source attacks in source code. It can be used as part of the CI system to make sure there are n

Have you debugged.it? 5 Jan 6, 2022
A Log4J Version 2 Detector written in golang

Installation From source: go install github.com/juergenhoetzel/log4j2go/cmd/log4

Jürgen Hötzel 1 Dec 20, 2021
Log4j detector and reporting server for scalable detection of vulnerable running processes.

Log4j Detector A client and reporting server to identify systems vulnerable to Log4j at scale. This work is based on Stripe's Remediation Tools, but w

Praetorian 8 Apr 8, 2022