concurrency utilities

Overview

concurrent

Sourcegraph GoDoc Build Status codecov rcard License

  • concurrent.Map: backport sync.Map for go below 1.9
  • concurrent.Executor: goroutine with explicit ownership and cancellable

concurrent.Map

because sync.Map is only available in go 1.9, we can use concurrent.Map to make code portable

m := concurrent.NewMap()
m.Store("hello", "world")
elem, found := m.Load("hello")
// elem will be "world"
// found will be true

concurrent.Executor

executor := concurrent.NewUnboundedExecutor()
executor.Go(func(ctx context.Context) {
    everyMillisecond := time.NewTicker(time.Millisecond)
    for {
        select {
        case <-ctx.Done():
            fmt.Println("goroutine exited")
            return
        case <-everyMillisecond.C:
            // do something
        }
    }
})
time.Sleep(time.Second)
executor.StopAndWaitForever()
fmt.Println("executor stopped")

attach goroutine to executor instance, so that we can

  • cancel it by stop the executor with Stop/StopAndWait/StopAndWaitForever
  • handle panic by callback: the default behavior will no longer crash your application
Issues
  • codecov in .travis.yml - security issue

    codecov in .travis.yml - security issue

    Good day, https://codecov.io/bash recently outputted a security issue. Details can be found at https://about.codecov.io/security-update/

    The following file and code are impacted. Please follow the recommended actions within the link above to secure your env and code integrity. file: .travis.yml Code: after_success:

    bash <(curl -s https://codecov.io/bash)

    opened by windsparks33 0
  • heads up: codecov.io security incident - https://about.codecov.io/security-update/

    heads up: codecov.io security incident - https://about.codecov.io/security-update/

    Hi there.

    This might be an unusual "issue" beeing reported.

    There has been a security incident in codecov.io with the bash-uploader script (see [1] for details) which potentially exposed secrets to 3rd parties.

    It seems you are using the referenced bash uploader in your .travis.yml file. I wanted to draw your attention to this incident in case you missed it.

    It would be great if you could verify that no code has been altered and check the impact of this security incident on your repository.

    Regards, Robert

    [1] https://about.codecov.io/security-update/

    opened by robert-heinzmann-logmein 0
  • Get rid of `context.TODO()`

    Get rid of `context.TODO()`

    Our code analysis is flagging the use of context.TODO() in NewUnboundedExecutor.

    What is the intent here? To pass a context to the constructor or if not, should context.TODO() be replaced by context.Background()?

    opened by JRBANCEL 0
  • Add a copyright / notice file

    Add a copyright / notice file

    This project doesn't appear to include any copyright information or a NOTICE file. Can you add one or update the Apache License with this information? This is desired to comply with the open source license conditions.

    opened by davidmateos 0
  • All tests are failing, if using gollvm

    All tests are failing, if using gollvm

    Hello.

    Here is my setup

    $ go get -v -x -u github.com/modern-go/concurrent cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git config remote.origin.url github.com/modern-go/concurrent (download) cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git pull --ff-only cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git submodule update --init --recursive cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git show-ref cd /home/oceanfish81/go/src/github.com/modern-go/concurrent git submodule update --init --recursive WORK=/tmp/go-build480857928 github.com/modern-go/concurrent mkdir -p $WORK/b001/ cd $WORK /usr/local/bin/llvm-goc -fgo-importcfg=/dev/null -c -x c - -o /dev/null || true cd /home/oceanfish81/go/src/github.com/modern-go/concurrent /usr/local/bin/llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -fgo-pkgpath=github.com/modern-go/concurrent -o $WORK/b001/go.o -I $WORK/b001/importcfgroot ./executor.go ./go_above_19.go ./log.go ./unbounded_executor.go echo ' .section .go.buildid,"e"' >> $WORK/b001/_buildid.s echo ' .byte 0x76,0x77,0x37,0x43,0x53,0x34,0x5a,0x72' >> $WORK/b001/_buildid.s echo ' .byte 0x5a,0x42,0x6f,0x4b,0x31,0x2d,0x30,0x6f' >> $WORK/b001/_buildid.s echo ' .byte 0x69,0x53,0x2d,0x7a,0x2f,0x76,0x77,0x37' >> $WORK/b001/_buildid.s echo ' .byte 0x43,0x53,0x34,0x5a,0x72,0x5a,0x42,0x6f' >> $WORK/b001/_buildid.s echo ' .byte 0x4b,0x31,0x2d,0x30,0x6f,0x69,0x53,0x2d' >> $WORK/b001/_buildid.s echo ' .byte 0x7a' >> $WORK/b001/_buildid.s echo ' .section .note.GNU-stack,"", progbits' >> $WORK/b001/_buildid.s echo ' .section .note.GNU-split-stack,"", progbits' >> $WORK/b001/_buildid.s echo '' >> $WORK/b001/_buildid.s /usr/local/bin/llvm-goc -xassembler-with-cpp -I $WORK/b001/ -c -o $WORK/b001/_buildid.o -D GOOS_linux -D GOARCH_amd64 -D GOPKGPATH=github.x2ecom..z2fmodern..z2dgo..z2fconcurrent -m64 $WORK/b001/_buildid.s ar rcD $WORK/b001/pkg.a $WORK/b001/go.o $WORK/b001/_buildid.o /usr/local/tools/buildid -w $WORK/b001/pkg.a # internal cp $WORK/b001/pkg.a /home/oceanfish81/.cache/go-build/35/35fa2bf6472674cdf2e146f2fab7911cd73474608b6a07d5790605cd8c9d8a3b-d # internal mkdir -p /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/ mv $WORK/b001/pkg.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a rm -r $WORK/b001/

    Checking if there wouldn't be any errors:

    $ go install -i -v -x github.com/modern-go/concurrent Checking an enforced build process: $ go build -i -v -x github.com/modern-go/concurrent WORK=/tmp/go-build024139235 Navigating into my source folder: $ cd ~/go/src/github.com/modern-go/concurrent

    And then I got two tests failing:

    $ go test map_test.go -x -v && go test unbounded_executor_test.go -v -x WORK=/tmp/go-build608777315 mkdir -p $WORK/b001/ mkdir -p $WORK/b009/ mkdir -p $WORK/b008/ cat >$WORK/b008/vet.cfg << 'EOF' # internal { "ID": "command-line-arguments_test", "Compiler": "gccgo", "Dir": "/home/oceanfish81/go/src/github.com/modern-go/concurrent", "ImportPath": "command-line-arguments_test", "GoFiles": [ "/home/oceanfish81/go/src/github.com/modern-go/concurrent/map_test.go" > ], "NonGoFiles": [], "ImportMap": { "github.com/modern-go/concurrent": "github.com/modern-go/concurrent", "testing": "testing" }, "PackageFile": { "github.com/modern-go/concurrent": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a" }, "Standard": { "testing": true }, "PackageVetx": { "github.com/modern-go/concurrent": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d" }, "VetxOnly": false, "VetxOutput": "$WORK/b008/vet.out", "SucceedOnTypecheckFailure": false } EOF cd /home/oceanfish81/go/src/github.com/modern-go/concurrent TERM='dumb' GCCGO='/usr/local/bin/llvm-goc' /usr/local/tools/vet -atomic -bool -buildtags -errorsas -ifaceassert -nilfunc -printf -stringintconv $WORK/b008/vet.cfg === RUN TestMap_Load --- PASS: TestMap_Load (0.00s) PASS ok command-line-arguments (cached) rm -r $WORK/b001/ WORK=/tmp/go-build599881499 mkdir -p $WORK/b001/ mkdir -p $WORK/b011/ mkdir -p $WORK/b008/ cd $WORK /usr/local/bin/llvm-goc -fgo-importcfg=/dev/null -c -x c - -o /dev/null || true mkdir -p $WORK/b008/importcfgroot/github.com/modern-go ln -s /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a $WORK/b008/importcfgroot/github.com/modern-go/libconcurrent.a cd /home/oceanfish81/go/src/github.com/modern-go/concurrent /usr/local/bin/llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -fgo-pkgpath=command-line-arguments_test -fgo-relative-import-path=_/home/oceanfish81/go/src/github.com/modern-go/concurrent -o $WORK/b008/go.o -I $WORK/b008/importcfgroot ./unbounded_executor_test.go echo ' .section .go.buildid,"e"' >> $WORK/b008/_buildid.s echo ' .byte 0x64,0x49,0x4c,0x30,0x61,0x6e,0x54,0x46' >> $WORK/b008/_buildid.s echo ' .byte 0x37,0x70,0x4b,0x68,0x47,0x70,0x50,0x32' >> $WORK/b008/_buildid.s echo ' .byte 0x76,0x79,0x79,0x30,0x2f,0x64,0x49,0x4c' >> $WORK/b008/_buildid.s echo ' .byte 0x30,0x61,0x6e,0x54,0x46,0x37,0x70,0x4b' >> $WORK/b008/_buildid.s echo ' .byte 0x68,0x47,0x70,0x50,0x32,0x76,0x79,0x79' >> $WORK/b008/_buildid.s echo ' .byte 0x30' >> $WORK/b008/_buildid.s echo ' .section .note.GNU-stack,"", progbits' >> $WORK/b008/_buildid.s echo ' .section .note.GNU-split-stack,"", progbits' >> $WORK/b008/_buildid.s echo '' >> $WORK/b008/_buildid.s /usr/local/bin/llvm-goc -xassembler-with-cpp -I $WORK/b008/ -c -o $WORK/b008/_buildid.o -D GOOS_linux -D GOARCH_amd64 -D GOPKGPATH=command..z2dline..z2darguments_test -m64 $WORK/b008/_buildid.s ar rcD $WORK/b008/pkg.a $WORK/b008/go.o $WORK/b008/_buildid.o /usr/local/tools/buildid -w $WORK/b008/pkg.a # internal cp $WORK/b008/pkg.a /home/oceanfish81/.cache/go-build/1f/1fd3390c5dd92a008d2092c605ef674dc8ab4e8cfe0b010f939d168507b69f1d-d # internal cat >$WORK/b008/vet.cfg << 'EOF' # internal { "ID": "command-line-arguments_test", "Compiler": "gccgo", "Dir": "/home/oceanfish81/go/src/github.com/modern-go/concurrent", "ImportPath": "command-line-arguments_test", "GoFiles": [ "/home/oceanfish81/go/src/github.com/modern-go/concurrent/unbounded_executor_test.go" > ], "NonGoFiles": [], "ImportMap": { "context": "context", "fmt": "fmt", "github.com/modern-go/concurrent": "github.com/modern-go/concurrent", "time": "time" }, "PackageFile": { "github.com/modern-go/concurrent": "/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a" }, "Standard": { "context": true, "fmt": true, "time": true }, "PackageVetx": { "github.com/modern-go/concurrent": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d" }, "VetxOnly": false, "VetxOutput": "$WORK/b008/vet.out", "SucceedOnTypecheckFailure": false } EOF TERM='dumb' GCCGO='/usr/local/bin/llvm-goc' /usr/local/tools/vet -atomic -bool -buildtags -errorsas -ifaceassert -nilfunc -printf -stringintconv $WORK/b008/vet.cfg mkdir -p $WORK/b001/importcfgroot ln -s $WORK/b008/pkg.a $WORK/b001/importcfgroot/libcommand-line-arguments_test.a cd $WORK/b001/ /usr/local/bin/llvm-goc -c -O2 -g -m64 -fdebug-prefix-map=$WORK=/tmp/go-build -gno-record-gcc-switches -o ./go.o -I ./importcfgroot ./_testmain.go echo ' .section .go.buildid,"e"' >> $WORK/b001/_buildid.s echo ' .byte 0x68,0x5f,0x4a,0x52,0x77,0x48,0x41,0x50' >> $WORK/b001/_buildid.s echo ' .byte 0x72,0x4a,0x42,0x65,0x56,0x37,0x32,0x51' >> $WORK/b001/_buildid.s echo ' .byte 0x70,0x6c,0x43,0x79,0x2f,0x68,0x5f,0x4a' >> $WORK/b001/_buildid.s echo ' .byte 0x52,0x77,0x48,0x41,0x50,0x72,0x4a,0x42' >> $WORK/b001/_buildid.s echo ' .byte 0x65,0x56,0x37,0x32,0x51,0x70,0x6c,0x43' >> $WORK/b001/_buildid.s echo ' .byte 0x79' >> $WORK/b001/_buildid.s echo ' .section .note.GNU-stack,"", progbits' >> $WORK/b001/_buildid.s echo ' .section .note.GNU-split-stack,"", progbits' >> $WORK/b001/_buildid.s echo '' >> $WORK/b001/_buildid.s /usr/local/bin/llvm-goc -xassembler-with-cpp -I ./ -c -o ./_buildid.o -D GOOS_linux -D GOARCH_amd64 -m64 ./_buildid.s ar rcD ./pkg.a ./go.o ./_buildid.o /usr/local/tools/buildid -w $WORK/b001/pkg.a # internal cp $WORK/b001/pkg.a /home/oceanfish81/.cache/go-build/07/072aa11d607b29519a08949c3fb6e70f5c63af6047da44221df21a9212ad5822-d # internal cat >$WORK/b001/importcfg.link << 'EOF' # internal packagefile command-line-arguments.test=$WORK/b001/pkg.a packagefile command-line-arguments_test=$WORK/b008/pkg.a packagefile github.com/modern-go/concurrent=/home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a EOF cd . /usr/local/bin/llvm-goc -o $WORK/b001/concurrent.test "-Wl,-(" -m64 -Wl,--whole-archive $WORK/b001/pkg.a $WORK/b008/pkg.a /home/oceanfish81/go/pkg/gccgo_linux_amd64/github.com/modern-go/libconcurrent.a -Wl,--no-whole-archive "-Wl,-)" -Wl,--build-id=0x66657a6d737562503939456e6e6161584575326b2f685f4a5277484150724a426556373251706c43792f34505f7041394f4e59306b7071657952414141662f66657a6d737562503939456e6e6161584575326b $WORK/b001/concurrent.test -test.testlogfile=$WORK/b001/testlog.txt -test.timeout=10m0s -test.v=true === RUN ExampleUnboundedExecutor_Go --- PASS: ExampleUnboundedExecutor_Go (1.00s) === RUN ExampleUnboundedExecutor_StopAndWaitForever --- PASS: ExampleUnboundedExecutor_StopAndWaitForever (1.10s) === RUN ExampleUnboundedExecutor_Go_panic --- FAIL: ExampleUnboundedExecutor_Go_panic (1.00s) got: concurrent_test.willPanic want: github.com/modern-go/concurrent_test.willPanic FAIL FAIL command-line-arguments 3.541s rm -r $WORK/b001/ FAIL

    Ivan

    opened by advancedwebdeveloper 1
Releases(1.0.3)
Owner
Modern Go Programming
Better developer experience
Modern Go Programming
Go-concurrency-patterns - Sample concurrency patterns with Goroutines

About This sample project provides some concurrency pattern examples in Go using

George Batagiannis ⚡ 1 Feb 21, 2022
concurrency utilities

concurrent concurrent.Map: backport sync.Map for go below 1.9 concurrent.Executor: goroutine with explicit ownership and cancellable concurrent.Map be

Modern Go Programming 219 Jun 26, 2022
Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Christian Muehlhaeuser 1.8k Jun 23, 2022
A simple logging interface that supports cross-platform color and concurrency.

WLog Package wlog creates simple to use UI structure. The UI is used to simply print to the screen. There a wrappers that will wrap each other to crea

Will Dixon 57 May 31, 2022
Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Christian Muehlhaeuser 1.8k Jul 3, 2022
Code Generation for Functional Programming, Concurrency and Generics in Golang

goderive goderive derives mundane golang functions that you do not want to maintain and keeps them up to date. It does this by parsing your go code fo

Walter Schulze 987 Jun 29, 2022
A sync.WaitGroup with error handling and concurrency control

go-waitgroup How to use An package that allows you to use the constructs of a sync.WaitGroup to create a pool of goroutines and control the concurrenc

Pieter Claerhout 31 Jun 17, 2022
golang worker pool , Concurrency limiting goroutine pool

golang worker pool 中文说明 Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks su

xxj 398 Jun 21, 2022
gpool - a generic context-aware resizable goroutines pool to bound concurrency based on semaphore.

gpool - a generic context-aware resizable goroutines pool to bound concurrency. Installation $ go get github.com/sherifabdlnaby/gpool import "github.c

Sherif Abdel-Naby 84 Feb 21, 2022
Structured Concurrency in Go

nursery: structured concurrency in Go RunConcurrently( // Job 1 func(context.Context, chan error) { time.Sleep(time.Millisecond * 10)

null 45 May 21, 2022
Concurrency limiting goroutine pool

workerpool Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting task

Andrew Gillis 841 Jun 24, 2022
Pholcus is a distributed high-concurrency crawler software written in pure golang

Pholcus Pholcus(幽灵蛛)是一款纯 Go 语言编写的支持分布式的高并发爬虫软件,仅用于编程学习与研究。 它支持单机、服务端、客户端三种运行模式,拥有Web、GUI、命令行三种操作界面;规则简单灵活、批量任务并发、输出方式丰富(mysql/mongodb/kafka/csv/excel等

henrylee2cn 7.1k Jun 24, 2022
Helper library for full uint64 randomness, pool backed for efficient concurrency

fastrand64-go Helper library for full uint64 randomness, pool backed for efficient concurrency Inspired by https://github.com/valyala/fastrand which i

Ryan Haksi 41 Dec 5, 2021
dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies

dagger is a blazing fast, concurrency safe, mutable, in-memory directed graph implementation with zero dependencies

Coleman Word 259 Jun 30, 2022
Concurrency in Go video course with in depth explanations & examples

Concurrency in Go Summary Coding Examples Introduction to Concurrency Go Routines Channels Select Concurrency Patterns Atomics Wait Groups - sync.Wait

Go Basics 167 Jun 26, 2022
yakv is a simple, in-memory, concurrency-safe key-value store for hobbyists.

yakv (yak-v. (originally intended to be "yet-another-key-value store")) is a simple, in-memory, concurrency-safe key-value store for hobbyists. yakv provides persistence by appending transactions to a transaction log and restoring data from the transaction log on startup.

Aadhav Vignesh 5 Feb 24, 2022
This repository collects common concurrency patterns in Golang

Go Concurrency Patterns This repository collects common concurrency patterns in Golang Materials Concurrency is not parallelism Go Concurrency Pattern

Kha Nguyen 1.7k Jun 23, 2022
Concurrency patterns in Go

Concurrency patterns in Go

Kha Nguyen 1.7k Jun 28, 2022
File Processor in Concurrency Pattern using Golang goroutine.

File Processor in Concurrency Pattern Implement a file processor solution in concurrency pattern using Golang goroutine. Get Started Run docker-compos

null 1 Oct 17, 2021
Concurrency Lab 2 Go Example

Concurrency Lab 2 If you're stuck look at examples on Go by Example Using the lab sheet There are two ways to use the lab sheet, you can either: creat

null 0 Oct 12, 2021
Concurrency Lab 1 Go Example

Concurrency Lab 1 If you're stuck look at examples on Go by Example Using the lab sheet There are two ways to use the lab sheet, you can either: creat

null 0 Oct 11, 2021
Concurrency Lab examples on Go

Concurrency Lab 1 If you're stuck look at examples on Go by Example Using the lab sheet There are two ways to use the lab sheet, you can either: creat

null 0 Oct 22, 2021
Go lang concurrency with channels and worker pools using sync.Pools.

Go Concurrency Go lang concurrency with channels and worker pools using sync.Pools. Concurrency Concurrency is an ability of a program to do multiple

Kushal Khadka 0 Oct 31, 2021
A CLI based Downloader utilising Golang's concurrency feature.

Downloader CLI based downloader utilising the concurrency feature of Golang. Takes the download URL of the file from the user. Splits the files into s

Mohammad Warid 0 Nov 6, 2021
Pengenalan Concurrency dan Parallel Programming

Golang Goroutine Sumber Tutorial: Udemy Slide Pengenalan Concurrency dan Parallel Programming Pengenalan Parallel Programming Saat ini kita hidup dima

Akbarputra 0 Nov 5, 2021
Understand go concurrency

CONTENT (click to expand or hide) What is Concurrency? Why we need to think about Concurrency? What is a Process? What is a Thread? Thread_States C10k

kemalcan bora 8 Feb 17, 2022
SGPM: A coroutine scheduling model for wound-wait concurrency control optimization

Environment set up off the mod go env -w GO111MODULE=off change to GPATH to project directory go env -w GOPATH=$HOME/sgpm Usage This project serves th

Jack Wang 0 Dec 12, 2021
`kawipiko` -- blazingly fast static HTTP server -- focused on low latency and high concurrency, by leveraging Go, `fasthttp` and the CDB embedded database

kawipiko -- blazingly fast static HTTP server kawipiko is a lightweight static HTTP server written in Go; focused on serving static content as fast an

Volution 14 Jun 22, 2022
Simple tuning work for go programs in high concurrency scenarios.

go-program-tuning Simple tuning work for go programs in high concurrency scenarios. Installation Run the following command under your project: go get

Afeyer 3 Mar 15, 2022