Concurrency patterns in Go

Overview

Go Concurrency Patterns

This repository collects common concurrency patterns in Golang

Materials

Context:

Name Description Playground
1-boring A hello world to goroutine play
2-chan A hello world to go channel play
3-generator A python-liked generator play
4-fanin Fan in pattern play
5-restore-sequence Restore sequence play
6-select-timeout Add Timeout to a goroutine play
7-quit-signal Quit signal play
8-daisy-chan Daisy chan pattern play
9-google1.0 Build a concurrent google search from the ground-up play
10-google2.0 Build a concurrent google search from the ground-up play
11-google2.1 Build a concurrent google search from the ground-up play
12-google3.0 Build a concurrent google search from the ground-up play
13-adv-pingpong A sample ping-pong table implemented in goroutine play
14-adv-subscription Subscription play
15-bounded-parallelism Bounded parallelism play
16-context How to user context in HTTP client and server play
17-ring-buffer-channel Ring buffer channel play
18-worker-pool worker pool pattern play
You might also like...
Practical concurrency guide in Go, communication by channels, patterns

Go Concurrency Guide This guide is built on top of the some examples of the book Go Concurrency in Go and Go Programming Language Race Condition and D

LogAnalyzer - Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory.
LogAnalyzer - Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory.

LogAnalyzer Analyze logs with custom regex patterns.Can search for particular patterns on multiple files in a directory

Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

A simple logging interface that supports cross-platform color and concurrency.
A simple logging interface that supports cross-platform color and concurrency.

WLog Package wlog creates simple to use UI structure. The UI is used to simply print to the screen. There a wrappers that will wrap each other to crea

Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Code Generation for Functional Programming, Concurrency and Generics in Golang

goderive goderive derives mundane golang functions that you do not want to maintain and keeps them up to date. It does this by parsing your go code fo

A sync.WaitGroup with error handling and concurrency control

go-waitgroup How to use An package that allows you to use the constructs of a sync.WaitGroup to create a pool of goroutines and control the concurrenc

golang worker pool , Concurrency limiting goroutine pool

golang worker pool 中文说明 Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks su

gpool - a generic context-aware resizable goroutines pool to bound concurrency based on semaphore.

gpool - a generic context-aware resizable goroutines pool to bound concurrency. Installation $ go get github.com/sherifabdlnaby/gpool import "github.c

Structured Concurrency in Go

nursery: structured concurrency in Go RunConcurrently( // Job 1 func(context.Context, chan error) { time.Sleep(time.Millisecond * 10)

Concurrency limiting goroutine pool

workerpool Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting task

concurrency utilities

concurrent concurrent.Map: backport sync.Map for go below 1.9 concurrent.Executor: goroutine with explicit ownership and cancellable concurrent.Map be

Pholcus is a distributed high-concurrency crawler software written in pure golang
Pholcus is a distributed high-concurrency crawler software written in pure golang

Pholcus Pholcus(幽灵蛛)是一款纯 Go 语言编写的支持分布式的高并发爬虫软件,仅用于编程学习与研究。 它支持单机、服务端、客户端三种运行模式,拥有Web、GUI、命令行三种操作界面;规则简单灵活、批量任务并发、输出方式丰富(mysql/mongodb/kafka/csv/excel等

Helper library for full uint64 randomness, pool backed for efficient concurrency

fastrand64-go Helper library for full uint64 randomness, pool backed for efficient concurrency Inspired by https://github.com/valyala/fastrand which i

dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies
dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies

dagger is a blazing fast, concurrency safe, mutable, in-memory directed graph implementation with zero dependencies

Concurrency  in Go video course with in depth explanations & examples
Concurrency in Go video course with in depth explanations & examples

Concurrency in Go Summary Coding Examples Introduction to Concurrency Go Routines Channels Select Concurrency Patterns Atomics Wait Groups - sync.Wait

yakv is a simple, in-memory, concurrency-safe key-value store for hobbyists.
yakv is a simple, in-memory, concurrency-safe key-value store for hobbyists.

yakv (yak-v. (originally intended to be "yet-another-key-value store")) is a simple, in-memory, concurrency-safe key-value store for hobbyists. yakv provides persistence by appending transactions to a transaction log and restoring data from the transaction log on startup.

 File Processor in Concurrency Pattern using Golang goroutine.
File Processor in Concurrency Pattern using Golang goroutine.

File Processor in Concurrency Pattern Implement a file processor solution in concurrency pattern using Golang goroutine. Get Started Run docker-compos

Concurrency Lab 2 Go Example
Concurrency Lab 2 Go Example

Concurrency Lab 2 If you're stuck look at examples on Go by Example Using the lab sheet There are two ways to use the lab sheet, you can either: creat

Comments
  • Different approach for ring buffer

    Different approach for ring buffer

    https://github.com/lotusirous/go-concurrency-patterns/blob/82c23134af1a6f4a847338e0eadfde940a35a608/17-ring-buffer-channel/main.go#L28

    This is a bug surely? Another routine could write between these two calls, and then this write would block.

    A safer approach would be to drop the current write AND one from the ring buffer. This would ensure that no code ever blocks, but you are still guaranteed new data (which IMO is a big use-case for ring buffers).

    This would ensure no back-pressure. It would make starvation technically possible very rarely by some complex timing and workload patterns if you have too many writers for the size of the ring buffer, but this is trivial to avoid by sizing the ring buffer appropriately.

    So as long as your ring buffer isn't too small, I think it would behave more like what people would expect.

    opened by ansoni-san 0
  • timeout example is unsuitable

    timeout example is unsuitable

    https://github.com/lotusirous/go-concurrency-patterns/blob/2c165868b5befa1f6da1c9cb7f6db7980c63d005/6-select-timeout/main.go#L25-L35

    We always think timeout as the time that one service has on response. In this case, timeout 's meaning should be ** we don't receive any message from c channel for 5 seconds.

    so maybe the right code is :

    timeout := time.After(5 * time.Second)
    for {
           select {
           case s := <-c:
                   fmt.Println(s)
                   timeout = time.After(5 * time.Second) // update timeout time
           case <-timeout:
                   fmt.Println("channel has no response for 5 seconds")
                   return
           }
    }
    

    the origin code seems that you want to receive from every channel for same time, like linux cfs scheduler. so maybe in that case we should use ticker

    package main
    
    import (
            "fmt"
            "time"
    )
    
    func boring(id int) <-chan string {
            c := make(chan string)
            go func() {
                     c <- "" // skip the first time
                    for i := 0; ; i++ {
                            c <- fmt.Sprintf("%d, %s", id, time.Now().Format("2006-01-02 15:04:05"))
                            time.Sleep(1 * time.Second)
                    }
            }()
            return c
    }
    
    func main() {
            timeout := time.NewTicker(5 * time.Second)
            c1 := boring(1)
            c2 := boring(2)
            jobchannels := []<-chan string{c1, c2}
            i := 0
            for {
                    select {
                    case s := <-jobchannels[i]:
                            fmt.Println(s)
                    case <-timeout.C:
                            fmt.Printf("%d has talk for 5 secs\n", i+1)
                            i = (i + 1) % len(jobchannels)
                    }
            }
    }
    

    and it's output is

    
    1, 2021-03-17 11:27:03
    1, 2021-03-17 11:27:04
    1, 2021-03-17 11:27:05
    1, 2021-03-17 11:27:06
    1, 2021-03-17 11:27:07
    1 has talk for 5 secs
    
    2, 2021-03-17 11:27:08
    2, 2021-03-17 11:27:09
    2, 2021-03-17 11:27:10
    2, 2021-03-17 11:27:11
    2, 2021-03-17 11:27:12
    2 has talk for 5 secs
    1, 2021-03-17 11:27:08
    1, 2021-03-17 11:27:14
    1, 2021-03-17 11:27:15
    1, 2021-03-17 11:27:16
    ....
    
    opened by kele1997 1
Owner
Kha Nguyen
I am an independent security enthusiast, AI engineer. I love Vim and Go. I use this corner to share tools that I work on.
Kha Nguyen
Practical concurrency guide in Go, communication by channels, patterns

Go Concurrency Guide This guide is built on top of the some examples of the book Go Concurrency in Go and Go Programming Language Race Condition and D

Lucas Alves 1.9k Nov 27, 2022
Understand go concurrency

CONTENT (click to expand or hide) What is Concurrency? Why we need to think about Concurrency? What is a Process? What is a Thread? Thread_States C10k

kemalcan bora 7 Jul 11, 2022
23 design patterns of GoF

GoF 设计模式 GoF所提出的23种设计模式主要基于以下面向对象设计原则: 对接口编程而不是对实现编程 优先使用对象组合而不是继承 23种设计模式分为三大类:创建型模式(Creational Patterns)、结构型模式(Structural Patterns)、行为型模式(Behavioral

Gina 167 Nov 7, 2022
Binaryscarf generates double-knitting patterns for some corpus of input text.

binaryscarf binaryscarf generates double-knit patterns for some corpus of input text. The layout follows the same style as described here. Output is s

null 1 Nov 28, 2021
Go-design-pattern-examples - Golang implementations of common design patterns

Design Patterns Golang implementations of common design patterns Build project T

null 4 Oct 29, 2022
Go Design Patterns

Creational patterns provide out-of-the-box objects for users instead relying on the user to know how to build the object which in some cases could be complex.

Ari V 0 Feb 8, 2022
Patternfinder - Find patterns in http output based on regex string. Display occurences

Patternfinder Find patterns in HTTP output based on regex string. Display occurr

YouGina 1 Feb 18, 2022
A faster RWLock primitive in Go, 2-3 times faster than RWMutex. A Go implementation of concurrency control algorithm in paper

Go Left Right Concurrency A Go implementation of the left-right concurrency control algorithm in paper <Left-Right - A Concurrency Control Technique w

wangyi 42 Aug 28, 2022
This repository collects common concurrency patterns in Golang

Go Concurrency Patterns This repository collects common concurrency patterns in Golang Materials Concurrency is not parallelism Go Concurrency Pattern

Kha Nguyen 2k Nov 27, 2022
Concurrency patterns in Go

Concurrency patterns in Go

Kha Nguyen 2k Nov 27, 2022