🐝 A Highly Performant and easy to use goroutine pool for Go

Overview

gohive

Package gohive implements a simple and easy to use goroutine pool for Go

Features

  • Pool can be created with a specific size as per the requirement
  • Offers efficient performance by implementing sync.Pool, which maintains pool of workers in which workers gets recycled automatically when not in use
  • Implements a Task Queue which can hold surplus tasks in waiting, if submitted more than the pool capacity
  • Implements PoolService type, which acts as an easy to use API with simple methods to interact with gohive
  • Gracefully handles panics and prevent the application from getting crashed or into deadlocks
  • Provides functions like: AvailableWorkers(), ActiveWorkers() and Close() etc.

Installation

Use go get to install and update:

$ go get -u github.com/loveleshsharma/gohive

Usage

  • Create an instance of PoolService type first
hive := gohive.NewFixedSizePool(5)
  • Invoke the Submit() function and pass the task to execute
hive.Submit(someTask())

Submit function accepts a function as an argument, which it passes to the pool if a worker is available, otherwise enqueues it in a waiting queue

  • To close the pool we can invoke the Close() function
hive.Close()

Once the pool is closed, we cannot assign any task to it

Example

Let's get into a full program where we can see how to use the gohive package in order to execute many goroutines simultaneously

package main

import (
	"github.com/loveleshsharma/gohive"
	"fmt"
	"sync"
)

func main() {

	var wg sync.WaitGroup
	hivePool := gohive.NewFixedSizePool(5)

	//wrap your executable function into another function with wg.Done()
	executableTask := func() {
		defer wg.Done()
		factorial(5)
	}

	wg.Add(1)
	hivePool.Submit(executableTask)

	wg.Wait()
}

func factorial(val int) {
	var fact = val
	var res = 1

	for i := fact; i > 0; i-- {
		res = res * i
	}

	fmt.Printf("Factorial: %v", res)
}

Important : Always put defer wg.Done() as the first statement of your wrapper function. It will prevent your program from deadlocks in case of panics

Workers implements a notifying mechanism, due to which they can notify to the pool that their task is completed and they are available to execute more tasks if in waiting queue

You might also like...
Go-miningcore-pool - Miningcore Pool written in GOlang

Go-Miningcore-Pool (COMING SOON) Miningcore Pool written in GOlang 0x01 Configur

Go-ldap-pool - A simple connection pool for go-ldap

Basic connection pool for go-ldap This little library use the go-ldap library an

Work pool channlege - An url hash retriever worker pool for getting hash digest for a collection of urls

Code challenge The aim of this project is to provide an url hash retriever worke

Worker pool library with auto-scaling, backpressure, and easy composability of pools into pipelines

workerpool Worker pool library with auto-scaling, backpressure, and easy composability of pools into pipelines. Uses Go 1.18 generics. Notable differe

A simple and useful goroutine concurrent library.

Taskgroup A simple and useful goroutine concurrent library. Installation go get github.com/anthhub/taskgroup

Provides some convenient API, includes Goid(), AllGoid(), and LocalStorage, which is a goroutine's local storage, just like ThreadLocal in other languages.

routine 中文版 routine encapsulates and provides some easy-to-use, high-performance goroutine context access interfaces, which can help you access corout

A cross goroutine storage tool with very simple implementation and function.

Simple-goroutine-local is a cross goroutine storage tool with very simple implementation and function (the concept is similar to Java ThreadLocal). Ge

errgroup with goroutine worker limits

neilotoole/errgroup neilotoole/errgroup is a drop-in alternative to Go's wonderful sync/errgroup but limited to N goroutines. This is useful for inter

Waiting group for collecting goroutine information.

在go语言waitGroup和errGroup都是用来控制goroutine的并发的方式,前者只能等待所有goroutine执行完成之后再执行Wait()函数后面的代码并且不

Comments
  • Can I use gohive for executing task one by one?

    Can I use gohive for executing task one by one?

    Can I use gohive for executing tasks one by one like a serial queue?

    For example:

         task1  ->  task2 -> task3
    

    when task1 is completed, task2 will be executed, and when task2 is completed, task3 will be executed.

    Finally, If yes, how to do it using gohive, can you give some examples in the Readme?

    opened by haifenghuang 0
  • hive.Submit(someTask()) not working properly

    hive.Submit(someTask()) not working properly

    var wg sync.WaitGroup
    hivePool := gohive.NewFixedSizePool(5)
    
    //wrap your executable function into another function with wg.Done()
    executableTask := func(i int) {
    	defer wg.Done()
    	factorial(i)
    }
    
    wg.Add(1)
    hivePool.Submit(executableTask(5))
    
    wg.Wait()
    

    what i get: ./example1.go:22:32: executableTask(5) used as value

    what i expected: Working working properly

    opened by raszia 0
Owner
Lovelesh
Software Enthusiast | Server-Side Engineer | Loves to CoDe❤️
Lovelesh
🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants 是一个高性能且低损耗的 goroutine 池。

A goroutine pool for Go English | ???? 中文 ?? Introduction Library ants implements a goroutine pool with fixed capacity, managing and recycling a massi

Andy Pan 9.2k Sep 30, 2022
golang worker pool , Concurrency limiting goroutine pool

golang worker pool 中文说明 Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks su

xxj 414 Sep 19, 2022
Minimalistic and High-performance goroutine worker pool written in Go

pond Minimalistic and High-performance goroutine worker pool written in Go Motivation This library is meant to provide a simple way to limit concurren

Alejandro Durante 646 Oct 1, 2022
Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Bo-Yi Wu 223 Sep 20, 2022
Queue is a Golang library for spawning and managing a Goroutine pool

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

golang-queue 223 Sep 20, 2022
Lightweight Goroutine pool

grpool Lightweight Goroutine pool Clients can submit jobs. Dispatcher takes job, and sends it to first available worker. When worker is done with proc

Ivan Pusic 711 Sep 20, 2022
A goroutine pool for Go

Tunny is a Golang library for spawning and managing a goroutine pool, allowing you to limit work coming from any number of goroutines with a synchrono

Ashley Jeffs 3.2k Sep 24, 2022
Concurrency limiting goroutine pool

workerpool Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting task

Andrew Gillis 923 Sep 27, 2022
goroutine pool in golang

goroutine pool in golang

wksw 1 Nov 1, 2021
Golang Implementation of Worker Pool/ Thread Pool

Golang Implementation of Worker Pool/ Thread Pool

Telkom DEV 1 Jun 18, 2022