Floc: Orchestrate goroutines with ease.

Overview

Gopher Floc Control

go-floc

Floc: Orchestrate goroutines with ease.

GoDoc Build Status Coverage Status Go Report Card Join the chat at https://gitter.im/go-floc/Lobby License

The goal of the project is to make the process of running goroutines in parallel and synchronizing them easy.

Announcements

Hooray! The new version v2 is released on the 1st of December, 2017!

Installation and requirements

The package requires Go v1.8 or later.

To install the package use go get gopkg.in/workanator/go-floc.v2

Documentation and examples

Please refer Godoc reference of the package for more details.

Some examples are available at the Godoc reference. Additional examples can be found in go-floc-showcase.

Features

  • Easy to use functional interface.
  • Simple parallelism and synchronization of jobs.
  • As little overhead as possible, in comparison to direct use of goroutines and sync primitives.
  • Provide better control over execution with one entry point and one exit point.

Introduction

Floc introduces some terms which are widely used through the package.

Flow

Flow is the overall process which can be controlled through floc.Flow. Flow can be canceled or completed with any arbitrary data at any point of execution. Flow has only one enter point and only one exit point.

// Design the job
flow := run.Sequence(do, something, here, ...)

// The enter point: Run the job
result, data, err := floc.Run(flow)

// The exit point: Check the result of the job.
if err != nil {
	// Handle the error
} else if result.IsCompleted() {
	// Handle the success
} else {
	// Handle other cases
}

Job

Job in Floc is a smallest piece of flow. The prototype of job function is floc.Job. Each job can read/write data with floc.Context and control the flow with floc.Control.

Cancel(), Complete(), Fail() methods of floc.Flow has permanent effect. Once finished flow cannot be canceled or completed anymore. Calling Fail and returning error from job is almost equal.

func ValidateContentLength(ctx floc.Context, ctrl floc.Control) error {
  request := ctx.Value("request").(http.Request)

  // Cancel the flow with error if request body size is too big
  if request.ContentLength > MaxContentLength {
    return errors.New("content is too big")
  }
  
  return nil
}

Example

Lets have some fun and write a simple example which calculates some statistics on text given.

const Text = `Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed
  do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim
  veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
  consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum
  dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident,
  sunt in culpa qui officia deserunt mollit anim id est laborum.`
  
const keyStatistics = 1

var sanitizeWordRe = regexp.MustCompile(`\W`)

type Statistics struct {
  Words      []string
  Characters int
  Occurrence map[string]int
}

// Split to words and sanitize them
SplitToWords := func(ctx floc.Context, ctrl floc.Control) error {
  statistics := ctx.Value(keyStatistics).(*Statistics)

  statistics.Words = strings.Split(Text, " ")
  for i, word := range statistics.Words {
    statistics.Words[i] = sanitizeWordRe.ReplaceAllString(word, "")
  }
  
  return nil
}

// Count and sum the number of characters in the each word
CountCharacters := func(ctx floc.Context, ctrl floc.Control) error {
  statistics := ctx.Value(keyStatistics).(*Statistics)

  for _, word := range statistics.Words {
    statistics.Characters += len(word)
  }
  
  return nil
}

// Count the number of unique words
CountUniqueWords := func(ctx floc.Context, ctrl floc.Control) error {
  statistics := ctx.Value(keyStatistics).(*Statistics)

  statistics.Occurrence = make(map[string]int)
  for _, word := range statistics.Words {
    statistics.Occurrence[word] = statistics.Occurrence[word] + 1
  }
  
  return nil
}

// Print result
PrintResult := func(ctx floc.Context, ctrl floc.Control) error {
  statistics := ctx.Value(keyStatistics).(*Statistics)

  fmt.Printf("Words Total       : %d\n", len(statistics.Words))
  fmt.Printf("Unique Word Count : %d\n", len(statistics.Occurrence))
  fmt.Printf("Character Count   : %d\n", statistics.Characters)
  
  return nil
}

// Design the flow and run it
flow := run.Sequence(
  SplitToWords,
  run.Parallel(
    CountCharacters,
    CountUniqueWords,
  ),
  PrintResult,
)

ctx := floc.NewContext()
ctx.AddValue(keyStatistics, new(Statistics))

ctrl := floc.NewControl(ctx)

_, _, err := floc.RunWith(ctx, ctrl, flow)
if err != nil {
	panic(err)
}

// Output:
// Words Total       : 64
// Unique Word Count : 60
// Character Count   : 370

Contributing

Please found information about contributing in CONTRIBUTING.md and the list of bravers who spent their priceless time and effort to make the project better in CONTRIBUTORS.md.

Issues
  • First - new architectural block

    First - new architectural block

    I created additional flow architectural block - First It similar to Parallel, except it waits only first completed or finished job

    enhancement 
    opened by devishot 4
  • Add a Gitter chat badge to README.md

    Add a Gitter chat badge to README.md

    workanator/go-floc now has a Chat Room on Gitter

    @workanator has just created a chat room. You can visit it here: https://gitter.im/go-floc/Lobby.

    This pull-request adds this badge to your README.md:

    Gitter

    If my aim is a little off, please let me know.

    Happy chatting.

    PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

    opened by gitter-badger 1
  • Change module version to v3

    Change module version to v3

    opened by workanator 0
  • Fix import path in sub-packages

    Fix import path in sub-packages

    opened by workanator 0
  • 🐛 bug fixes

    🐛 bug fixes

    Screenshot 2021-08-10 at 12 16 12 PM Screenshot 2021-08-10 at 12 12 38 PM

    We have several issue on the v3 released

    • Checksum mismatch (after investigation, this is probably due to some breaking changes on go 1.11.3 and go 1.11.4)
    • Improper test case (test doesn't handle correctly, the routine will escape the test result)

    Basically, my PR will fixed the checksum mismatch and test case issue, this had been tested and proved to work.

    opened by si3nloong 0
  • Apply changes from main

    Apply changes from main

    null

    opened by workanator 0
Owner
Andrew Bashkatov
Andrew Bashkatov
Minimalistic and High-performance goroutine worker pool written in Go

pond Minimalistic and High-performance goroutine worker pool written in Go Motivation This library is meant to provide a simple way to limit concurren

Alejandro Durante 268 Sep 6, 2021
gpool - a generic context-aware resizable goroutines pool to bound concurrency based on semaphore.

gpool - a generic context-aware resizable goroutines pool to bound concurrency. Installation $ go get github.com/sherifabdlnaby/gpool import "github.c

Sherif Abdel-Naby 81 Aug 20, 2021
Limits the number of goroutines that are allowed to run concurrently

Golang Concurrency Manager Golang Concurrency Manager package limits the number of goroutines that are allowed to run concurrently. Installation Run t

Bui Thang 25 Aug 30, 2021
A goroutine pool for Go

Tunny is a Golang library for spawning and managing a goroutine pool, allowing you to limit work coming from any number of goroutines with a synchrono

Ashley Jeffs 2.6k Sep 14, 2021
CyclicBarrier golang implementation

cyclicbarrier CyclicBarrier is a synchronizer that allows a set of goroutines to wait for each other to reach a common execution point, also called a

Bator Tsyrendylykov 74 Sep 5, 2021
A sync.WaitGroup with error handling and concurrency control

go-waitgroup How to use An package that allows you to use the constructs of a sync.WaitGroup to create a pool of goroutines and control the concurrenc

Pieter Claerhout 20 Jun 19, 2021
errgroup with goroutine worker limits

neilotoole/errgroup neilotoole/errgroup is a drop-in alternative to Go's wonderful sync/errgroup but limited to N goroutines. This is useful for inter

Neil O'Toole 90 Sep 5, 2021
Reactive Extensions for the Go language.

RxGo Reactive Extensions for the Go Language ReactiveX ReactiveX, or Rx for short, is an API for programming with Observable streams. This is the offi

ReactiveX 3.7k Sep 10, 2021
Structured Concurrency in Go

nursery: structured concurrency in Go RunConcurrently( // Job 1 func(context.Context, chan error) { time.Sleep(time.Millisecond * 10)

null 34 Aug 30, 2021
Simply way to control goroutines execution order based on dependencies

Goflow Goflow is a simply package to control goroutines execution order based on dependencies. It works similar to async.auto from node.js async packa

Kamil Drazkiewicz 172 Sep 1, 2021
🐝 A Highly Performant and easy to use goroutine pool for Go

gohive Package gohive implements a simple and easy to use goroutine pool for Go Features Pool can be created with a specific size as per the requireme

Lovelesh 13 Jul 13, 2021
Run functions in parallel :comet:

Parallel fn Run functions in parallel. Limit the number of goroutines running at the same time. Installation go get -u github.com/rafaeljesus/parallel

Rafael Jesus 31 Aug 3, 2021
:speedboat: a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation

Package pool Package pool implements a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation. Features

Go Playgound 634 Sep 5, 2021
🚦 Semaphore pattern implementation with timeout of lock/unlock operations.

?? semaphore Semaphore pattern implementation with timeout of lock/unlock operations. ?? Idea The semaphore provides API to control access to a shared

Kamil Samigullin 88 Sep 15, 2021
parallel: a Go Parallel Processing Library

parallel: a Go Parallel Processing Library Concurrency is hard. This library doesn't aim to make it easy, but it will hopefully make it a little less

Ryan Skidmore 29 Feb 14, 2021
Hunch provides functions like: All, First, Retry, Waterfall etc., that makes asynchronous flow control more intuitive.

Hunch Hunch provides functions like: All, First, Retry, Waterfall etc., that makes asynchronous flow control more intuitive. About Hunch Go have sever

null 56 Jul 18, 2021
Alternative sync library for Go

Alternative sync library for Go. Overview Future - A placeholder object for a value that may not yet exist. Promise - While futures are defined as a t

Eugene R. 24 Aug 4, 2021
🚧 Flexible mechanism to make execution flow interruptible.

?? breaker Flexible mechanism to make execution flow interruptible. ?? Idea The breaker carries a cancellation signal to interrupt an action execution

Kamil Samigullin 0 Jul 11, 2021
Fast resizable golang semaphore primitive

semaphore Fast resizable golang semaphore based on CAS allows weighted acquire/release; supports cancellation via context; allows change semaphore lim

Bator Tsyrendylykov 122 Aug 20, 2021