Resiliency patterns for golang

Overview

go-resiliency

Build Status GoDoc Code of Conduct

Resiliency patterns for golang. Based in part on Hystrix, Semian, and others.

Currently implemented patterns include:

  • circuit-breaker (in the breaker directory)
  • semaphore (in the semaphore directory)
  • deadline/timeout (in the deadline directory)
  • batching (in the batcher directory)
  • retriable (in the retrier directory)

Follows semantic versioning using https://gopkg.in/ - import from gopkg.in/eapache/go-resiliency.v1 for guaranteed API stability.

Comments
  • Protect PRNG against simultaneous access. (Was: Add setter for Retrier random number generator)

    Protect PRNG against simultaneous access. (Was: Add setter for Retrier random number generator)

    Thank you for this project.

    This pull request adds SetRand(Float64er) in order to provides ones own random number generator.

    Uses cases include the following:

    • providing an alternate RNG.
    • providing a thread safe RNG.
    opened by gglachant 5
  • Add RunCtx to retrier

    Add RunCtx to retrier

    Thank you very much for this project. I often use retrier package.

    There are three changes

    • Add retrier.RunCtx(ctx context.Context work func(ctx context.Context) error) error.
    • Add test for retrier.RunCtx
    • Modify travis.yml for supporting only 1.7 and 1.10

    fixes #16

    opened by tukeJonny 4
  • retrier: support context.Context

    retrier: support context.Context

    Greetings, I'm a user of the retry package, mostly because it's a dependency of the shopify/sarama package - so it's already available in my vendor directory.

    I'm interested to know if you'd be in favor of supporting the context package for retrier -- I imagine the interface to be

    (r *Retrier) RunCtx(context.Context, func() error) error
    

    This would allow the retrier to exit early if the provided context is cancelled prior to the retry wait period. If the context is cancelled, it could return the error of the last failure, or ctx.Err() - I'm unsure which would be preferred.

    I would imagine the retier struct would need to be extended to include a time.Timer that would be reset upon each call to Run/RunCtx, and each iteration of attempts.

    Exiting early would me a simple matter of having a function such as

    // sleep will return true if the time channel has a message
    // sent on it before the context expires
    //
    // accepts a <-chan time.Time for more deterministic testing
    // no need to rely on the runtime to fire the timer
    func sleep(ctx context.Context, t <-chan time.Time) bool {
        select {
        case <-t:
            return true
        case <-ctx.Done():
            return false
        }
    }
    

    the call to time.Sleep() could then be replaced with

    if !sleep(ctx, r.timer.C) {
        // return previous error?
        return ctx.Err()
    }
    

    The implementation of Run could then become

    (r *Retrier) Run(fn func() error) error {
        return r.RunCtx(context.Background(), fn)
    }
    

    This approach is seen in the standard library, and is a nice way to maintain backwards compatibility.

    Please let me know your thoughts - if this is something you'd approve of, I could likely find the time to do the implementation.

    opened by chrisguiney 3
  • added leaky bucket rate limiter

    added leaky bucket rate limiter

    First off thanks for the awesome project. Really love your clean style.

    I needed a leaky bucket rate limiter to add to a client i was building and thought that this might be useful to this repo.

    This isn't the final commit as we need to add README to package, just wanting to see if this is something you are interested in?

    opened by kidtronnix 3
  • Error unwrapping for `retrier/`

    Error unwrapping for `retrier/`

    Hello, I've some feature request regarding the title

    Basically, I propose we use errors.Is() from standard library to check for the blacklisted/whitelisted error types (the err == pass bit)

    https://github.com/eapache/go-resiliency/blob/b98ce2825f7276cc41027348986a241ef5d62fd8/retrier/classifier.go#L40-L44

    The reason is that currently we need to specify exacly what the error kind is, without checking if an error actually contains ("wraps") another error which are actually listed

    Say we've implements a custom type to better handle specific error usecase

    type HTTPError struct {
        error
        host string
        path string
    }
    
    func (h HTTPError) Error() string { // implements Go's `error` type
        return "failed on HTTP request to " + h.host + h.path + " reason: " + h.error.Error()
    }
    
    func (h HTTPError) Unwrap() error { // will be called from standard library error unwrapping
        return h.error
    }
    

    And that the error type wraps context.Canceled, which is defined as a blacklist

    
    backoff := // ...
    class := retrier.BlacklistClassifier{
    	context.Canceled,
    }
    retry := retrier.New(backoff, class)
    
    // ...
    
    retry.Run(func() {
        // ...
        return HTTPError{error: context.Cancelled, host: "...", path: "..."}
    })
    

    The blacklist check would fail, because err == pass won't know what the "internal" value of the error actually is

    opened by taufik-rama 2
  • add log for breaker error

    add log for breaker error

    I feel that you should add log when some occur in function func (b *Breaker) doWork(state uint32, work func() error) error, it convenient for us to find error

    opened by Stephan14 2
  • Spelling fix: aquire -> acquire

    Spelling fix: aquire -> acquire

    This is to increase the report card score for avelino/awesome-go#1448 so I can add this to the awesome Go list.

    Go report card: https://goreportcard.com/report/github.com/eapache/go-resiliency

    opened by jessedearing 2
  • please tag and version this project

    please tag and version this project

    Hello,

    Can you please tag and version this project?

    I am the Debian Maintainer for go-resiliency and versioning would help Debian keep up with development.

    opened by aviau 2
  • clarify deadline example

    clarify deadline example

    The deadline example says: // check stopper function and give up if timed out What is the stopper function? Stopper is a channel. Perhaps the example can be expanded to show how to do this?

    opened by predmond 2
  • Add LimitedExponentialBackoff

    Add LimitedExponentialBackoff

    @eapache Thank you very much for this project.

    Currently, there is no upper limit for ExponentialBackoff . Therefore, n (number of times) needs to be set so that 2^n is an appropriate value for a back-off.

    However, there are some use-cases where we want to limit the maximum back-off while ensuring a sufficient number of retries. LimitedExponentialBackoff has been added to handle such use-case.

    So, there are two changes.

    • Add LimitedExponentialBackoff
    • Add tests for LimitedExponentialBackoff
    opened by tukeJonny 1
  • Adding code to run on Powersystem

    Adding code to run on Powersystem

    Hi Here is my contribution to your code, its working good on powersystems.

    Thanks for the code, its working good.

    What do these changes do?

    Added Architecture "ppc64le"

    Are there changes in behavior for the user?

    No

    opened by genisysram 1
  • Support service oriented calls for retrier

    Support service oriented calls for retrier

    I like to use retrier to wrap calls to external services one thing that is generating a bit of verbose code right now is that the work functions do not return any value in their signature (totally understandable) .

    The idea here would be to create a new Run/RunCtx signature using generics and make the work function return the type that is passed to the function...

    func (r *Retrier) RunR[R any](work func() (R, error)) error
    
    func (r *Retrier) RunCtxR[R any](ctx context.Context, work func(ctx context.Context) (R, error)) error
    

    This way the caller can just wrap their normal function without having to var the return...

    I understand this would force the library to jump all the way to go 1.18 but just wondering if you would be open to me PRing this..

    I can also try to use some type of interface{} approach so that there is no need to go go 1.18

    opened by eleduardo 1
  • feature request: retrier: reset number of retries/backoff time

    feature request: retrier: reset number of retries/backoff time

    Imagine that I have an application that is an event processor that connects to some event log and for every event that arrives I want to do something with it. Also imagine that I have a Retrier with exponential backoff (100ms initial backoff time) and 5 retries.

    We can get errors because of network issues (server not ready to accept connections, server disconnected for some reason like server LoadBalancer downscaling instances) or errors processing events.

    If the app fails 5 times to process an event the retry loop should end and return error. This is good. If my app fails to connect 5 times in a row the same should happen.

    But if my app connects to the server, happily processes messages for 2hours then gets a server disconnection. It will retry the connection and (if successful) continue processing events. In the next few hours we receive 4 more server disconnections, but after each one we're able to reconnect and continue processing messages for some time. It does not make sense to say "we've failed 5 times so let's just finish here and return an error".

    So basically the Retrier should expose a Reset() method which would cause the retrier to "forget" that it encountered errors before so that the next time that it encounters an error, instead of sleeping for a long time due to exponential backoff it will just sleep for the initial 100ms (as per the example) and it could tolerate 5 more errors later.

    Here's a very simple skeleton of code that shows how this feature could be used:

    r := retrier.New(retrier.ExponentialBackoff(5, 100 * time.Milliseconds), nil)
    err := r.Run(func() error {
      server, err := connect(address)
      if err != nil {
        return err
      }
    
      for {
        event, err := server.getNextEvent()
        if err != nil {
          return err
        }
        err = processEvent(event)
        if err != nil {
          return err
        }
        // Because we've been able to process an event it seems that all is working correctly
        // so it doesn't matter that in the past we had (for example 2 errors), the next time we
        // see an error should be like it was the first error we've seen, and the backoff time to
        // to wait until next try should be the initial one passed by the user.
        r.Reset()
      }
    }
    
    opened by kampde 5
Releases(v1.3.0)
  • v1.3.0(Jun 28, 2022)

    • Increased minimum Golang version to 1.13.
    • Fix a goroutine leak in Deadline.Run() on ErrTimeOut.
    • Add a go.mod file to conform to more recent Golang version standards.
    • Use errors.Is when classifying errors for the Retrier (thanks to Taufik Rama).
    • Add implementation of LimitedExponentialBackoff for the Retrier (thanks to tukeJonny).
    Source code(tar.gz)
    Source code(zip)
  • v1.2.0(Jun 14, 2019)

    Note: This release requires Golang at least 1.7, which is higher than the previous release. All the versions being dropped are multiple years old and no longer supported upstream, so I'm not counting this as a breaking change.

    • Add RunCtx method on Retrier to support running with a context.
    • Ensure the Retrier's use of random numbers is concurrency-safe.
    • Bump CI to ensure we support newer Golang versions.
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Mar 26, 2018)

Owner
Evan Huus
Amateur musician and actor. Trying to end up with the right regrets.
Evan Huus
Belajar Golang Install Golang

Golang belajar Golang Install Golang = download di https://golang.org/dl/ = pilih yg Zip = extract file zipnya = buka foldernya - copy folder go = pas

Arif Fadilah 1 Nov 15, 2021
Golang-module-references - A reference for how to setup a Golang project with modules - Task Management + Math Examples

Golang Module Project The purpose of this project is to act as a reference for setting up future Golang projects using modules. This project has a mat

Bob Bass 0 Jan 2, 2022
Golang-echo-sample - Make an out-of-the-box backend based on golang-echo

Golang-echo-sample - Make an out-of-the-box backend based on golang-echo

Haitwang 0 Dec 31, 2021
Minimalistic, pluggable Golang evloop/timer handler with dependency-injection

Anagent Minimalistic, pluggable Golang evloop/timer handler with dependency-injection - based on codegangsta/inject - go-macaron/inject and chuckpresl

Ettore Di Giacinto 15 Sep 27, 2022
GoLang Library for Browser Capabilities Project

Browser Capabilities GoLang Project PHP has get_browser() function which tells what the user's browser is capable of. You can check original documenta

Maksim N. 43 Sep 27, 2022
Golang counters for readers/writers

Datacounter Golang counters for readers/writers. Examples ReaderCounter buf := bytes.Buffer{} buf.Write(data) counter := datacounter.NewReaderCounter(

Artem Andreenko 44 Oct 9, 2022
Golang beautify data display for Humans

Golang beautify data display for Humans English 简体中文 Install # Stable version go get -u -v gopkg.in/ffmt.v1 # Latest version go get -u -v github.com/

ffmt 281 Nov 10, 2022
a generic object pool for golang

Go Commons Pool The Go Commons Pool is a generic object pool for Golang, direct rewrite from Apache Commons Pool. Features Support custom PooledObject

jolestar 1.1k Nov 20, 2022
psutil for golang

gopsutil: psutil for golang This is a port of psutil (https://github.com/giampaolo/psutil). The challenge is porting all psutil functions on some arch

shirou 8.4k Nov 20, 2022
Type-safe Prometheus metrics builder library for golang

gotoprom A Prometheus metrics builder gotoprom offers an easy to use declarative API with type-safe labels for building and using Prometheus metrics.

Cabify 98 Oct 11, 2022
Simple licensing library for golang.

license-key A simple licensing library in Golang, that generates license files containing arbitrary data. Note that this implementation is quite basic

Hyperboloide 264 Nov 17, 2022
Some utilities for Persian language in Go (Golang)

persian Some utilities for Persian language in Go (Golang). Installation go get github.com/mavihq/persian API .ToPersianDigits Converts all English d

هلو | آموزش زبان با بازی 68 Oct 22, 2022
A Golang library to manipulate strings according to the word parsing rules of the UNIX Bourne shell.

shellwords A Golang library to manipulate strings according to the word parsing rules of the UNIX Bourne shell. Installation go get github.com/Wing924

Wei He 18 Sep 27, 2022
A golang URL Shortener

url-shortener A golang URL Shortener with mysql support. Using Bijective conversion between natural numbers (IDs) and short strings Installation Using

Leonidas Maroulis 39 Nov 17, 2022
:guardsman: A teeny tiny and somewhat opinionated generator for your next golang project

A Yeoman Golang Generator We are very sorry Gophers, but other names for the generator where taken, so we choose go-lang. But we have gocreate as an a

Axel Springer SE 25 Sep 27, 2022
Flow-based and dataflow programming library for Go (golang)

GoFlow - Dataflow and Flow-based programming library for Go (golang) Status of this branch (WIP) Warning: you are currently on v1 branch of GoFlow. v1

Vladimir Sibirov 1.5k Nov 15, 2022
a Go (Golang) MusicBrainz WS2 client library - work in progress

gomusicbrainz a Go (Golang) MusicBrainz WS2 client library - a work in progress. Current state Currently GoMusicBrainz provides methods to perform sea

Michael Wendland 47 Sep 28, 2022
Consistent hashing with bounded loads in Golang

consistent This library provides a consistent hashing function which simultaneously achieves both uniformity and consistency. For detailed information

Burak Sezer 529 Nov 17, 2022
Парсер технологического журнала, основанный на стеке технологий Golang goroutines + Redis + Elasticsearch.

go-techLog1C Парсер технологического журнала, основанный на стеке технологий Golang goroutines + Redis + Elasticsearch. Стек является кросс-платформен

Vadim 26 Nov 6, 2022