Queue is a Golang library for spawning and managing a Goroutine pool

Related tags

Goroutines queue
Overview

Queue

Run Tests codecov

Queue is a Golang library for spawning and managing a Goroutine pool, Alloowing you to create multiple worker according to limit CPU number of machine.

Features

  • Support buffered channel queue.
  • Support NSQ (A realtime distributed messaging platform) as backend.
  • Support NATS (Connective Technology for Adaptive Edge & Distributed Systems) as backend.

Installation

Install the stable version:

go get github.com/golang-queue/queue

Install the latest verison:

go get github.com/golang-queue/[email protected]

Usage

Basic usage of Pool (use Task function)

By calling QueueTask() method, it schedules the task executed by worker (goroutines) in the Pool.

package main

import (
  "context"
  "fmt"
  "time"

  "github.com/golang-queue/queue"
)

func main() {
  taskN := 100
  rets := make(chan string, taskN)

  // initial queue pool
  q := queue.NewPool(5)
  // shutdown the service and notify all the worker
  // wait all jobs are complete.
  defer q.Release()

  // assign tasks in queue
  for i := 0; i < taskN; i++ {
    go func(i int) {
      if err := q.QueueTask(func(ctx context.Context) error {
        rets <- fmt.Sprintf("Hi Gopher, handle the job: %02d", +i)
        return nil
      }); err != nil {
        panic(err)
      }
    }(i)
  }

  // wait until all tasks done
  for i := 0; i < taskN; i++ {
    fmt.Println("message:", <-rets)
    time.Sleep(20 * time.Millisecond)
  }
}

Basic usage of Pool (use message queue)

Define the new message struct and implement the Bytes() func to encode message. Give the WithFn func to handle the message from Queue.

package main

import (
  "context"
  "encoding/json"
  "fmt"
  "log"
  "time"

  "github.com/golang-queue/queue"
)

type job struct {
  Name    string
  Message string
}

func (j *job) Bytes() []byte {
  b, err := json.Marshal(j)
  if err != nil {
    panic(err)
  }
  return b
}

func main() {
  taskN := 100
  rets := make(chan string, taskN)

  // initial queue pool
  q := queue.NewPool(5, queue.WithFn(func(ctx context.Context, m queue.QueuedMessage) error {
    v, ok := m.(*job)
    if !ok {
      if err := json.Unmarshal(m.Bytes(), &v); err != nil {
        return err
      }
    }

    rets <- "Hi, " + v.Name + ", " + v.Message
    return nil
  }))
  // shutdown the service and notify all the worker
  // wait all jobs are complete.
  defer q.Release()

  // assign tasks in queue
  for i := 0; i < taskN; i++ {
    go func(i int) {
      if err := q.Queue(&job{
        Name:    "Gopher",
        Message: fmt.Sprintf("handle the job: %d", i+1),
      }); err != nil {
        log.Println(err)
      }
    }(i)
  }

  // wait until all tasks done
  for i := 0; i < taskN; i++ {
    fmt.Println("message:", <-rets)
    time.Sleep(50 * time.Millisecond)
  }
}
Issues
  • chore(deps): bump github.com/stretchr/testify from 1.7.3 to 1.7.5

    chore(deps): bump github.com/stretchr/testify from 1.7.3 to 1.7.5

    Bumps github.com/stretchr/testify from 1.7.3 to 1.7.5.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 1
  • chore(deps): bump github.com/stretchr/testify from 1.7.1 to 1.7.2

    chore(deps): bump github.com/stretchr/testify from 1.7.1 to 1.7.2

    Bumps github.com/stretchr/testify from 1.7.1 to 1.7.2.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 1
  • test(benchmark): add queue performance check

    test(benchmark): add queue performance check

    Before

    49894bf

    goos: linux
    goarch: amd64
    pkg: github.com/golang-queue/queue
    cpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
    BenchmarkQueueTask
    INFO: 2022/05/29 04:18:27 logger.go:35: shutdown all tasks: 5 workers
    BenchmarkQueueTask-2   	     584	   2079968 ns/op	    1801 B/op	      30 allocs/op
    BenchmarkQueue
    INFO: 2022/05/29 04:18:28 logger.go:35: shutdown all tasks: 5 workers
    INFO: 2022/05/29 04:18:28 logger.go:35: shutdown all tasks: 5 workers
    INFO: 2022/05/29 04:18:29 logger.go:35: shutdown all tasks: 5 workers
    BenchmarkQueue-2       	 1892185	       624.9 ns/op	     214 B/op	       5 allocs/op
    PASS
    ok  	github.com/golang-queue/queue	2.924s
    

    5d2819e

    goos: linux
    goarch: amd64
    pkg: github.com/golang-queue/queue
    cpu: Intel(R) Xeon(R) Platinum 8272CL CPU @ 2.60GHz
    BenchmarkQueueTask
    BenchmarkQueueTask-2   	     582	   2055769 ns/op	    1403 B/op	      22 allocs/op
    BenchmarkQueue
    INFO: 2022/05/29 06:47:35 logger.go:35: shutdown all tasks: 4 workers
    INFO: 2022/05/29 06:47:36 logger.go:35: shutdown all tasks: 5 workers
    INFO: 2022/05/29 06:47:37 logger.go:35: shutdown all tasks: 4 workers
    BenchmarkQueue-2       	 1996585	       616.1 ns/op	     212 B/op	       5 allocs/op
    PASS
    ok  	github.com/golang-queue/queue	2.932s
    

    15c43ec remove wait time in benchmark

    After

    goos: linux
    goarch: amd64
    pkg: github.com/golang-queue/queue
    cpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
    BenchmarkQueueTask
    INFO: 2022/05/29 14:53:36 logger.go:35: shutdown all tasks: 5 workers
    INFO: 2022/05/29 14:53:36 logger.go:35: shutdown all tasks: 4 workers
    INFO: 2022/05/29 14:53:37 logger.go:35: shutdown all tasks: 5 workers
    INFO: 2022/05/29 14:53:38 logger.go:35: shutdown all tasks: 5 workers
    BenchmarkQueueTask-2   	14850574	        86.25 ns/op	      54 B/op	       1 allocs/op
    BenchmarkQueue
    INFO: 2022/05/29 14:53:38 logger.go:35: shutdown all tasks: 5 workers
    INFO: 2022/05/29 14:53:38 logger.go:35: shutdown all tasks: 5 workers
    INFO: 2022/05/29 14:53:39 logger.go:35: shutdown all tasks: 5 workers
    INFO: 2022/05/29 14:53:40 logger.go:35: shutdown all tasks: 5 workers
    BenchmarkQueue-2       	 2270523	       484.3 ns/op	     191 B/op	       4 allocs/op
    PASS
    ok  	github.com/golang-queue/queue	3.738s
    
    enhancement 
    opened by appleboy 1
  • chore(deps): bump goreleaser/goreleaser-action from 2 to 3

    chore(deps): bump goreleaser/goreleaser-action from 2 to 3

    Bumps goreleaser/goreleaser-action from 2 to 3.

    Release notes

    Sourced from goreleaser/goreleaser-action's releases.

    v3.0.0

    What's Changed

    New Contributors

    Full Changelog: https://github.com/goreleaser/goreleaser-action/compare/v2.9.1...v3.0.0

    v2.9.1

    What's Changed

    Full Changelog: https://github.com/goreleaser/goreleaser-action/compare/v2...v2.9.1

    v2.9.0

    What's Changed

    Full Changelog: https://github.com/goreleaser/goreleaser-action/compare/v2.8.1...v2.9.0

    v2.8.1

    What's Changed

    Full Changelog: https://github.com/goreleaser/goreleaser-action/compare/v2.8.0...v2.8.1

    v2.8.0

    What's Changed

    ... (truncated)

    Commits
    • 68acf3b chore(deps): bump @​actions/tool-cache from 1.7.2 to 2.0.1 (#355)
    • 46da113 chore: node 16 as default runtime (#343)
    • 223909a chore: update
    • c56d8df Revert "chore(deps): bump @​actions/core from 1.6.0 to 1.8.2 (#354)"
    • d1c2f83 chore(deps): bump @​actions/core from 1.6.0 to 1.8.2 (#354)
    • 5c65fd8 chore(deps): bump @​actions/http-client from 1.0.11 to 2.0.1 (#353)
    • 46cd12b chore(deps): bump yargs from 17.4.1 to 17.5.1 (#352)
    • 822d1bf chore(deps): bump docker/bake-action from 1 to 2 (#346)
    • c25888f chore: update dev dependencies and workflow (#342)
    • ec57748 chore(deps): bump yargs from 17.4.0 to 17.4.1 (#339)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies github_actions 
    opened by dependabot[bot] 1
  • chore(deps): bump github/codeql-action from 1 to 2

    chore(deps): bump github/codeql-action from 1 to 2

    Bumps github/codeql-action from 1 to 2.

    Changelog

    Sourced from github/codeql-action's changelog.

    2.1.8 - 08 Apr 2022

    • Update default CodeQL bundle version to 2.8.5. #1014
    • Fix error where the init action would fail due to a GitHub API request that was taking too long to complete #1025

    2.1.7 - 05 Apr 2022

    • A bug where additional queries specified in the workflow file would sometimes not be respected has been fixed. #1018

    2.1.6 - 30 Mar 2022

    • [v2+ only] The CodeQL Action now runs on Node.js v16. #1000
    • Update default CodeQL bundle version to 2.8.4. #990
    • Fix a bug where an invalid commit_oid was being sent to code scanning when a custom checkout path was being used. #956
    Commits
    • 2c03704 Allow the version of the ML-powered pack to depend on the CLI version
    • dd6b592 Simplify ML-powered query status report definition
    • a90d8bf Merge pull request #1011 from github/henrymercer/ml-powered-queries-pr-check
    • dc0338e Use latest major version of actions/upload-artifact
    • 57096fe Add a PR check to validate that ML-powered queries are run correctly
    • b0ddf36 Merge pull request #1012 from github/henrymercer/update-actions-major-versions
    • 1ea2f2d Merge branch 'main' into henrymercer/update-actions-major-versions
    • 9dcc141 Merge pull request #1010 from github/henrymercer/stop-running-ml-powered-quer...
    • ea751a9 Update other Actions from v2 to v3
    • a2949f4 Update actions/checkout from v2 to v3
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies github_actions 
    opened by dependabot[bot] 1
  • chore(deps): bump codecov/codecov-action from 2 to 3

    chore(deps): bump codecov/codecov-action from 2 to 3

    Bumps codecov/codecov-action from 2 to 3.

    Release notes

    Sourced from codecov/codecov-action's releases.

    v3.0.0

    Breaking Changes

    • #689 Bump to node16 and small fixes

    Features

    • #688 Incorporate gcov arguments for the Codecov uploader

    Dependencies

    • #548 build(deps-dev): bump jest-junit from 12.2.0 to 13.0.0
    • #603 [Snyk] Upgrade @​actions/core from 1.5.0 to 1.6.0
    • #628 build(deps): bump node-fetch from 2.6.1 to 3.1.1
    • #634 build(deps): bump node-fetch from 3.1.1 to 3.2.0
    • #636 build(deps): bump openpgp from 5.0.1 to 5.1.0
    • #652 build(deps-dev): bump @​vercel/ncc from 0.30.0 to 0.33.3
    • #653 build(deps-dev): bump @​types/node from 16.11.21 to 17.0.18
    • #659 build(deps-dev): bump @​types/jest from 27.4.0 to 27.4.1
    • #667 build(deps): bump actions/checkout from 2 to 3
    • #673 build(deps): bump node-fetch from 3.2.0 to 3.2.3
    • #683 build(deps): bump minimist from 1.2.5 to 1.2.6
    • #685 build(deps): bump @​actions/github from 5.0.0 to 5.0.1
    • #681 build(deps-dev): bump @​types/node from 17.0.18 to 17.0.23
    • #682 build(deps-dev): bump typescript from 4.5.5 to 4.6.3
    • #676 build(deps): bump @​actions/exec from 1.1.0 to 1.1.1
    • #675 build(deps): bump openpgp from 5.1.0 to 5.2.1

    v2.1.0

    2.1.0

    Features

    • #515 Allow specifying version of Codecov uploader

    Dependencies

    • #499 build(deps-dev): bump @​vercel/ncc from 0.29.0 to 0.30.0
    • #508 build(deps): bump openpgp from 5.0.0-5 to 5.0.0
    • #514 build(deps-dev): bump @​types/node from 16.6.0 to 16.9.0

    v2.0.3

    2.0.3

    Fixes

    • #464 Fix wrong link in the readme
    • #485 fix: Add override OS and linux default to platform

    Dependencies

    • #447 build(deps): bump openpgp from 5.0.0-4 to 5.0.0-5
    • #458 build(deps-dev): bump eslint from 7.31.0 to 7.32.0
    • #465 build(deps-dev): bump @​typescript-eslint/eslint-plugin from 4.28.4 to 4.29.1
    • #466 build(deps-dev): bump @​typescript-eslint/parser from 4.28.4 to 4.29.1
    • #468 build(deps-dev): bump @​types/jest from 26.0.24 to 27.0.0
    • #470 build(deps-dev): bump @​types/node from 16.4.0 to 16.6.0
    • #472 build(deps): bump path-parse from 1.0.6 to 1.0.7
    • #473 build(deps-dev): bump @​types/jest from 27.0.0 to 27.0.1

    ... (truncated)

    Changelog

    Sourced from codecov/codecov-action's changelog.

    3.1.0

    Features

    • #699 Incorporate xcode arguments for the Codecov uploader

    Dependencies

    • #694 build(deps-dev): bump @​vercel/ncc from 0.33.3 to 0.33.4
    • #696 build(deps-dev): bump @​types/node from 17.0.23 to 17.0.25
    • #698 build(deps-dev): bump jest-junit from 13.0.0 to 13.2.0

    3.0.0

    Breaking Changes

    • #689 Bump to node16 and small fixes

    Features

    • #688 Incorporate gcov arguments for the Codecov uploader

    Dependencies

    • #548 build(deps-dev): bump jest-junit from 12.2.0 to 13.0.0
    • #603 [Snyk] Upgrade @​actions/core from 1.5.0 to 1.6.0
    • #628 build(deps): bump node-fetch from 2.6.1 to 3.1.1
    • #634 build(deps): bump node-fetch from 3.1.1 to 3.2.0
    • #636 build(deps): bump openpgp from 5.0.1 to 5.1.0
    • #652 build(deps-dev): bump @​vercel/ncc from 0.30.0 to 0.33.3
    • #653 build(deps-dev): bump @​types/node from 16.11.21 to 17.0.18
    • #659 build(deps-dev): bump @​types/jest from 27.4.0 to 27.4.1
    • #667 build(deps): bump actions/checkout from 2 to 3
    • #673 build(deps): bump node-fetch from 3.2.0 to 3.2.3
    • #683 build(deps): bump minimist from 1.2.5 to 1.2.6
    • #685 build(deps): bump @​actions/github from 5.0.0 to 5.0.1
    • #681 build(deps-dev): bump @​types/node from 17.0.18 to 17.0.23
    • #682 build(deps-dev): bump typescript from 4.5.5 to 4.6.3
    • #676 build(deps): bump @​actions/exec from 1.1.0 to 1.1.1
    • #675 build(deps): bump openpgp from 5.1.0 to 5.2.1

    2.1.0

    Features

    • #515 Allow specifying version of Codecov uploader

    Dependencies

    • #499 build(deps-dev): bump @​vercel/ncc from 0.29.0 to 0.30.0
    • #508 build(deps): bump openpgp from 5.0.0-5 to 5.0.0
    • #514 build(deps-dev): bump @​types/node from 16.6.0 to 16.9.0

    2.0.3

    Fixes

    • #464 Fix wrong link in the readme
    • #485 fix: Add override OS and linux default to platform

    Dependencies

    • #447 build(deps): bump openpgp from 5.0.0-4 to 5.0.0-5

    ... (truncated)

    Commits
    • 81cd2dc Merge pull request #699 from codecov/feat-xcode
    • a03184e feat: add xcode support
    • 6a6a9ae Merge pull request #694 from codecov/dependabot/npm_and_yarn/vercel/ncc-0.33.4
    • 92a872a Merge pull request #696 from codecov/dependabot/npm_and_yarn/types/node-17.0.25
    • 43a9c18 Merge pull request #698 from codecov/dependabot/npm_and_yarn/jest-junit-13.2.0
    • 13ce822 Merge pull request #690 from codecov/ci-v3
    • 4d6dbaa build(deps-dev): bump jest-junit from 13.0.0 to 13.2.0
    • 98f0f19 build(deps-dev): bump @​types/node from 17.0.23 to 17.0.25
    • d3021d9 build(deps-dev): bump @​vercel/ncc from 0.33.3 to 0.33.4
    • 2c83f35 Update makefile to v3
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies github_actions 
    opened by dependabot[bot] 1
  • chore(deps): bump actions/cache from 2 to 3

    chore(deps): bump actions/cache from 2 to 3

    Bumps actions/cache from 2 to 3.

    Release notes

    Sourced from actions/cache's releases.

    v3.0.0

    • This change adds a minimum runner version(node12 -> node16), which can break users using an out-of-date/fork of the runner. This would be most commonly affecting users on GHES 3.3 or before, as those runners do not support node16 actions and they can use actions from github.com via github connect or manually copying the repo to their GHES instance.

    • Few dependencies and cache action usage examples have also been updated.

    v2.1.7

    Support 10GB cache upload using the latest version 1.0.8 of @actions/cache

    v2.1.6

    • Catch unhandled "bad file descriptor" errors that sometimes occurs when the cache server returns non-successful response (actions/cache#596)

    v2.1.5

    • Fix permissions error seen when extracting caches with GNU tar that were previously created using BSD tar (actions/cache#527)

    v2.1.4

    • Make caching more verbose #650
    • Use GNU tar on macOS if available #701

    v2.1.3

    • Upgrades @actions/core to v1.2.6 for CVE-2020-15228. This action was not using the affected methods.
    • Fix error handling in uploadChunk where 400-level errors were not being detected and handled correctly

    v2.1.2

    • Adds input to limit the chunk upload size, useful for self-hosted runners with slower upload speeds
    • No-op when executing on GHES

    v2.1.1

    • Update @actions/cache package to v1.0.2 which allows cache action to use posix format when taring files.

    v2.1.0

    • Replaces the http-client with the Azure Storage SDK for NodeJS when downloading cache content from Azure. This should help improve download performance and reliability as the SDK downloads files in 4 MB chunks, which can be parallelized and retried independently
    • Display download progress and speed
    Changelog

    Sourced from actions/cache's changelog.

    Releases

    3.0.0

    • Updated minimum runner version support from node 12 -> node 16

    3.0.1

    • Added support for caching from GHES 3.5.
    • Fixed download issue for files > 2GB during restore.
    Commits
    • 48af2dc Update actions/cache version (#786)
    • f63a711 Merge pull request #781 from actions/t-dedah/cacheSize
    • 770a27c Update licenses
    • c2131ab New build
    • 5751604 Updated @​actions/cache to 2.0.1
    • 136d96b Enabling actions/cache for GHES based on presence of AC service (#774)
    • 7d4f40b Bumping up the version to fix download issue for files > 2 GB. (#775)
    • 2d8d0d1 Updated what's new. (#771)
    • 7799d86 Updated the usage and docs to the major version release. (#770)
    • 4b0cf6c Merge pull request #769 from actions/users/ashwinsangem/bump_major_version
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies github_actions 
    opened by dependabot[bot] 1
  • fix(worker): don't start request task

    fix(worker): don't start request task

    check worker count is zero or not before fetch the task.

    fix https://github.com/golang-queue/queue/issues/59

    Signed-off-by: Bo-Yi Wu [email protected]

    opened by appleboy 1
  • feat(metric): add success, failure and submitted tasks

    feat(metric): add success, failure and submitted tasks

    type metric struct {
    	busyWorkers    uint64
    	successTasks   uint64
    	failureTasks   uint64
    	submittedTasks uint64
    }
    

    fix https://github.com/golang-queue/queue/issues/47

    Signed-off-by: Bo-Yi Wu [email protected]

    enhancement 
    opened by appleboy 1
  • chore(consumer): performance tinning

    chore(consumer): performance tinning

    BenchmarkConsumerRunUnmarshal
    BenchmarkConsumerRunUnmarshal-2   	  494060	      2644 ns/op	     624 B/op	      11 allocs/op
    BenchmarkConsumerRunTask
    BenchmarkConsumerRunTask-2        	  631615	      2368 ns/op	     592 B/op	      10 allocs/op
    PASS
    ok  	github.com/golang-queue/queue	6.979s
    
    opened by appleboy 1
Releases(v0.1.3)
  • v0.1.3(Jun 24, 2022)

    Changelog

    Enhancements

    • c420418 chore(benchmark): improve add task in queue performance (#72)
    • 063bb63 chore(benchmark): move testing to new file.
    • 68f9e46 chore(deps): bump github.com/stretchr/testify from 1.7.1 to 1.7.2 (#74)
    • 00c8a31 chore(deps): bump github.com/stretchr/testify from 1.7.2 to 1.7.3 (#75)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.2(May 28, 2022)

    Changelog

    Bug fixes

    • b3dd500 fix(error): missing panic error message (#71)

    Enhancements

    • 367a680 chore(queue): add example testing. (#70)

    Others

    • e1a9717 test(queue): add submitted task count
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Apr 29, 2022)

  • v0.1.0(Apr 26, 2022)

    Changelog

    Features

    • 1a06aab feat(consumer): add default timeout in Request new task (#58)
    • bfe4374 feat(metric): add success, failure and submitted tasks (#48)
    • 3404d97 feat(mock): support mock testing (#62)
    • 18f5bb4 feat(queue): all task has been done after closing the channel (#56)
    • f2b7091 feat(queue): auto scale the task worker (#44)
    • f4bc970 feat(worker): update node count dynamically (#52)

    Bug fixes

    • 8d75d3e fix(worker): don't start request task (#60)
    • fe0fb0e fix(worker): handle all jobs with graceful shutdown (#54)

    Enhancements

    • fd4ed0b chore(comsumer): remove Usage and Capacity
    • 94512c6 chore(consumer): remove metrics (#51)
    • 965207c chore(deps): bump actions/cache from 2 to 3 (#64)
    • f7ab872 chore(deps): bump codecov/codecov-action from 2 to 3 (#65)
    • 003e160 chore(deps): bump github/codeql-action from 1 to 2 (#66)
    • c4a8df5 chore(github): upgrade some action package
    • d431277 chore(go): upgrade go module to go1.18 version
    • 03d90b7 chore(queue): don't prepare request data before worker done. (#63)
    • d1132ff chore(testing): remove empty worker.
    • 8af744c chore(worker): remove BeforeRun and AfterRun interface (#46)
    • 6ea7d46 chore: actions/checkout from 2 to 3

    Others

    • 98c9f44 chore(CI/CD): add dependabot
    • 4c2d646 chore(CI/CD): add go1.18 version
    • ff5edbc chore(CI/CD): bump golangci-lint to v1.45.0
    • ed24fa1 refactor(options): add new interface and new Apply method.
    • 7af73e0 refactor: create task worker testing
    Source code(tar.gz)
    Source code(zip)
  • v0.0.12(Mar 7, 2022)

    Changelog

    Bug fixes

    • 90e93fd fix(lint): add comments.

    Others

    • a69c1e0 chore(CI): add codeql
    • 5196cb8 chore(CI): setup go version to ^1
    • c3c10a5 chore(golangci): update run time
    • 50ef568 chore(golangci): upgrade to v1.44
    • 478c3ae chore(golangci): upgrade to v1.44.2
    • 600f61f chore(queue): update body to Payload
    • 56600ec chore(queue): update member
    • b1a4def chore(release): add goreleaser config
    • 2c41650 chore(style): add empty line
    • ab1bb24 chore: skip build binary after create new version
    • dae8274 chore: update go module
    • 3bfe1d0 chore: update go module
    Source code(tar.gz)
    Source code(zip)
  • v0.0.10(Dec 24, 2021)

Owner
golang-queue
golang-queue
:speedboat: a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation

Package pool Package pool implements a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation. Features

Go Playgound 687 Jun 20, 2022
🐜🐜🐜 ants is a high-performance and low-cost goroutine pool in Go, inspired by fasthttp./ ants 是一个高性能且低损耗的 goroutine 池。

A goroutine pool for Go English | ???? 中文 ?? Introduction Library ants implements a goroutine pool with fixed capacity, managing and recycling a massi

Andy Pan 8.6k Jun 26, 2022
golang worker pool , Concurrency limiting goroutine pool

golang worker pool 中文说明 Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks su

xxj 398 Jun 21, 2022
Unlimited job queue for go, using a pool of concurrent workers processing the job queue entries

kyoo: A Go library providing an unlimited job queue and concurrent worker pools About kyoo is the phonetic transcription of the word queue. It provide

Dirk Pahl 38 Jun 8, 2022
goroutine pool in golang

goroutine pool in golang

wksw 1 Nov 1, 2021
🐝 A Highly Performant and easy to use goroutine pool for Go

gohive Package gohive implements a simple and easy to use goroutine pool for Go Features Pool can be created with a specific size as per the requireme

Lovelesh 36 Jun 21, 2022
Minimalistic and High-performance goroutine worker pool written in Go

pond Minimalistic and High-performance goroutine worker pool written in Go Motivation This library is meant to provide a simple way to limit concurren

Alejandro Durante 557 Jun 22, 2022
Lightweight Goroutine pool

grpool Lightweight Goroutine pool Clients can submit jobs. Dispatcher takes job, and sends it to first available worker. When worker is done with proc

Ivan Pusic 694 Jun 16, 2022
A goroutine pool for Go

Tunny is a Golang library for spawning and managing a goroutine pool, allowing you to limit work coming from any number of goroutines with a synchrono

Ashley Jeffs 3.1k Jun 22, 2022
Concurrency limiting goroutine pool

workerpool Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting task

Andrew Gillis 841 Jun 24, 2022
Golang Implementation of Worker Pool/ Thread Pool

Golang Implementation of Worker Pool/ Thread Pool

Telkom DEV 1 Jun 18, 2022
Go-miningcore-pool - Miningcore Pool written in GOlang

Go-Miningcore-Pool (COMING SOON) Miningcore Pool written in GOlang 0x01 Configur

miningcore.com 2 Apr 24, 2022
Go-ldap-pool - A simple connection pool for go-ldap

Basic connection pool for go-ldap This little library use the go-ldap library an

Vincent 2 May 9, 2022
Work pool channlege - An url hash retriever worker pool for getting hash digest for a collection of urls

Code challenge The aim of this project is to provide an url hash retriever worke

null 0 Feb 16, 2022
A simple and useful goroutine concurrent library.

Taskgroup A simple and useful goroutine concurrent library. Installation go get github.com/anthhub/taskgroup

Tangqy 4 May 19, 2021
Provides some convenient API, includes Goid(), AllGoid(), and LocalStorage, which is a goroutine's local storage, just like ThreadLocal in other languages.

routine 中文版 routine encapsulates and provides some easy-to-use, high-performance goroutine context access interfaces, which can help you access corout

null 58 Jun 27, 2022
A cross goroutine storage tool with very simple implementation and function.

Simple-goroutine-local is a cross goroutine storage tool with very simple implementation and function (the concept is similar to Java ThreadLocal). Ge

null 0 Jan 13, 2022
errgroup with goroutine worker limits

neilotoole/errgroup neilotoole/errgroup is a drop-in alternative to Go's wonderful sync/errgroup but limited to N goroutines. This is useful for inter

Neil O'Toole 126 Jun 21, 2022
Waiting group for collecting goroutine information.

在go语言waitGroup和errGroup都是用来控制goroutine的并发的方式,前者只能等待所有goroutine执行完成之后再执行Wait()函数后面的代码并且不

Jarvib Ding 111 Dec 3, 2021