A Go queue manager on top of Redis

Related tags

Data Structures redis golang queue
Overview

Queue

Go Lang GoDoc Build Status Coverage Status Go Report Card Gitter

A Go library for managing queues on top of Redis. It is based on a hiring exercise but later I found it useful for myself in a custom task processing project. I thought it might be useful in general.

Installation

$ go get github.com/kavehmz/queue

Usage

package main

import (
	"fmt"
	"time"

	"github.com/kavehmz/queue"
)

func main() {
	var q queue.Queue
	q.Urls([]string{"redis://localhost:6379"})
	q.AddTask(1, "start")
	q.AddTask(2, "start")
	q.AddTask(1, "stop")
	q.AddTask(2, "stop")
	analyzer := func(id int, task chan string, success chan bool) {
		for {
			select {
			case msg := <-task:
				fmt.Println(id, msg)
				if msg == "stop" {
					success <- true
					return
				}
			case <-time.After(2 * time.Second):
				fmt.Println("no new events for 2 seconds for ID", id)
				success <- false
				return
			}
		}
	}
	exitOnEmpty := func() bool {
		return true
	}
	q.AnalysePool(1, exitOnEmpty, analyzer)
}

Approach

Focus of this design is mainly horizontal scalability via concurrency, partitioning and fault-detection.

You might also like...
Fast Raft framework using the Redis protocol for Go
Fast Raft framework using the Redis protocol for Go

This project has been archived. Please check out Uhaha for a fitter, happier, more productive Raft framework. Finn is a fast and simple framework for

Josh Baker 543 Oct 10, 2022
ZSet is an in-memory Redis like sorted set datastructure

zset Getting Started Installing To start using hash, install Go and run go get: $ go get -u github.com/arriqaaq/zset This will retrieve the library. U

Farhan 7 Oct 13, 2022
painless task queue manager for shell commands with an intuitive cli interface (execute shell commands in distributed cloud-native queue manager).

EXEQ DOCS STILL IN PROGRESS. Execute shell commands in queues via cli or http interface. Features Simple intuitive tiny cli app. Modular queue backend

Mohammed Al Ashaal 13 Dec 14, 2022
Redisq is a queue-over-redis that provides simple way to works with queues stored in Redis.

Redisq is a queue-over-redis that provides simple way to works with queues stored in Redis.

null 0 Feb 10, 2022
top in container - Running the original top command in a container
top in container - Running the original top command in a container

Running the original top command in a container will not get information of the container, many metrics like uptime, users, load average, tasks, cpu, memory, are about the host in fact. topic(top in container) will retrieve those metrics from container instead, and shows the status of the container, not the host.

silenceshell 131 Dec 2, 2022
Redis-shake is a tool for synchronizing data between two redis databases. Redis-shake是一个用于在两个redis之间同步数据的工具，满足用户非常灵活的同步、迁移需求。
Redis-shake is a tool for synchronizing data between two redis databases. Redis-shake是一个用于在两个redis之间同步数据的工具，满足用户非常灵活的同步、迁移需求。

RedisShake is mainly used to synchronize data from one redis to another. Thanks to the Douyu's WSD team for the support. 中文文档 English tutorial 中文使用文档

Alibaba 2.9k Dec 29, 2022
Gue is Golang queue on top of PostgreSQL that uses transaction-level locks.

Gue is Golang queue on top of PostgreSQL that uses transaction-level locks.

Vladimir Garvardt 128 Jan 4, 2023
Unlimited job queue for go, using a pool of concurrent workers processing the job queue entries

kyoo: A Go library providing an unlimited job queue and concurrent worker pools About kyoo is the phonetic transcription of the word queue. It provide

Dirk Pahl 39 Dec 21, 2022
Machinery is an asynchronous task queue/job queue based on distributed message passing.
Machinery is an asynchronous task queue/job queue based on distributed message passing.

Machinery Machinery is an asynchronous task queue/job queue based on distributed message passing. V2 Experiment First Steps Configuration Lock Broker

Richard Knop 6.7k Jan 7, 2023
Machinery is an asynchronous task queue/job queue based on distributed message passing.
Machinery is an asynchronous task queue/job queue based on distributed message passing.

Machinery Machinery is an asynchronous task queue/job queue based on distributed message passing. V2 Experiment First Steps Configuration Lock Broker

Richard Knop 6.7k Dec 24, 2022
A Multi Consumer per Message Queue with persistence and Queue Stages.
A Multi Consumer per Message Queue with persistence and Queue Stages.

CrimsonQ A Multi Consumer per Message Queue with persistence and Queue Stages. Under Active Development Crimson Queue allows you to have multiple cons

Yousef Wadi 11 Jul 30, 2022
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is a distributed POSIX file system built on top of Redis and S3.

JuiceFS is an open-source POSIX file system built on top of Redis and object storage (e.g. Amazon S3), designed and optimized for cloud native environ

Juicedata, Inc 7.2k Jan 2, 2023
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is a distributed POSIX file system built on top of Redis and S3.

JuiceFS is an open-source POSIX file system built on top of Redis and object storage

Juicedata, Inc 7.2k Jan 5, 2023
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is a distributed POSIX file system built on top of Redis and S3.

JuiceFS is a high-performance POSIX file system released under GNU Affero General Public License v3.0. It is specially optimized for the cloud-native

Juicedata, Inc 7.2k Jan 4, 2023
REST based Redis client built on top of Upstash REST API

An HTTP/REST based Redis client built on top of Upstash REST API.

Andreas Thomas 5 Jul 31, 2022
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is a distributed POSIX file system built on top of Redis and S3.

JuiceFS is a high-performance POSIX file system released under GNU Affero General Public License v3.0. It is specially optimized for the cloud-native

Juicedata, Inc 7.2k Jan 1, 2023
redisqueue provides a producer and consumer of a queue that uses Redis streams

redisqueue redisqueue provides a producer and consumer of a queue that uses Redis streams. Features A Producer struct to make enqueuing messages easy.

Robin Joseph 96 Dec 29, 2022
Redis as backend for Queue Package
Redis as backend for Queue Package

redis Redis as backend for Queue package Setup start the redis server redis-server start the redis cluster, see the config # server 01 mkdir server01

golang-queue 11 Oct 16, 2022
A lightweight, distributed and reliable message queue based on Redis

nmq A lightweight, distributed and reliable message queue based on Redis Get Started Download go get github.com/inuggets/nmq Usage import "github.com

Nuggets 2 Nov 22, 2021
Comments
Owner
Kaveh Mousavi Zamani
A software janitor with coding, DBA and sysadmin knowledge. I call my os GNU and Linux a kernel.
Kaveh Mousavi Zamani
GitHub
A highly optimized double-ended queue

Overview Deque is a highly optimized double-ended queue. Benchmark Benchmark_PushBack/Deque<harden> 100000000 10.3 ns/op 9 B/op

Edwin 92 Dec 13, 2022
Fast ring-buffer deque (double-ended queue)

deque Fast ring-buffer deque (double-ended queue) implementation. For a pictorial description, see the Deque diagram Installation $ go get github.com/

Andrew Gillis 412 Dec 26, 2022
Go concurrent-safe, goroutine-safe, thread-safe queue

goconcurrentqueue - Concurrent safe queues The package goconcurrentqueue offers a public interface Queue with methods for a queue. It comes with multi

Enrique Bris 234 Dec 31, 2022
go.fifo provides a simple fifo thread-safe queue for the Go programming language

go.fifo Description go.fifo provides a simple FIFO thread-safe queue. *fifo.Queue supports pushing an item at the end with Add(), and popping an item

Foize 42 Aug 29, 2022
Simple priority queue in Go

Priority Queue in Go ==================== This package provides a priority queue implementation and scaffold interfaces. Installation ------------ U

Kris Kovalik 14 Apr 5, 2022
High-performance minimalist queue implemented using a stripped-down lock-free ringbuffer, written in Go (golang.org)

This project is no longer maintained - feel free to fork the project! gringo A high-performance minimalist queue implemented using a stripped-down loc

Darren Elwood 129 Oct 24, 2022
Cross-platform beanstalkd queue server admin console.

Overview aurora is a web-based Beanstalkd queue server console written in Go and works on macOS, Linux, and Windows machines. The main idea behind usi

null 571 Dec 30, 2022
Fast golang queue using ring-buffer

Queue A fast Golang queue using a ring-buffer, based on the version suggested by Dariusz Górecki. Using this instead of other, simpler, queue implemen

Evan Huus 478 Jan 3, 2023
Go implementation of the van Emde Boas tree data structure: Priority queue for positive whole numbers in O(log log u) time.

vEB Go implementation of the van Emde Boas tree data structure: Priority queue for positive whole numbers in O(log log u) time. Supports the following

null 4 Mar 7, 2022
A tree like tool help you to explore data structures in your redis server

Redis-view is a tree like tool help you explore data structures in your redis server

dreamersdw 19 Mar 17, 2022