A single binary, simple, message queue.

Related tags

Messaging miniqueue
Overview

MiniQueue

Tests

A stupid simple, single binary message queue using HTTP/2.

Most messaging workloads don't require enormous amounts of data, endless features or infinite scaling. Instead, they'd probably be better off with something dead simple.

MiniQueue is just that. A simple queue. You can publish bytes to topics and your consumers will receive what you published, nothing more.

Features

  • Simple to run
  • Very fast
  • Not infinitely scalable
  • Multiple topics
  • HTTP/2
  • Publish
  • Subscribe
  • Acknowledgements
  • Persistent
  • Prometheus metrics

API

  • POST /publish/:topic
  • GET /subscribe/:topic

Usage

Run MiniQueue where you would like.

It will then expose an HTTP/2 server used for publishing and consuming.

Benchmarks

As MiniQueue is under heavy development, take these benchmarks with a grain of salt. However, for those curious:

λ ~/ go-wrk -c 12 -d 10 -M POST -body "helloworld" https://localhost:8080/publish/test
Running 10s test @ https://localhost:8080/publish/test
  12 goroutine(s) running concurrently
104084 requests in 9.942585489s, 5.76MB read
Requests/sec:           10468.50
Transfer/sec:           592.94KB
Avg Req Time:           1.146295ms
Fastest Request:        262.281µs
Slowest Request:        867.958415ms
Number of Errors:       0

Running on my MacBook Pro (15-inch, 2019), with a 2.6 GHz 6-Core Intel Core i7.

Contributing

Contributors are more than welcome. Please feel free to open a PR to improve anything you don't like, or would like to add. No PR is too small!

License

This project is licensed under the MIT license.

Issues
  • Subscription command issue?

    Subscription command issue?

    I love this queue -- go, simple, secure. Thank you. However I am having a terrible time getting subscriptions to work.

    curl --insecure --cacert certs/ca.key --key certs/node.key --cert certs/node.crt -X POST https://localhost:9999/subscribe/foo                                                     
    {"error":"error decoding command"}
    

    Is it not possible to test subscriptions with curl? I publish just fine using the example in README. Thanks.

    opened by darsenault 3
  • TLS handshake error from 127.0.0.1:57977: local error: tls: bad record MAC

    TLS handshake error from 127.0.0.1:57977: local error: tls: bad record MAC

    Hi tomarrell:

    I just clone the repository, and run the following:

    1. build the exetutable in the root directory
    go build .
    
    1. start the server
    ./miniqueue.exe --human
    
    1. publish a message to a topic using curl
    curl -X POST https://localhost:8080/publish/foo --data "helloworld"
    

    the server produce:

    2021/02/19 19:28:17 http: TLS handshake error from 127.0.0.1:57977: local error: tls: bad record MAC
    

    and curl produce:

    curl: (60) SSL certificate problem: unable to get local issuer certificate
    More details here: https://curl.haxx.se/docs/sslcerts.html
    
    curl failed to verify the legitimacy of the server and therefore could not
    establish a secure connection to it. To learn more about this situation and
    how to fix it, please visit the web page mentioned above.
    

    the echo example has similar problem.

    maybe I should create a certificate on my own, and try again?

    I'm using Windows10 and git-bash

    opened by brownchow 2
  • Allow subscription to topic which doesn't exist yet

    Allow subscription to topic which doesn't exist yet

    Currently if a subscriber attempts to read from a non-existent queue they will receive an error.

    This behaviour may be desirable in certain circumstances.

    enhancement 
    opened by tomarrell 0
Releases(v0.8.0)
Owner
Tom Arrell
Senior Backend Engineer @sumup — Lover of Rust and Go, hack around building keyboards when I'm bored
Tom Arrell
A Multi Consumer per Message Queue with persistence and Queue Stages.

CrimsonQ A Multi Consumer per Message Queue with persistence and Queue Stages. Under Active Development Crimson Queue allows you to have multiple cons

Yousef Wadi 10 Apr 23, 2022
Simple docker container to publish a fixed message to a specified queue. Created to be used with k8s CRON scheduling.

RabbitMQ Publish CRON Simple docker container to publish a fixed message to a specified rabbitmq exchange. Created to be used as part of a Kubernetes

Daniel Emery 0 Dec 20, 2021
Alertmanager go message broker - A simple message broker made to integrate with alertmanager/prometheus

Alertmanager message broker Prerequisites Go 1.16+ Sqllite driver About: The alertmanager message broker is a project made to meet some of my needs to

Davi Araújo 0 Dec 27, 2021
A lightweight, distributed and reliable message queue based on Redis

nmq A lightweight, distributed and reliable message queue based on Redis Get Started Download go get github.com/inuggets/nmq Usage import "github.com

Nuggets 2 Nov 22, 2021
KubeMQ is a Kubernetes native message queue broker

KubeMQ Community is the open-source version of KubeMQ, the Kubernetes native message broker. More about KubeMQ

Temur Yunusov 0 Nov 20, 2021
Kafka implemented in Golang with built-in coordination (No ZooKeeper, single binary install, Cloud Native)

Jocko Distributed commit log service in Go that is wire compatible with Kafka. Created by @travisjeffery, continued by nash. Goals: Protocol compatibl

Nash.io 104 Aug 9, 2021
Asynq: simple, reliable, and efficient distributed task queue in Go

Asynq Overview Asynq is a Go library for queueing tasks and processing them asynchronously with workers. It's backed by Redis and is designed to be sc

Ken Hibino 3.6k Jul 2, 2022
A simple persistent directory-backed FIFO queue.

pqueue pqueue is a simple persistent directory-backed FIFO queue. It provides the typical queue interface Enqueue and Dequeue and may store any byte s

Philipp C. Heckel 4 May 7, 2022
Go simple async message bus

?? message-bus Go simple async message bus. ?? ABOUT Contributors: Rafał Lorenz Want to contribute ? Feel free to send pull requests! Have problems, b

Rafał Lorenz 221 Jun 22, 2022
Messagebus - Simple Message Bus Written in Golang

MessageBus Simple Message Bus Written in Golang How to Use go get gopkg.io/Usada

Usada Pekora 2 Apr 21, 2022
RapidMQ is a pure, extremely productive, lightweight and reliable library for managing of the local messages queue

RapidMQ RapidMQ is a pure, extremely productive, lightweight and reliable library for managing of the local messages queue in the Go programming langu

Vadim Shakun 63 Apr 24, 2022
redisqueue provides a producer and consumer of a queue that uses Redis streams

redisqueue redisqueue provides a producer and consumer of a queue that uses Redis streams. Features A Producer struct to make enqueuing messages easy.

Robin Joseph 83 Jun 14, 2022
dque is a fast, embedded, durable queue for Go

dque - a fast embedded durable queue for Go dque is: persistent -- survives program restarts scalable -- not limited by your RAM, but by your disk spa

Jon Carlson 696 Jun 15, 2022
Gue is Golang queue on top of PostgreSQL that uses transaction-level locks.

Gue is Golang queue on top of PostgreSQL that uses transaction-level locks.

Vladimir Garvardt 91 Jun 15, 2022
Queue with NATS Jetstream to remove all the erlangs from cloud

Saf in Persian means Queue. One of the problems, that we face on projects with queues is deploying RabbitMQ on the cloud which brings us many challenges for CPU load, etc. I want to see how NATS with Jetstream can work as the queue to replace RabbitMQ.

Parham Alvani 11 Jan 7, 2022
A fast durable queue for Go

pqueue - a fast durable queue for Go pqueue is thread-safety, serves environments where more durability is required (e.g., outages last longer than me

Linh Tran Tuan 12 Jun 10, 2022
Redis as backend for Queue Package

redis Redis as backend for Queue package Setup start the redis server redis-server start the redis cluster, see the config # server 01 mkdir server01

golang-queue 10 Apr 23, 2022
NSQ as backend for Queue Package

NSQ as backend for Queue Package

golang-queue 9 Jan 22, 2022
Kudruk helps you to create queue channels and manage them gracefully.

kudruk Channels are widely used as queues. kudruk (means queue in Turkish) helps you to easily create queue with channel and manage the data in the qu

Erhan Yakut 8 Feb 21, 2022