A persistent queue implemented in Go.

Related tags

Network goq
Overview

goq

goq is a persistent queue implemented in Go.

Features

  • Communication over HTTP
  • RESTful with JSON responses (trivial to write client libraries)
  • Minimal API (enqueue, dequeue, statistics, version)
  • Basic configuration (address, port, sync, and path)
  • Operations are journaled for persistence via LevelDB
  • Database is health checked during startup, allowing for safe recovery

Dependencies

goq only has one external dependency (LevelDB). Before you start installing anything, go ahead and execute following command:

`go get github.com/syndtr/goleveldb`

Installation

  1. Download/clone this repository to your system.
  2. cd into the repository and execute go build. Note, this will create a binary called goq.
  3. After successfully creating the binary, read about how to configure and run your first instance.

Configuration

There are only a handful of arguments needed to configure goq. Instead of managing a specific file, these are simply binary arguments/flags.

These arguments include:

  • port - The port that this instance of goq should listen on. Default: 11311.

  • sync - Synchronize to LevelDB on every write. Default: true.

  • path - The path to the LevelDB database directory. goq will create this directory if needed. Default: ./db.

goq listen on all addresses by default. If you want to listen on only one address use this parameter:

  • address - The address that this instance of goq should listen on.

Initialization

Now that you've created a binary and read about the configuration parameters above, you are ready to fire up an instance. Go ahead and execute the following command:

./goq -port=11311 -sync=true -journals=/var/log/goq/

After you execute the command, you should see your terminal contain the following log information:

2014/03/10 13:44:17 Listening on :11311
2014/03/10 13:44:17 DB Path: /var/log/goq/
2014/03/10 13:44:17 goq: Starting health check
2014/03/10 13:44:17 goq: Health check successful
2014/03/10 13:44:17 Ready...

This informs you that a LevelDB instance was created in the specified directory and that goq is listening on the desired port. Learn about goq's API from the documentation below.

API

There are only 4 API endpoints of interest, so this should be quick.

POST /enqueue

To enqueue something, execute the following command from your client:

POST /enqueue data=>I am the first item!

The only input parameter required is data, which is a string. Depending on your use case, you may need to URL encode this parameter.

The response you got back should be:

{success:true,message:"worked"}

Go ahead and enqueue more items using the above process.

GET /dequeue

To dequeue data, execute the following from your client:

GET /dequeue?count=1

Note, if the count query parameter isn't specified, goq will only return one item at a time.

If you followed the first enqueue command from above, you should see:

{success:true,data:["I am the first item!"],message:"worked"}

Your data is returned in the exact format that it was enqueued. If you enqueued anything else, try dequeueing again. If you call dequeue on an empty queue, you will receive the following:

{success:true,data:[],message:"worked"}

GET /statistics

To see current goq statistics, execute the following from your client:

GET /statistics

You should see a JSON structure of the server's current statistics in a structure that resembles the following:

{"enqueues":0,"dequeues":0,"empties":0}

Your values may be different from above if you've enqueued or dequeued other items. empties refers to the number of dequeues that were made to an empty queue.

GET /version

To see the current goq version, execute the following from your client:

GET /version

As of the most recent version, you should see:

{version:"1.0.0"}

This should be parsed as a string. The version number will become increasingly important as new features are introduced.

Restarts / Health Checks

When restarted, goq replays all of the items in LevelDB to ensure consistency. You can restore a goq instance just from its LevelDB directory. Please raise an issue if you come across an issue replaying transactions.

Contributions

goq was developed by Kunal Anand.

License

This code is completely free under the MIT License: http://mit-license.org/.

Issues
  • Adding parameter -[a]ddress

    Adding parameter -[a]ddress

    By default goq listen on all addresses. It may be useful to listen on a given address. This patch adds the -[a]ddress parameter to set address to listen on.

    opened by StephaneBunel 1
Owner
Kunal Anand
Kunal Anand
Simple ssh client for persistent host info.

zssh Simple ssh cli for persistent hosts information. Getting started (1) Go get and install $ go get -u github.com/zacscoding/zssh/... $ zssh --help

evan.kim 1 Dec 19, 2021
Go language interface to the Libcircle distributed-queue API

Circle Description The Circle package provides a Go interface to the Libcircle distributed-queue API. Despite the name, Circle has nothing to do with

Los Alamos National Laboratory 24 Nov 1, 2020
A pizza store design using NATS pub sub queue.

A pizza store design using NATS pub sub queue.

Pulak Kanti Bhowmick 21 Aug 2, 2022
BGP implemented in the Go Programming Language

GoBGP: BGP implementation in Go GoBGP is an open source BGP implementation designed from scratch for modern environment and implemented in a modern pr

null 2.9k Aug 10, 2022
URI Templates (RFC 6570) implemented in Go

uritemplates -- import "github.com/jtacoma/uritemplates" Package uritemplates is a level 4 implementation of RFC 6570 (URI Template, http://tools.ietf

Joshua Tacoma 71 Jan 15, 2022
communicate with iOS devices implemented with Golang

Golang-iDevice much more easy to use ?? electricbubble/gidevice-cli Installation go get github.com/electricbubble/gidevice Devices package main impor

雷系泡泡 191 Aug 2, 2022
A TCP socket based chat server implemented using Go

Go Chat Server A better TCP socket chat server implemented using Go Connecting nc localhost 5000 Docker Build the container image docker build -t grub

Christopher Gruber 0 Oct 16, 2021
many tools implemented in Golang

Go-Tools many tools implemented in Golang tools project comments abtest AB测试分流 dag_flow DAG工作流 hugo_themes Hugo主题 log_monitor 日志监控服务 pepper_cache 内存kv

殷雅俊 0 Dec 12, 2021
A Twirp RPC OpenAPI generator implemented as `protoc` plugin

twirp-openapi-gen A Twirp RPC OpenAPI generator implemented as protoc plugin Currently supports only OpenAPI 2.0 Usage Installing the generator for pr

Albenik's Golang Projects 1 May 26, 2022
A simple FTP protocol with client and server implemented in TypeScript and Golang

websocket-ftp A simple FTP protocol with client and server implemented in TypeScript and Golang. Example (Client) const buffer: Uint8Array = (new Text

LQR471814 0 Apr 14, 2022
Pacemaker - Rate limit library. Currently implemented rate limits are

PaceMaker Rate limit library. Currently implemented rate limits are Fixed window

Marquitos 2 Apr 2, 2022
Absystem - Golang implemented absystem core api

环境初始化 export GOPROXY=https://goproxy.cn,direct && export GO111MODULE=on # OR go

liangchengming 2 Feb 16, 2022
The HomeKit Accessory Protocol (hap) implemented in Go

hap hap (previously hc) is a lightweight library to develop HomeKit accessories in Go. It abstracts the HomeKit Accessory Protocol (HAP) and makes it

Matthias 84 Aug 2, 2022
Sandglass is a distributed, horizontally scalable, persistent, time sorted message queue.

Sandglass is a distributed, horizontally scalable, persistent, time ordered message queue. It was developed to support asynchronous tasks and message

Sandglass 1.5k Aug 12, 2022
A simple persistent directory-backed FIFO queue.

pqueue pqueue is a simple persistent directory-backed FIFO queue. It provides the typical queue interface Enqueue and Dequeue and may store any byte s

Philipp C. Heckel 4 May 7, 2022
Persistent queue in Go based on BBolt

Persistent queue Persistent queue based on bbolt DB. Supposed to be used as embeddable persistent queue to any Go application. Features: messages are

Aleksandr Baryshnikov 2 Jun 30, 2022
Unlimited job queue for go, using a pool of concurrent workers processing the job queue entries

kyoo: A Go library providing an unlimited job queue and concurrent worker pools About kyoo is the phonetic transcription of the word queue. It provide

Dirk Pahl 38 Jun 8, 2022
Machinery is an asynchronous task queue/job queue based on distributed message passing.

Machinery Machinery is an asynchronous task queue/job queue based on distributed message passing. V2 Experiment First Steps Configuration Lock Broker

Richard Knop 6.4k Aug 8, 2022
Machinery is an asynchronous task queue/job queue based on distributed message passing.

Machinery Machinery is an asynchronous task queue/job queue based on distributed message passing. V2 Experiment First Steps Configuration Lock Broker

Richard Knop 6.4k Aug 1, 2022
painless task queue manager for shell commands with an intuitive cli interface (execute shell commands in distributed cloud-native queue manager).

EXEQ DOCS STILL IN PROGRESS. Execute shell commands in queues via cli or http interface. Features Simple intuitive tiny cli app. Modular queue backend

Mohammed Al Ashaal 12 Jan 29, 2022
A Multi Consumer per Message Queue with persistence and Queue Stages.

CrimsonQ A Multi Consumer per Message Queue with persistence and Queue Stages. Under Active Development Crimson Queue allows you to have multiple cons

Yousef Wadi 11 Jul 30, 2022
High-performance minimalist queue implemented using a stripped-down lock-free ringbuffer, written in Go (golang.org)

This project is no longer maintained - feel free to fork the project! gringo A high-performance minimalist queue implemented using a stripped-down loc

Darren Elwood 127 Jul 18, 2022
ZenQ - A low-latency thread-safe queue in golang implemented using a lock-free ringbuffer

ZenQ A low-latency thread-safe queue in golang implemented using a lock-free ringbuffer Features Much faster than native channels in both SPSC (single

Anish Mukherjee 453 Aug 8, 2022
persistent storage for flags in go

ingo is a simple Go library helping you to persist flags in a ini-like config file. Features and limitations Requires Go 1.5 or later automatically cr

null 36 Jun 22, 2022
A simple, fast, embeddable, persistent key/value store written in pure Go. It supports fully serializable transactions and many data structures such as list, set, sorted set.

NutsDB English | 简体中文 NutsDB is a simple, fast, embeddable and persistent key/value store written in pure Go. It supports fully serializable transacti

徐佳军 2.3k Aug 8, 2022
A persistent and flexible background jobs library for go.

Jobs Development Status Jobs is no longer being actively developed. I will still try my best to respond to issues and pull requests, but in general yo

Alex Browne 489 Jul 23, 2022
CSI Driver for dynamic provisioning of Persistent Local Volumes for Kubernetes using LVM.

OpenEBS LVM CSI Driver CSI driver for provisioning Local PVs backed by LVM and more. Project Status Currently the LVM CSI Driver is in alpha

OpenEBS 96 Jul 24, 2022
a persistent real-time key-value store, with the same redis protocol with powerful features

a fast NoSQL DB, that uses the same RESP protocol and capable to store terabytes of data, also it integrates with your mobile/web apps to add real-time features, soon you can use it as a document store cause it should become a multi-model db. Redix is used in production, you can use it in your apps with no worries.

Mohammed Al Ashaal 1.1k Jul 29, 2022