go.fifo provides a simple fifo thread-safe queue for the Go programming language

Overview

go.fifo

Description

go.fifo provides a simple FIFO thread-safe queue. *fifo.Queue supports pushing an item at the end with Add(), and popping an item from the front with Next(). There is no intermediate type for the stored data. Data is directly added and retrieved as type interface{} The queue itself is implemented as a single-linked list of chunks containing max 64 items each.

Installation

go get github.com/foize/go.fifo

Usage

package main

import (
	"github.com/foize/go.fifo"
	"fmt"
)

func main() {
	// create a new queue
	numbers := fifo.NewQueue()

	// add items to the queue
	numbers.Add(42)
	numbers.Add(123)
	numbers.Add(456)

	// retrieve items from the queue
	fmt.Println(numbers.Next()) // 42
	fmt.Println(numbers.Next()) // 123
	fmt.Println(numbers.Next()) // 456
}
package main

import (
	"github.com/foize/go.fifo"
	"fmt"
)

type thing struct {
	Text string
	Number int
}

func main() {
	// create a new queue
	things := fifo.NewQueue()

	// add items to the queue
	things.Add(&thing{
		Text: "one thing",
		Number: 1,
	})
	things.Add(&thing {
		Text: "another thing",
		Number: 2,
	})

	// retrieve items from the queue
	for  {
		// get a new item from the things queue
		item := things.Next();

		// check if there was an item
		if item == nil {
			fmt.Println("queue is empty")
			return
		}

		// assert the type for the item
		someThing := item.(*thing)

		// print the fields
		fmt.Println(someThing.Text)
		fmt.Printf("with number: %d\n", someThing.Number)
	}
}

/* output: */
// one thing
// with number: 1
// another thing
// with number: 2
// queue is empty

Documentation

Documentation can be found at godoc.org/github.com/foize/go.fifo. For more detailed documentation, read the source.

History

This package is based on github.com/yasushi-saito/fifo_queue There are several differences:

  • renamed package to fifo to make usage simpler
  • removed intermediate type Item and now directly using interface{} instead.
  • renamed (*Queue).PushBack() to (*Queue).Add()
  • renamed (*Queue).PopFront() to (*Queue).Next()
  • Next() will not panic on empty queue, will just return nil interface{}
  • Add() does not accept nil interface{} and will panic when trying to add nil interface{}.
  • Made fifo.Queue thread/goroutine-safe (sync.Mutex)
  • Added a lot of comments
  • renamed internal variable/field names
You might also like...
High-performance minimalist queue implemented using a stripped-down lock-free ringbuffer, written in Go (golang.org)

This project is no longer maintained - feel free to fork the project! gringo A high-performance minimalist queue implemented using a stripped-down loc

A Go queue manager on top of Redis

Queue A Go library for managing queues on top of Redis. It is based on a hiring exercise but later I found it useful for myself in a custom task proce

Cross-platform beanstalkd queue server admin console.
Cross-platform beanstalkd queue server admin console.

Overview aurora is a web-based Beanstalkd queue server console written in Go and works on macOS, Linux, and Windows machines. The main idea behind usi

Fast golang queue using ring-buffer

Queue A fast Golang queue using a ring-buffer, based on the version suggested by Dariusz Górecki. Using this instead of other, simpler, queue implemen

Go implementation of the van Emde Boas tree data structure: Priority queue for positive whole numbers in O(log log u) time.

vEB Go implementation of the van Emde Boas tree data structure: Priority queue for positive whole numbers in O(log log u) time. Supports the following

Null Types, Safe primitive type conversion and fetching value from complex structures.

Typ Typ is a library providing a powerful interface to impressive user experience with conversion and fetching data from built-in types in Golang Feat

Type-safe, zero-allocation sets for Go

Set Package set is a type-safe, zero-allocation port of the excellent package fatih/set. It contains sets for most of the basic types and you can gene

dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies
dagger is a fast, concurrency safe, mutable, in-memory directed graph library with zero dependencies

dagger is a blazing fast, concurrency safe, mutable, in-memory directed graph implementation with zero dependencies

BTree provides a simple, ordered, in-memory data structure for Go programs.

BTree implementation for Go This package provides an in-memory B-Tree implementation for Go, useful as an ordered, mutable data structure. The API is

Comments
  • Added Unsafe Methods for Group Processing

    Added Unsafe Methods for Group Processing

    I changed a bit the library to be able to do batch processing.

    It doesn't change the behavior of the current methods, but I added few:

    • UnsafeAdd => Adds without locking
    • UnsafeNext => Get's next without locking
    • UnsafeLock => Locks
    • UnsafeUnlock => Unlocks

    That might seens a bit odd, but I have a use case here. Let's suppose I have a routine that needs N samples from the fifo, and other routine that puts V samples into the fifo.

    In the current version you would do:

    func RoutineThatPuts() {
       for i:= 0; i<V; i++ {
          fifo.Add(item[V]);
       }
    }
    
    func RoutineThatGets() {
        myitems := make([]item, N)
       for i:= 0; i<V; i++ {
          myitems[N] = fifo.Next()
       }
    }
    

    So the thing is that two things happen here:

    1. Each iteration of the loop will call lock/unlock from the mutex,
    2. If the two operations are concurrent, they will add and remove at "same" time

    For sure that won't give any data corruption, but can impact performance due the mutex.lock() / unlock() calls (which are sort of expensive according to pprof). With my changes I can do this:

    func RoutineThatPuts() {
       fifo.UnsafeLock();
       defer fifo.UnsafeUnlock();
       for i:= 0; i<V; i++ {
          fifo.UnsafeAdd(item[V]);
       }
    }
    
    func RoutineThatGets() {
        myitems := make([]item, N)
       fifo.UnsafeLock();
       defer fifo.UnsafeUnlock();
       for i:= 0; i<V; i++ {
          myitems[N] = fifo.UnsafeNext()
       }
    }
    

    This way, it will ensure that the loop runs without lock/unlocking the fifo unnecessarily and both operations will write all data that it wants before anyone reading it.

    It doesn't impact in any way the current flow of Add / Next, so that is just a new feature I would like to add to the oficial library, since it might be useful depending on the use case.

    opened by racerxdl 6
Owner
Foize
Discontinued company. Github organization is not removed to avoid breaking go.fifo and go.sgr usage. Maintainance by @GeertJohan
Foize
Package ring provides a high performance and thread safe Go implementation of a bloom filter.

ring - high performance bloom filter Package ring provides a high performance and thread safe Go implementation of a bloom filter. Usage Please see th

Tanner Ryan 128 Sep 26, 2022
A simple and efficient thread-safe sharded hashmap for Go

shardmap A simple and efficient thread-safe sharded hashmap for Go. This is an alternative to the standard Go map and sync.Map, and is optimized for w

Josh Baker 192 Sep 21, 2022
A thread safe map which has expiring key-value pairs

~ timedmap ~ A map which has expiring key-value pairs. go get github.com/zekroTJA/timedmap Intro This package allows to set values to a map which will

zekro 47 Sep 16, 2022
A Golang lock-free thread-safe HashMap optimized for fastest read access.

hashmap Overview A Golang lock-free thread-safe HashMap optimized for fastest read access. Usage Set a value for a key in the map: m := &HashMap{} m.S

Cornel 1.3k Sep 23, 2022
Simple priority queue in Go

Priority Queue in Go ==================== This package provides a priority queue implementation and scaffold interfaces. Installation ------------ U

Kris Kovalik 14 Apr 5, 2022
Dynamic object-oriented programming support for the Go language

Goop Description The Goop (Go Object-Oriented Programming) package provides support for dynamic object-oriented programming constructs in Go, much lik

Los Alamos National Laboratory 106 Aug 17, 2022
Generates data structure definitions from JSON files for any kind of programming language

Overview Archivist generates data structure definitions from JSON files for any kind of programming language. It also provides a library for golang to

Kingsgroup 45 Jun 28, 2022
A serverless cluster computing system for the Go programming language

Bigslice Bigslice is a serverless cluster data processing system for Go. Bigslice exposes composable API that lets the user express data processing ta

GRAIL 508 Aug 31, 2022
A highly optimized double-ended queue

Overview Deque is a highly optimized double-ended queue. Benchmark Benchmark_PushBack/Deque<harden> 100000000 10.3 ns/op 9 B/op

Edwin 66 Sep 15, 2022
Fast ring-buffer deque (double-ended queue)

deque Fast ring-buffer deque (double-ended queue) implementation. For a pictorial description, see the Deque diagram Installation $ go get github.com/

Andrew Gillis 383 Sep 27, 2022