Consistent hashing hashring implementation.

Overview

hashring

GoDoc CI

Consistent hashing hashring implementation.

Overview

This is an implementation of the consistent hashing hashring data structure.

In general, consistent hashing is all about mapping objects from a very big set of values (e.g., request id) to objects from a quite small set (e.g., server address). The word "consistent" means that it can produce consistent mapping on different machines or processes without additional state exchange and communication.

For more theory about the subject please see this great document.

For more info about the package please read the docs.

The Why?

This is an approach for load distribution across servers which I found convinient to be used in a few projects I had worked on in the past.

I think it is good to implement it from scratch to synthesize all my experience working with it and share it with the community in hope it will help to build something good.

The key points of the package:

  1. Efficient concurrent read operations
  2. Correct handling of write operations (order of insertions on different processes doesn't matter; hash collisions are handled carefully).

Installation

go get github.com/gobwas/hashring

Usage

package main

import (
	"strings"
	"io"

	"github.com/gobwas/hashring"
)

func main() {
	var ring hashring.Ring
	_ = ring.Insert(StringItem("server01"))
	_ = ring.Insert(StringItem("server02"))
	_ = ring.Insert(StringItem("server03"))
	_ = ring.Insert(StringItem("server04"))

	ring.Get(StringItem("user01")) // server04
	ring.Get(StringItem("user02")) // server04
	ring.Get(StringItem("user03")) // server02
	ring.Get(StringItem("user04")) // server01
}

type StringItem string

func (s StringItem) WriteTo(w io.Writer) (int64, error) {
	n, err := io.WriteString(w, string(s))
	return int64(n), err
}

Contributing

If you find some bug or want to improve this package in any way feel free to file an issue to discuss your findings or ideas.

Debugging

Note that this package comes with built in debug tracing (which only takes place if you pass hashring_trace build tag, thanks to gtrace zero-cost tracing).

This means that you can make hashring to print out each meaningful step it does to understand better what's happening under the hood.

It also consumes hashring_debug build tag, which allows you to hook into hash calculation process and override the value it returns. For example, this was useful to test hash collisions.

Releases(v0.1.1)
Owner
Sergey Kamardin
Sergey Kamardin
Argon2 password hashing package for go with constant time hash comparison

argon2pw Argon2 password hashing package with constant time hash comparison Preface: Argon2 was selected as the winner of the Password Hashing Competi

Raja Bhatia 87 Sep 15, 2021
ID hashing and Obfuscation using Knuth's Algorithm

ID Obfuscation/Hashing Transformer for Go There are many times when you want to generate obfuscated ids. This package utilizes Knuth's Hashing Algorit

pj 284 Sep 7, 2021
Hashing algorithms simplified (supports Argon2, Bcrypt, Scrypt, PBKDF2, Chacha20poly1305 and more in the future)

PHC Crypto Inspired by Upash, also implementing PHC string format Usage Currently there are two options of using this package: Import all Import speci

Reinaldy Rafli 14 Sep 11, 2021
Implementation of polynomial KZG proofs and 257-ary verkle trie

257-ary verkle trie Disclaimer: the code in this package is experimental. It can only be used in research and is not suitable for use in production. T

Evaldas Drasutis 3 Sep 6, 2021
Git-like capabilities for your object storage

What is lakeFS lakeFS is an open source layer that delivers resilience and manageability to object-storage based data lakes. With lakeFS you can build

Treeverse 1.6k Sep 18, 2021
Pure Go implementation of the NaCL set of API's

go-nacl This is a pure Go implementation of the API's available in NaCL: https://nacl.cr.yp.to. Compared with the implementation in golang.org/x/crypt

Kevin Burke 509 Sep 8, 2021
An easy-to-use SHA-1 hash-cracker written in Golang.

wrench - An easy-to-use SHA-1 hash-cracker. Wrench is an SHA-1 hash-cracker that relies on wordlists for comparing hashes, and cracking them. Before W

null 4 Aug 29, 2021
Product Analytics, Business Intelligence, and Product Management in a fully self-contained box

Engauge Concept It's not pretty but it's functional. Track user interactions in your apps and products in real-time and see the corresponding stats in

Engauge 92 Sep 7, 2021
A framework for creating COM-based bypasses utilizing vulnerabilities in Microsoft's WDAPT sensors.

Dent More Information If you want to learn more about the techniques utlized in this framework please take a look at this article. Description This fr

Optiv Security 258 Sep 16, 2021
Automatic HTTPS for any Go program: fully-managed TLS certificate issuance and renewal

Easy and Powerful TLS Automation The same library used by the Caddy Web Server Caddy's automagic TLS features—now for your own Go programs—in one powe

Caddy 3.7k Sep 13, 2021
PHP functions implementation to Golang. This package is for the Go beginners who have developed PHP code before. You can use PHP like functions in your app, module etc. when you add this module to your project.

PHP Functions for Golang - phpfuncs PHP functions implementation to Golang. This package is for the Go beginners who have developed PHP code before. Y

Serkan Algur 42 Aug 19, 2021
Implementation of io/fs.FS that appends SHA256 hashes to filenames to allow for aggressive HTTP caching.

hashfs Implementation of io/fs.FS that appends SHA256 hashes to filenames to allow for aggressive HTTP caching.

Ben Johnson 183 Sep 1, 2021
How to systematically secure anything: a repository about security engineering

How to Secure Anything Security engineering is the discipline of building secure systems. Its lessons are not just applicable to computer security. In

Veeral Patel 6.1k Sep 16, 2021
A Go language implementation of the proposed ads.cert protocols for integration in programmatic ads solutions.

go-adscert A Go language implementation of the proposed ads.cert protocols for integration in programmatic ads solutions. This repository is a work-in

Curtis Light 3 Jun 4, 2021
Implementation of Secret Service API

Secret Service Implementation of Secret Service API What does this project do? By using secret service, you don't need to use KeePassXC secretservice

Remisa Yousefvand 32 Sep 21, 2021
Shellcode implementation of Reflective DLL Injection by Golang. Convert DLLs to position independent shellcode

?? Frog For Automatic Scan ?? Doge For Defense Evasion&Offensive Security Doge-sRDI Shellcode implementation of Reflective DLL Injection by Golang. Co

TimWhite 29 Sep 21, 2021
:lock: acmetool, an automatic certificate acquisition tool for ACME (Let's Encrypt)

acmetool is an easy-to-use command line tool for automatically acquiring certificates from ACME servers (such as Let's Encrypt). Designed to flexibly

Hugo Landau 1.9k Sep 18, 2021