Easy to use Raft library to make your app distributed, highly available and fault-tolerant

Overview

Go Report Card Go Reference GitHub go.mod Go version of a Go module GitHub release

An easy to use customizable library to make your Go application Distributed, Highly available, Fault Tolerant etc... using Hashicorp's Raft library which implements the Raft Consensus Algorithm.

Features

  • Configure and start a fully functional Raft node by writing ~10 lines of code
  • Automatic Node discovery (nodes are discovering each other using Discovery method)
    1. Built-in discovery methods:
      1. Static Discovery (having a fixed list of nodes addresses)
      2. mDNS Discovery for local network node discovery
      3. Kubernetes discovery
  • Cloud Native because of kubernetes discovery and easy to load balance features
  • Automatic forward to leader - you can contact any node to perform operations, everything will be forwarded to the actual leader node
  • Node monitoring/removal - the nodes are monitoring each other and if there are some failures then the offline nodes get removed automatically from cluster
  • Simplified state machine - there is an already implemented generic state machine which handles the basic operations and routes requests to State Machine Services (see Examples)
  • All layers are customizable - you can select or implement your own State Machine Service, Message Serializer and Discovery Method
  • gRPC transport layer - the internal communications are done through gRPC based communication, if needed you can add your own services

Note: snapshots are not supported at the moment, will be handled at later point Note: at the moment the communication between nodes are insecure, I recommend to not expose that port

Get Started

You can create a simple EasyRaft Node with local mDNS discovery, an in-memory Map service and MsgPack as serializer(this is the only one built-in at the moment)

import (
"github.com/ksrichard/easyraft"
"github.com/ksrichard/easyraft/discovery"
"github.com/ksrichard/easyraft/fsm"
"github.com/ksrichard/easyraft/serializer"
)

func main() {
    raftPort := 5000
    discoveryPort := 5001
    dataDir := "s1"
    node, err := easyraft.NewNode(
        raftPort,
        discoveryPort,
        dataDir,
        []fsm.FSMService{fsm.NewInMemoryMapService()},
        serializer.NewMsgPackSerializer(),
        discovery.NewMDNSDiscovery(),
        false,
    )
    
    if err != nil {
        panic(err)
    }
    stoppedCh, err := node.Start()
    if err != nil {
        panic(err)
    }
    defer node.Stop()
}

Examples

Examples can be found in the examples directory

TODO

  • Add more examples
  • Test coverage
  • Secure communication between nodes (SSL/TLS)
  • Backup/Restore backup handling
  • Allow configuration option to pass any custom raft.FSM
You might also like...
A distributed MySQL binlog storage system built on Raft
A distributed MySQL binlog storage system built on Raft

What is kingbus? 中文 Kingbus is a distributed MySQL binlog store based on raft. Kingbus can act as a slave to the real master and as a master to the sl

Distributed-Services - Distributed Systems with Golang to consequently build a fully-fletched distributed service

Distributed-Services This project is essentially a result of my attempt to under

Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant. A feature complete and high performance multi-group Raft library in Go.
A feature complete and high performance multi-group Raft library in Go.

Dragonboat - A Multi-Group Raft library in Go / 中文版 News 2021-01-20 Dragonboat v3.3 has been released, please check CHANGELOG for all changes. 2020-03

Golang implementation of the Raft consensus protocol

raft raft is a Go library that manages a replicated log and can be used with an FSM to manage replicated state machines. It is a library for providing

The TinyKV course builds a key-value storage system with the Raft consensus algorithm.
The TinyKV course builds a key-value storage system with the Raft consensus algorithm.

The TinyKV Course The TinyKV course builds a key-value storage system with the Raft consensus algorithm. It is inspired by MIT 6.824 and TiKV Project.

A naive implementation of Raft consensus algorithm.

This implementation is used to learn/understand the Raft consensus algorithm. The code implements the behaviors shown in Figure 2 of the Raft paper wi

Raft: a consensus algorithm for managing a replicated log

Raft Consensus Algorithm Raft is a consensus algorithm for managing a replicated

This is my implementation of Raft consensus algorithm that I did for own learning.

This is my implementation of Raft consensus algorithm that I did for own learning. Please follow the link to learn more about raft consensus algorithm https://raft.github.io. And Soon, I will be developing same algorithm in Java as well

Releases(v1.0.0)
  • v1.0.0(Dec 3, 2021)

Owner
Richard Bertok
Richard Bertok
Dkron - Distributed, fault tolerant job scheduling system https://dkron.io

Dkron - Distributed, fault tolerant job scheduling system for cloud native environments Website: http://dkron.io/ Dkron is a distributed cron service,

Distributed Works 3.3k Sep 23, 2022
A distributed fault-tolerant order book matching engine

Go - Between A distributed fault-tolerant order book matching engine. Features Limit orders Market orders Order book depth Calculate market price for

Daniel Gatis 1 Dec 24, 2021
An implementation of a distributed KV store backed by Raft tolerant of node failures and network partitions 🚣

barge A simple implementation of a consistent, distributed Key:Value store which uses the Raft Concensus Algorithm. This project launches a cluster of

Shehjad Khan 0 Nov 24, 2021
Lightweight, fault-tolerant message streams.

Liftbridge provides lightweight, fault-tolerant message streams by implementing a durable stream augmentation for the NATS messaging system. It extend

Liftbridge 2.4k Sep 28, 2022
Scalable, fault-tolerant application-layer sharding for Go applications

ringpop-go (This project is no longer under active development.) Ringpop is a library that brings cooperation and coordination to distributed applicat

Uber Open Source 752 Sep 26, 2022
Raft library Raft is a protocol with which a cluster of nodes can maintain a replicated state machine.

Raft library Raft is a protocol with which a cluster of nodes can maintain a replicated state machine. The state machine is kept in sync through the u

Kalyan Akella 0 Oct 15, 2021
null 1 Feb 3, 2022
Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

Sergey Shepelev 25 Dec 24, 2019
A linearizability distributed database by raft and wisckey.

AlfheimDB A linearizability distributed database by raft and wisckey, which supports redis client. Build This project build by mage, you will need ins

chunming.dong 73 Jul 18, 2022
Distributed disk storage database based on Raft and Redis protocol.

IceFireDB Distributed disk storage system based on Raft and RESP protocol. High performance Distributed consistency Reliable LSM disk storage Cold and

IceFireDB 921 Sep 24, 2022