A distributed Configuration Center server that manages config in a container. The container is composed of fields (abstract layer includes: KV, LIST, DICT type). The Field contains basic datatypes (int, float, bool, string, list, dict).

Overview

cassem

Go Report Card go.de │ v reference

config assembler from key-value pairs' container which include basic datatypes, such as int, string, float, bool, list, dict

Features

  • HTTP Restful API.
  • Export container (config container) into different file format (JSON / TOML).
  • Manage CTL / UI support.
  • Master / Slave architecture support based raft (only write on master).
    • Master / Slave architecture and only allow to write on master.
    • Slave need to forward request to master.
  • RESTful API permission control.
  • Changes watching and notifying.
    • Watcher component.
    • Watcher Server.
    • Client SDK support.
  • Distributed Cache middleware to speed up the API which downloads container in specified format.

Documentation

- cassemctl

- cassemd

Benchmark

benchmark tested core RESTful API and try to optimize them, each benchmark test displays the final QPS result.

README

References

Issues
  • develop `cassemctl` command

    develop `cassemctl` command

    features:

    1. genconf generates the default config file for users.
    2. migrate command to initialize persistence component (for now, DB support).
    3. basic resource managing commands (CURD) + watch. Maybe, this would build on the client SDK ...
    4. health diagnosis (TODO). metrics and cluster infos.
    opened by yeqown 1
  • move `cache` instance from `core.Core`

    move `cache` instance from `core.Core`

    For now, cache is held by the core.Core, but the cache component is mainly used in the raft.FSM. To close up all of the cache component entries, so move it into FSMWrapper maybe make it cleraly to read.

    enhancement 
    opened by yeqown 1
  • optimize join action

    optimize join action

    Background

    cassemd must get leader address so that it could join to cluster successfully, this is not easy to use and is unreasonable.

    Solution

    • forward join request to the leader while a slave node receives a join request
    • allow cassemd to receive an array of node address so that it could try to join one by one.
    enhancement 
    opened by yeqown 1
  • Remove pair concepts from cassem

    Remove pair concepts from cassem

    • Redesign the concepts of config container.
    • Focus on the edge problems in the situation of config center.
    • The reuse of pair maybe not the main issue on which the config center should take care.
    opened by yeqown 0
  • feature/refactor persistence

    feature/refactor persistence

    • feat: adjust Repository constructor API
    • feat: independent convertor from repository
    • feat: reduce authorizer from server but in coordinator
    • refactor: pull away raft from core which makes core as daemon rather than a component.
    • refactor: merge user repo, repo and policy adapter.
    opened by yeqown 0
  • persistence refactor

    persistence refactor

    Background

    Need a distributed persistence component to support what helps cassem become a system satisfied CP and provide available as much as possible. On another way, DO NOT want to import another dependency to make the save procedure complicated.

    Risk

    What is a risk is that would changes the cassem software architecture, even makes HTTP API changes.

    Thoughts

    bbolt would be used to replace mysql.

    opened by yeqown 0
  • replace cache implementation with LRU-K(2) replacement algorithm

    replace cache implementation with LRU-K(2) replacement algorithm

    now cache components are implemented by go built-in map, it's too simple to manage caches. So that cassemd needs a new version of the cache component to manage caches.

    The duty of the cache component includes:

    // IStore required ICache to implement persist and restore functions so that the cache could be
    // save into file and restore from that.
    type IStore interface {
    	// Persist serializes cache into []byte data.
    	Persist() ([]byte, error)
    
    	// Restore apply data to override all the cache data, data is coming from Persist.
    	Restore(data []byte) error
    }
    
    // ICache represents the proxy to operate cache data,
    // also need to replace data while cache size is over it's limits.
    type ICache interface {
    	IStore
    
    	// Set returns wasSet means this key needs to synchronous, err means set failed.
    	Set(key string, v []byte) SetResult
    
    	// Get
    	Get(key string) ([]byte, error)
    
    	// Del if any error is returned, core.Core would not trigger synchronous of the cache.
    	Del(key string) SetResult
    }
    
    // SetResult represents what operations would be caused by ICache.Set, operations include:
    //
    // 1. NeedSync tells users that them should trigger setting apply.
    // 2. NeedDeleteKey should be set while Cache-Replacing happened or core need to delete cache by itself.
    //
    type SetResult struct {
    	err           error
    	NeedSync      bool
    	NeedDeleteKey string
    }
    
    opened by yeqown 0
  • REFACTOR: cassemdb bootstrap procedure refactoring

    REFACTOR: cassemdb bootstrap procedure refactoring

    Now, cassemdb bootstrapped with fixed configuration of cassemdb cluster and it's members. But it should be another way:

    1. root node1 bootstrap or recover the cluster.
    2. node-2 to node-n join to the cluster which contains node1 only.
    enhancement good first issue 
    opened by yeqown 0
Releases(v0.2.1)
Owner
Yeqllo
Want more
Yeqllo
A simple golang api generator that stores struct fields in key/value based databases

Backgen A simple golang API generator that uses key/value based databases. It does not provide the database itself, only uses a interface to access se

null 0 Feb 4, 2022
A course to build the SQL layer of a distributed database.

TinySQL TinySQL is a course designed to teach you how to implement a distributed relational database in Go. TinySQL is also the name of the simplifed

TiDB Incubator 931 May 13, 2022
Distributed reliable key-value store for the most critical data of a distributed system

etcd Note: The master branch may be in an unstable or even broken state during development. Please use releases instead of the master branch in order

etcd-io 39.9k May 17, 2022
🛡️ Synchronize AdGuard Home config to replicas

AdGuardHome sync Synchronize AdGuardHome config to a replica instance. Current sync features General Settings Filters Rewrites Services Clients Instal

Marc Brugger 175 May 11, 2022
Parsing of numeric ranges from string

expand-range Parsing of numeric ranges from string. Convert "1,3-5" into [1,3,4,5]. Installation go get -u github.com/n0madic/expand-range Usage Impor

Nomadic 0 Nov 28, 2021
A rudimentary implementation of a basic document (NoSQL) database in Go

tiedot Documentation Keywords: Golang, go, document database, NoSQL, JSON tiedot - Your NoSQL database powered by Golang tiedot is a document database

Howard 2.7k May 19, 2022
Beerus-DB: a database operation framework, currently only supports Mysql, Use [go-sql-driver/mysql] to do database connection and basic operations

Beerus-DB · Beerus-DB is a database operation framework, currently only supports Mysql, Use [go-sql-driver/mysql] to do database connection and basic

Beerus 6 Mar 5, 2022
OcppManager-go - A library for dynamically managing OCPP configuration (variables). It can read, update, and validate OCPP variables.

?? ocppManager-go A library for dynamically managing OCPP configuration (variables). It can read, update, and validate OCPP variables. Currently, only

Blaž 0 Jan 3, 2022
Eventually consistent distributed in-memory cache Go library

bcache A Go Library to create distributed in-memory cache inside your app. Features LRU cache with configurable maximum keys Eventual Consistency sync

Iwan Budi Kusnanto 86 May 12, 2022
CockroachDB - the open source, cloud-native distributed SQL database.

CockroachDB is a cloud-native SQL database for building global, scalable cloud services that survive disasters. What is CockroachDB? Docs Quickstart C

CockroachDB 24.6k May 22, 2022
The lightweight, distributed relational database built on SQLite.

rqlite is a lightweight, distributed relational database, which uses SQLite as its storage engine. Forming a cluster is very straightforward, it grace

rqlite 10.3k May 15, 2022
TiDB is an open source distributed HTAP database compatible with the MySQL protocol

Slack Channel Twitter: @PingCAP Reddit Mailing list: lists.tidb.io For support, please contact PingCAP What is TiDB? TiDB ("Ti" stands for Titanium) i

PingCAP 31.3k May 18, 2022
A distributed key-value store. On Disk. Able to grow or shrink without service interruption.

Vasto A distributed high-performance key-value store. On Disk. Eventual consistent. HA. Able to grow or shrink without service interruption. Vasto sca

Chris Lu 237 Apr 29, 2022
A distributed MySQL binlog storage system built on Raft

What is kingbus? 中文 Kingbus is a distributed MySQL binlog store based on raft. Kingbus can act as a slave to the real master and as a master to the sl

Fei Chen 843 May 12, 2022
Distributed cache with gossip peer membership enrollment.

Autocache Groupcache enhanced with memberlist for distributed peer discovery. TL;DR See /_example/ for usage. Run docker-compose -f _example/docker-co

null 95 Feb 19, 2022
LBADD: An experimental, distributed SQL database

LBADD Let's build a distributed database. LBADD is an experimental distributed SQL database, written in Go. The goal of this project is to build a dat

Tom Arrell 376 Mar 15, 2022
TalariaDB is a distributed, highly available, and low latency time-series database for Presto

TalariaDB is a distributed, highly available, and low latency time-series database that stores real-time data. It's built on top of Badger DB.

Grab 92 Apr 17, 2022
Distributed RWMutex in Go

Distributed Read-Write Mutex in Go The default Go implementation of sync.RWMutex does not scale well to multiple cores, as all readers contend on the

Jon Gjengset 321 Apr 20, 2022
A distributed key value store in under 1000 lines. Used in production at comma.ai

minikeyvalue Fed up with the complexity of distributed filesystems? minikeyvalue is a ~1000 line distributed key value store, with support for replica

George Hotz 2.3k May 18, 2022