BlobStore is a highly reliable,highly available and ultra-large scale distributed storage system

Overview

BlobStore

Overview

BlobStore is a highly reliable,highly available and ultra-large scale distributed storage system. The system adopts Reed-Solomon code, which provides higher data durability with less storage cost than use three copies backup technology, and supports multiple erasure code modes multiple availability zones,and optimizes for small file to meet the storage needs of different scenarios. Some key features of BlobStore include:

  • ultra-large scale
  • high reliability
  • flexible deployment
  • low cost

Documents

English version: https://cubefs.readthedocs.io/en/latest/

Chinese version: https://cubefs.readthedocs.io/zh_CN/latest/

Build BlobStore

$ git clone http://github.com/cubefs/blobstore.git
$ cd blobstore
$ source env.sh
$ ./build.sh 

Deploy BlobStore

For more details please refer to documentation.

Manage BlobStore

For more details please refer to documentation.

License

BlobStore is licensed under the Apache License, Version 2.0. For detail see LICENSE and NOTICE.

You might also like...
Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

The TinyKV course builds a key-value storage system with the Raft consensus algorithm.
The TinyKV course builds a key-value storage system with the Raft consensus algorithm.

The TinyKV Course The TinyKV course builds a key-value storage system with the Raft consensus algorithm. It is inspired by MIT 6.824 and TiKV Project.

Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

Gleam Gleam is a high performance and efficient distributed execution system, and also simple, generic, flexible and easy to customize. Gleam is built

💡 A Distributed and High-Performance Monitoring System.  The next generation of Open-Falcon
💡 A Distributed and High-Performance Monitoring System. The next generation of Open-Falcon

夜莺简介 夜莺是一套分布式高可用的运维监控系统,最大的特点是混合云支持,既可以支持传统物理机虚拟机的场景,也可以支持K8S容器的场景。同时,夜莺也不只是监控,还有一部分CMDB的能力、自动化运维的能力,很多公司都基于夜莺开发自己公司的运维平台。开源的这部分功能模块也是商业版本的一部分,所以可靠性有保

A distributed and coördination-free log management system
A distributed and coördination-free log management system

OK Log is archived I hoped to find the opportunity to continue developing OK Log after the spike of its creation. Unfortunately, despite effort, no su

JuiceFS is a distributed POSIX file system built on top of Redis and S3.
JuiceFS is a distributed POSIX file system built on top of Redis and S3.

JuiceFS is a high-performance POSIX file system released under GNU Affero General Public License v3.0. It is specially optimized for the cloud-native

Distributed-system - Practicing and learning the foundations of DS with Go

Distributed-System For practicing and learning the foundations of distributed sy

A distributed system for embedding-based retrieval
A distributed system for embedding-based retrieval

Overview Vearch is a scalable distributed system for efficient similarity search of deep learning vectors. Architecture Data Model space, documents, v

a dynamic configuration framework used in distributed system
a dynamic configuration framework used in distributed system

go-archaius This is a light weight configuration management framework which helps to manage configurations in distributed system The main objective of

Comments
  • blocking key deadlock problem

    blocking key deadlock problem

    问题场景:

    if kl == nil {
    		l.lock.RUnlock()
    		l.lock.Lock()
    		kl = newBlocker(l.limit)
    		l.keyMap[key] = kl
    		kl.addRef()
    		l.lock.Unlock()
    
    1. In a concurrent scenario, when aquire a single key, the obtained key maybe nil and no secondary check is performed, which may result in the same key being overwritten.
    if kl.loadRef() < 0 {
    			l.lock.Unlock()
    			panic("internal error: refs < 0")
    		}
    		if kl.loadRef() == 0 {
    			delete(l.keyMap, key)
    		}
    		kls = append(kls, kl)
    	}
    	for _, kl := range kls {
    		kl.release()
    	}
    
    1. When the key is released, there is a problem with the reference count judgment, resulting in the repeated release of the key's lock
    opened by JasonHu520 0
Owner
CubeFS
CubeFS
Easy to use Raft library to make your app distributed, highly available and fault-tolerant

An easy to use customizable library to make your Go application Distributed, Highly available, Fault Tolerant etc... using Hashicorp's Raft library wh

Richard Bertok 59 Sep 5, 2022
Ultra performant API Gateway with middlewares

The KrakenD framework An open framework to assemble ultra performance API Gateways with middlewares; core service of the KrakenD API Gateway. Looking

Devops Faith - Open source for DevOps 5.2k Sep 26, 2022
Asynq: simple, reliable, and efficient distributed task queue in Go

Asynq: simple, reliable, and efficient distributed task queue in Go

Ken Hibino 4.3k Sep 28, 2022
Distributed-Services - Distributed Systems with Golang to consequently build a fully-fletched distributed service

Distributed-Services This project is essentially a result of my attempt to under

Hamza Yusuff 6 Jun 1, 2022
A distributed MySQL binlog storage system built on Raft

What is kingbus? 中文 Kingbus is a distributed MySQL binlog store based on raft. Kingbus can act as a slave to the real master and as a master to the sl

Fei Chen 851 Sep 11, 2022
A distributed key-value storage system developed by Alibaba Group

Product Overview Tair is fast-access memory (MDB)/persistent (LDB) storage service. Using a high-performance and high-availability distributed cluster

Alibaba 2k Sep 23, 2022
Distributed disk storage database based on Raft and Redis protocol.

IceFireDB Distributed disk storage system based on Raft and RESP protocol. High performance Distributed consistency Reliable LSM disk storage Cold and

IceFireDB 921 Sep 24, 2022