MIT 6.824: Distributed Systems

Overview

MIT 6.824: Distributed Systems

Labs of MIT 6.824 Spring 2020

Course Description

MIT 6.824 is a core 12-unit graduate subject with lectures, readings, programming labs, an optional project, a mid-term exam, and a final exam. It will present abstractions and implementation techniques for engineering distributed systems. Major topics include fault tolerance, replication, and consistency. Much of the class consists of studying and discussing case studies of distributed systems.

GitHub go.mod Go version of a Go module

Lab1 MapReduce

  • MapReduce framework consists of a coordinator process that hands out tasks to workers and copes with failed workers for big data processing on large clusters.
  • Coordinator process
  • Worker process

Lab2 Raft

  • Raft is a consensus algorithm for managing a replicated log. It ensures that all the replica servers see the same log. Each replica executes client requests in log order, applying them to its local copy of the service's state. If a server fails but later recovers, Raft takes care of bringing its log up to date. Raft will continue to operate as long as at least a majority of the servers are alive and can talk to each other. If there is no such majority, Raft will make no progress, but will pick up where it left off as soon as a majority can communicate again.
  • Part A - Raft leader election and heartbeats
  • Part B - Append new log entries
  • Part C - Write Raft's persistent state each time it changed, and read the state back when restarting after a reboot

Lab3 Fault-tolerant Key/Value Service

  • A strongly consistent replication database based on Raft consensus algorithm, which supports Get/Put/Append operations and is available under all non-Byzantine conditions as long as a majority of the servers are operational and can communicate with each others.
  • Part A - Key/Value service without log compaction
  • Part B - Key/Value service with log compaction

Lab4 Sharded Key/Value Service

  • A database with architecture resembling Google BigTable for processing client operations in parallel on a set of replica groups to enhance performance; the system is able to shift the assignment of shards among replica groups automatically for load balancing.
  • Part A - Shard Master
  • Part B - Sharded Key/Value Server
  • Challenge1 - Garbage collection of state
  • Challenge2 - Client requests during configuration changes
Owner
CC Chang
CC Chang
MIT6.824 Distributed Systems

MIT6.824-Distributed-Systems My Solutions for MIT6.824

null 0 Jan 28, 2022
Distributed-Services - Distributed Systems with Golang to consequently build a fully-fletched distributed service

Distributed-Services This project is essentially a result of my attempt to under

Hamza Yusuff 6 Jun 1, 2022
Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

Sergey Shepelev 25 Dec 24, 2019
null 1 Feb 3, 2022
Go Micro is a framework for distributed systems development

Go Micro Go Micro is a framework for distributed systems development. Overview Go Micro provides the core requirements for distributed systems develop

Asim Aslam 18.6k Jul 6, 2022
Go Micro is a standalone framework for distributed systems development

Go Micro Go Micro is a framework for distributed systems development. Overview Go Micro provides the core requirements for distributed systems develop

Asim Aslam 18.6k Jul 1, 2022
Tarmac is a unique framework designed for the next generation of distributed systems

Framework for building distributed services with Web Assembly

Benjamin Cane 192 Jun 26, 2022
A distributed systems library for Kubernetes deployments built on top of spindle and Cloud Spanner.

hedge A library built on top of spindle and Cloud Spanner that provides rudimentary distributed computing facilities to Kubernetes deployments. Featur

null 21 Jan 4, 2022
Distributed Systems 2021 - Miniproject 3

Distributed Systems 2021 -- Miniproject 3 Hand-in Date: 1 December 2021 (at 23:59) What to submit on learnit: a single zip-compressed file containing:

null 0 Dec 11, 2021
Distributed reliable key-value store for the most critical data of a distributed system

etcd Note: The main branch may be in an unstable or even broken state during development. For stable versions, see releases. etcd is a distributed rel

etcd-io 40.4k Jun 30, 2022
distributed data sync with operational transformation/transforms

DOT The DOT project is a blend of operational transformation, CmRDT, persistent/immutable datastructures and reactive stream processing. This is an im

DOT & Chain 71 Jul 1, 2022
High performance, distributed and low latency publish-subscribe platform.

Emitter: Distributed Publish-Subscribe Platform Emitter is a distributed, scalable and fault-tolerant publish-subscribe platform built with MQTT proto

emitter 3.3k Jul 4, 2022
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

Gleam Gleam is a high performance and efficient distributed execution system, and also simple, generic, flexible and easy to customize. Gleam is built

Chris Lu 3.1k Jun 28, 2022
Simplified distributed locking implementation using Redis

redislock Simplified distributed locking implementation using Redis. For more information, please see examples. Examples import ( "fmt" "time"

Black Square Media 729 Jun 30, 2022
A distributed lock service in Go using etcd

locker A distributed lock service client for etcd. What? Why? A distributed lock service is somewhat self-explanatory. Locking (mutexes) as a service

James Gregory 48 Dec 2, 2021
Skynet is a framework for distributed services in Go.

##Introduction Skynet is a communication protocol for building massively distributed apps in Go. It is not constrained to Go, so it will lend itself n

null 2k Jun 12, 2022
Go Open Source, Distributed, Simple and efficient Search Engine

Go Open Source, Distributed, Simple and efficient full text search engine.

ego 6.1k Jul 4, 2022
Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.

Dapr is a portable, serverless, event-driven runtime that makes it easy for developers to build resilient, stateless and stateful microservices that run on the cloud and edge and embraces the diversity of languages and developer frameworks.

Dapr 18.4k Jun 28, 2022