Distributed Lab 2: RPC in Go

Overview

Distributed Lab 2: RPC in Go

Using the lab sheet

There are two ways to use the lab sheet, you can either:

Ratings

Each question is rated to help you balance your work:

  • 🔴 - Easy, strictly necessary.
  • 🔴 🔴 - Medium, still necessary.
  • 🔴 🔴 🔴 - Hard, necessary if you're aiming for higher marks.
  • 🔴 🔴 🔴 🔴 - Hard, useful for coursework extensions.
  • 🔴 🔴 🔴 🔴 🔴 - Hard, beyond what you need for any part of the coursework.

1: Deploying the Chat System 🔴

Following from the end of last week's lab, you should now have a distributed system -- a chat client and server. Hopefully you've already tested these out locally, but now you need to put them to the test by running them in a properly distributed fashion.

For this, we will make use of AWS. Follow the guides for setting up instances, accessing them, and opening required ports.

In particular, you should:

  • create two t2.micro instances, (guide)
  • make sure Go is installed on your instances (guide)
  • load your client/server code onto the instances (e.g., via git)
  • open ports to allow communication between instances (guide)
  • start your server running on one instance (ssh guide)
  • connect to your server with your client on another instance
  • connect to your server from your local machine
  • communicate between clients via your (genuinely) distributed system!

Make sure you stop your instances when you have finished!

Frequently Asked Questions

  • My code worked locally but it doesn't on AWS: The main reason for this happening is incorrect IP addresses. To connect to the server from another AWS instance, you will need to use the private IP address. To connect to the server from a non-AWS computer (i.e. your personal computer) you need to use the public IP address. Think about why this may be the case, how do private and public IP addresses work?

  • I use go run client.go and get panic: runtime error: invalid memory address or nil pointer dereference: You are not passing in the IP address of the server to the client. Run it like this go run client.go -ip ip:port replacing ip and port with the appropriate values.

  • What is git? How do I copy files with it? Git is a version control system, it allows you to keep track of your changes. When we suggest loading files to AWS, we are suggesting that you should use a git repository and pushing your changes to the remote repository, then cloning/pulling these changes to your AWS instance. GitHub, GitLab and many others are services that allow for this process. Still stuck? You can try using SCP - we do not recommend this though. You should be familiar with git at this point.

2: Using RPC - Secret Strings 🔴

Follow the video from this week to create a simple RPC system that allows clients to call the functions of a server via a defined interface. As with last week, this is best attempted in stages:

  • Stage 1: Write server code to enable access to the "secret" string manipulation function. Test it by writing a client that sends a string to be reversed.
  • Stage 2: Enable Premium Tier service by implementing the FastReverse function in the server.
  • Stage 3: Update your client to read words from the wordlist file and reverse them all.
  • (Optional) run multiple server instances, and speed up the work by having your client split the load between servers.

3: 99 Bottles of Beer 🔴 🔴 🔴

So far we've been focusing on client-server systems. However, there are times when we don't really want a distinction in roles between components -- when we might want them to act as peers, all running the same code.

For this task, you are going to solve the "99 Bottles of Beer" problem in a distributed fashion. Rather than a single process singing the lyrics all by itself, you're going to run at least three instances of your code, on different machines, and have these buddies share the work of singing the song verse-by-verse, in order.

Here's how it should go:

  • Buddy 1: "99 bottles of beer on the wall, 99 bottles of beer. Take one down, pass it around..."
  • Buddy 2: "98 bottles of beer on the wall, 98 bottles of beer. Take one down, pass it around..."
  • Buddy 3: "97 bottles of beer on the wall, 97 bottles of beer. Take one down, pass it around..."
  • Buddy 1: "96 bottles of beer on the wall, 96 bottles of beer. Take one down, pass it around..."

All the way down until there are no more bottles of beer on the wall, which the final singer should note appropriately with your preferred ending to the song.

Use 3 for testing, but your solution should be able to handle any number of buddies singing along. There should be no difference in the code, just different flags on the commandline.

Hints:

  1. Each process needs to accept an ip:port string for the 'next' buddy who will follow on from them in the song. You'll have to configure them in a loop.
  2. You don't want clients to try connect to each other straight away, or you won't have time to set the final process running so that the first can connect.
  3. When you set up the processes, you'll also need some way to indicate which of them should start the song (I suggest allowing any n bottles of beer, for testing purposes). Only the last process you set up should need to be told the n to count down from.
  4. You may need to look at client.Go rather than client.Call.

Don't forget to stop any AWS instances!

Distributed lock manager. Warning: very hard to use it properly. Not because it's broken, but because distributed systems are hard. If in doubt, do not use this.

What Dlock is a distributed lock manager [1]. It is designed after flock utility but for multiple machines. When client disconnects, all his locks are

Sergey Shepelev 25 Dec 24, 2019
Distributed reliable key-value store for the most critical data of a distributed system

etcd Note: The main branch may be in an unstable or even broken state during development. For stable versions, see releases. etcd is a distributed rel

etcd-io 40.4k Jun 30, 2022
Simple, fast and scalable golang rpc library for high load

gorpc Simple, fast and scalable golang RPC library for high load and microservices. Gorpc provides the following features useful for highly loaded pro

Aliaksandr Valialkin 651 Jun 20, 2022
The Go language implementation of gRPC. HTTP/2 based RPC

gRPC-Go The Go implementation of gRPC: A high performance, open source, general RPC framework that puts mobile and HTTP/2 first. For more information

grpc 16.3k Jun 24, 2022
Hprose is a cross-language RPC. This project is Hprose for Golang.

Hprose 3.0 for Golang Introduction Hprose is a High Performance Remote Object Service Engine. It is a modern, lightweight, cross-language, cross-platf

Hprose 1.2k Jun 17, 2022
The jsonrpc package helps implement of JSON-RPC 2.0

jsonrpc About Simple, Poetic, Pithy. No reflect package. But reflect package is used only when invoke the debug handler. Support GAE/Go Standard Envir

Osamu TONOMORI 166 Jun 23, 2022
A simple go implementation of json rpc 2.0 client over http

JSON-RPC 2.0 Client for golang A go implementation of an rpc client using json as data format over http. The implementation is based on the JSON-RPC 2

Alexander Gehres 232 Jun 22, 2022
A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services

sleuth sleuth is a Go library that provides master-less peer-to-peer autodiscovery and RPC between HTTP services that reside on the same network. It w

null 352 Jun 21, 2022
distributed data sync with operational transformation/transforms

DOT The DOT project is a blend of operational transformation, CmRDT, persistent/immutable datastructures and reactive stream processing. This is an im

DOT & Chain 69 May 11, 2022
High performance, distributed and low latency publish-subscribe platform.

Emitter: Distributed Publish-Subscribe Platform Emitter is a distributed, scalable and fault-tolerant publish-subscribe platform built with MQTT proto

emitter 3.3k Jun 26, 2022
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.

Gleam Gleam is a high performance and efficient distributed execution system, and also simple, generic, flexible and easy to customize. Gleam is built

Chris Lu 3.1k Jun 22, 2022
Go Micro is a framework for distributed systems development

Go Micro Go Micro is a framework for distributed systems development. Overview Go Micro provides the core requirements for distributed systems develop

Asim Aslam 18.6k Jun 25, 2022
Simplified distributed locking implementation using Redis

redislock Simplified distributed locking implementation using Redis. For more information, please see examples. Examples import ( "fmt" "time"

Black Square Media 721 Jun 20, 2022
A distributed lock service in Go using etcd

locker A distributed lock service client for etcd. What? Why? A distributed lock service is somewhat self-explanatory. Locking (mutexes) as a service

James Gregory 48 Dec 2, 2021
Skynet is a framework for distributed services in Go.

##Introduction Skynet is a communication protocol for building massively distributed apps in Go. It is not constrained to Go, so it will lend itself n

null 2k Jun 12, 2022
Go Open Source, Distributed, Simple and efficient Search Engine

Go Open Source, Distributed, Simple and efficient full text search engine.

ego 6.1k Jun 25, 2022
Dapr is a portable, event-driven, runtime for building distributed applications across cloud and edge.

Dapr is a portable, serverless, event-driven runtime that makes it easy for developers to build resilient, stateless and stateful microservices that run on the cloud and edge and embraces the diversity of languages and developer frameworks.

Dapr 18.4k Jun 28, 2022
A distributed, proof of stake blockchain designed for the financial services industry.

Provenance Blockchain Provenance is a distributed, proof of stake blockchain designed for the financial services industry.

Provenance Blockchain, Inc. 55 Jun 23, 2022