In memory Key/Value store in go using gRPC.

Related tags

Network gRPC-cache
Overview

In memory cache, using gRPC

Build Status Coverage Status Go Report Card Documentation

Contents

About

Go in memory cache using gRPC to generate API. Functionalities include

  • Adding/Replacing key/value
  • Getting value using a key
  • Getting all keys
  • Deleting a particular key
  • Deleting all keys
  • Concurrency safe and on client side calls can be made using goroutines

Running server

You can run server locally the following ways

  • Using go run
  • Docker
  • Kubernetes
  • Helm
  • Terraform

Local

You can run the server locally but make sure you have the following requirements

Now you can start the server using below commands

Start your server `./server-cache` or `make server`
`./server-cache -addr=":5001"` to run server on port `5001`

Docker

Make sure you have following installed

After the basic requirements are met, you can easily run server from below command.

  • Run this command make docker // This will build the image
  • Run the server using make dockerServer

After running the server, start your client ./client-cache or make client in a different terminal

./client-cache -addr=":5001" is server is running running on port 5001

Kubernetes

You need to have following things installed on your OS to run server using kubernetes

Important:- Make sure your ingress-controller in running. To enable in minikube

minikube addons enable ingress

You can run server on your kubernetes cluster. All the resources are created on grpc-cache namespace. Before running the command below make sure your cluster is up and running.

  • Run this command make run-k8s-server

This will create all the required resources needed to run your grpc server. You will be asked for passowrd once, to enter ingress address to host mapping automatically to /etc/hosts

Example

# To list all services exposed by a server, use the "list" verb.
grpcurl --insecure grpc-cache.example.com:443 list

# The "describe" verb will print the type of any symbol that the server knows about or that is found in a given protoset file. It also prints a description of that symbol, in the form of snippets of proto source. It won't necessarily be the original source that defined the element, but it will be equivalent.
grpcurl --insecure grpc-cache.example.com:443 list cacheService.CacheService

# To add a key
grpcurl --insecure -d '{"key": "knrt10", "value": "pro", "expiration": "3m"}' grpc-cache.example.com:443 cacheService.CacheService/Add

# To get key
grpcurl --insecure -d '{"key": "knrt10"}' grpc-cache.example.com:443 cacheService.CacheService/Get

# To get all keys
grpcurl --insecure grpc-cache.example.com:443 cacheService.CacheService/GetAllItems

Similarly you can use all the methods as shown in API below

Helm

You need to have following things installed on your OS to run server using helm

Important:- Make sure your ingress-controller in running. To enable in minikube

minikube addons enable ingress

Just run make run-helm-server and it will deploy your application to your kubernetes cluster. This will create all the required resources needed to run your grpc server. You will be asked for passowrd once, to enter ingress address to host mapping automatically to /etc/hosts

Now you can easily test it using grpcurl. For API usage you can refer to example

Terraform

You need to have following things installed on your OS to run server using terraform

Important:- Make sure your ingress-controller in running. To enable in minikube

minikube addons enable ingress

Running with Terraform is the most easiest way. Just run make run-terraform-server and it will deploy your application to your kubernetes cluster.

Now you can easily test it using grpcurl. For API usage you can refer to example

API

Proto syntax proto3 is used. You can find the proto file here

Add

This is used to add key/value to the cache

func (c Cache) Add(ctx context.Context, item *api.Item) (*api.Item, error)

Get

This is used to get key value pair for a particular key

func (c Cache) Get(ctx context.Context, args *api.GetKey) (*api.Item, error)

GetByPrefix

Used to get all key value pairs by prefix

func (c Cache) GetByPrefix(ctx context.Context, args *api.GetKey) (*api.AllItems, error)

GetAllItems

Used to get all key value pairs

func (c Cache) GetAllItems(ctx context.Context, in *empty.Empty) (*api.AllItems, error)

DeleteKey

Used to delete item by a particular key from the cache

func (c Cache) DeleteKey(ctx context.Context, args *api.GetKey) (*api.Success, error)

DeleteAll

Used to clear the whole cache

func (c Cache) DeleteAll(ctx context.Context, in *empty.Empty) (*api.Success, error)

Testing

After running make build just run make test to run the tests. It has coverage of 92.7%

go test api/server/* -v -cover -race
=== RUN   TestAdd
--- PASS: TestAdd (0.03s)
=== RUN   TestGet
--- PASS: TestGet (0.01s)
=== RUN   TestGetByPrefix
--- PASS: TestGetByPrefix (0.01s)
=== RUN   TestGetAllItems
--- PASS: TestGetAllItems (0.01s)
=== RUN   TestDeleteKey
--- PASS: TestDeleteKey (0.00s)
=== RUN   TestDeleteAll
--- PASS: TestDeleteAll (0.00s)
=== RUN   TestGetDeletedKey
--- PASS: TestGetDeletedKey (0.01s)
=== RUN   TestDeleteKeyByExpiration
--- PASS: TestDeleteKeyByExpiration (2.01s)
PASS
coverage: 92.7% of statements
ok  	command-line-arguments	3.709s	coverage: 92.7% of statements

Development

You can develop and debug the application locally, but you need to have below softwares installed locally

Once you've cloned this repo, run these commands in this directory:

# Only needed the first time:
$ make build

# Then run to start server
$ ./server-cache --help

Usage of ./server-cache:
  -addr string
      Address on which you want to run server (default ":5001")
  -cln int
      Cleanup interval duration of expired cache is 5 min (default 5)
  -exp int
      Default expiration duration of cache is 10 min (default 10)

# To use client
$ ./client-cache --help

Usage of ./client-cache:
  -addr string
      Address on which you want to run server (default ":5001")

Example

Please refer to examples directory for more information

You might also like...
A suite of gRPC debugging tools. Like Fiddler/Charles but for gRPC.

grpc-tools A suite of tools for gRPC debugging and development. Like Fiddler/Charles but for gRPC! The main tool is grpc-dump which transparently inte

grpc-http1: A gRPC via HTTP/1 Enabling Library for Go

grpc-http1: A gRPC via HTTP/1 Enabling Library for Go This library enables using all the functionality of a gRPC server even if it is exposed behind a

Server and client implementation of the grpc go libraries to perform unary, client streaming, server streaming and full duplex RPCs from gRPC go introduction

Description This is an implementation of a gRPC client and server that provides route guidance from gRPC Basics: Go tutorial. It demonstrates how to u

Go based grpc - grpc gateway micro service example

go-grpc-gateway-server This repository provides an example for go based microservice. Go micro services developed based on gRPC protobuf's and also us

Simple grpc web and grpc transcoding with Envoy
Simple grpc web and grpc transcoding with Envoy

gRPC Web and gRPC Transcoding with Envoy This is a simple stand-alone set of con

Go-grpc - This is grpc server for golang.

go-grpc This is grpc server for golang. protocのインストール brew install protoc Golang用のプラグインのインストール go install google.golang.org/protobuf/cmd/protoc-gen-go

GRPC - Creating a gRPC service from scratch

#Go gRPC services course Creating a gRPC service from scratch Command line colle

Totem - A Go library that can turn a single gRPC stream into bidirectional unary gRPC servers

Totem is a Go library that can turn a single gRPC stream into bidirectional unar

Todo-app-grpc - Go/GRPC codebase containing RealWorld examples (CRUD, auth, advanced patterns, etc)

Go/GRPC codebase containing RealWorld examples (CRUD, auth, advanced patterns, e

Comments
  • Documentation bug

    Documentation bug

    https://github.com/knrt10/gRPC-cache/blob/4136cb10cfd825cda858f2837d5cca6dec2bb017/api/server/cache_service.go#L35 says that you ignore the values if they are less than one but both of the places where you check you check if they are less than 0 AFAICT.

    opened by WesternStar 1
Owner
Kautilya Tripathi
CKA | Distributed Systems | Systems Programming | OSS ❤️
Kautilya Tripathi
Tapestry is an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate objects. This distributed system provides an interface for storing and retrieving key-value pairs.

Tapestry This project implements Tapestry, an underlying distributed object location and retrieval system (DOLR) which can be used to store and locate

Han Cai 1 Mar 16, 2022
protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript clients that connect the web frontend and golang backend fronted by grpc-gateway.

protoc-gen-grpc-gateway-ts protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript

gRPC Ecosystem 94 Dec 19, 2022
Backend implementation using go, proto3 and gRPC for a mock online store

Backend implementation using go, proto3 and gRPC for a mock online store Ricardo RICO URIBE Tasks I - Order service The current system exposes a produ

Ricardo Rico 0 Oct 10, 2021
Using Envoy Proxy to load-balance gRPC services on GKE with header value based Session Affinity

Using Envoy Proxy to load-balance gRPC services on GKE with header value based S

Daniel William Clarke 2 Aug 24, 2022
Grpc-gateway-map-null - gRPC Gateway test using nullable values in map

Demonstrate gRPC gateway behavior with nullable values in maps Using grpc-gatewa

null 1 Jan 6, 2022
GRPC - A client-server mockup, using gRPC to expose functionality.

gRPC This is a mockup application that I built to help me visualise and understand the basic concepts of gRPC. In this exchange, the client can use a

Fergal Bittles 0 Jan 4, 2022
Grpc-train - Train booking demo using gRPC

gRPC Demo: Train Booking Service Description Usage Contributing Development Tool

Fadi Asfour 0 Feb 6, 2022
Go-grpc-tutorial - Simple gRPC server/client using go

Simple gRPC server/client using go Run server go run usermgmt_server/usermgmt_

Renner Poveda 0 Feb 14, 2022
Use Consul to do service discovery, use gRPC +kafka to do message produce and consume. Use redis to store result.

目录 gRPC/consul/kafka简介 gRPC+kafka的Demo gRPC+kafka整体示意图 限流器 基于redis计数器生成唯一ID kafka生产消费 kafka生产消费示意图 本文kafka生产消费过程 基于pprof的性能分析Demo 使用pprof统计CPU/HEAP数据的

null 43 Jul 9, 2022
Traefik proxy plugin to extract HTTP header value and create a new header with extracted value

Copy header value Traefik plugin Traefik plugin that copies HTTP header value with format key1=value1; key2=value2 into a new header. Motivation for t

Argyle 3 May 26, 2022