Simple, fast and scalable golang rpc library for high load

Overview

gorpc

Simple, fast and scalable golang RPC library for high load and microservices.

Gorpc provides the following features useful for highly loaded projects with RPC:

  • It minimizes the number of connect() syscalls by pipelining request and response messages over a single connection.

  • It minimizes the number of send() syscalls by packing as much as possible pending requests and responses into a single compressed buffer before passing it into send() syscall.

  • It minimizes the number of recv() syscalls by reading and buffering as much as possible data from the network.

  • It supports RPC batching, which allows preparing multiple requests and sending them to the server in a single batch.

These features help the OS minimizing overhead (CPU load, the number of TCP connections in TIME_WAIT and CLOSE_WAIT states, the number of network packets and the amount of network bandwidth) required for RPC processing under high load.

Additionally gorpc provides the following features missing in net/rpc:

  • Client automatically manages connections and automatically reconnects to the server on connection errors.
  • Client supports response timeouts.
  • Client supports RPC batching.
  • Client supports async requests' canceling.
  • Client prioritizes new requests over old pending requests if server fails to handle the given load.
  • Client detects stuck servers and immediately returns error to the caller.
  • Client supports fast message passing to the Server, i.e. requests without responses.
  • Both Client and Server provide network stats and RPC stats out of the box.
  • Commonly used RPC transports such as TCP, TLS and unix socket are available out of the box.
  • RPC transport compression is provided out of the box.
  • Server provides graceful shutdown out of the box.
  • Server supports RPC handlers' councurrency throttling out of the box.
  • Server may pass client address to RPC handlers.
  • Server gracefully handles panic in RPC handlers.
  • Dispatcher accepts functions as RPC handlers.
  • Dispatcher supports registering multiple receiver objects of the same type under distinct names.
  • Dispatcher supports RPC handlers with zero, one (request) or two (client address and request) arguments and zero, one (either response or error) or two (response, error) return values.

Dispatcher API provided by gorpc allows easily converting usual functions and/or struct methods into RPC versions on both client and server sides. See Dispatcher examples for more details.

By default TCP connections are used as underlying gorpc transport. But it is possible using arbitrary underlying transport - just provide custom implementations for Client.Dial and Server.Listener. RPC authentication, authorization and encryption can be easily implemented via custom underlying transport and/or via OnConnect callbacks. Currently gorpc provides TCP, TLS and unix socket transport out of the box.

Currently gorpc with default settings is successfully used in highly loaded production environment serving up to 40K qps. Switching from http-based rpc to gorpc reduced required network bandwidth from 300 Mbit/s to 24 Mbit/s.

Docs

See http://godoc.org/github.com/valyala/gorpc .

Usage

Server:

s := &gorpc.Server{
	// Accept clients on this TCP address.
	Addr: ":12345",

	// Echo handler - just return back the message we received from the client
	Handler: func(clientAddr string, request interface{}) interface{} {
		log.Printf("Obtained request %+v from the client %s\n", request, clientAddr)
		return request
	},
}
if err := s.Serve(); err != nil {
	log.Fatalf("Cannot start rpc server: %s", err)
}

Client:

c := &gorpc.Client{
	// TCP address of the server.
	Addr: "rpc.server.addr:12345",
}
c.Start()

// All client methods issuing RPCs are thread-safe and goroutine-safe,
// i.e. it is safe to call them from multiple concurrently running goroutines.
resp, err := c.Call("foobar")
if err != nil {
	log.Fatalf("Error when sending request to server: %s", err)
}
if resp.(string) != "foobar" {
	log.Fatalf("Unexpected response from the server: %+v", resp)
}

Both client and server collect connection stats - the number of bytes read / written and the number of calls / errors to send(), recv(), connect() and accept(). This stats is available at Client.Stats and Server.Stats.

See tests for more usage examples.

Comments
  • Simpler compression library

    Simpler compression library

    I tried to create simple streaming compression library for this project

    https://github.com/funny-falcon/go-funlz

    Format derived from lzf. It is byte oriented, minimum compressed chunk is 4 bytes, backref window is 4096 bytes. It uses circular buffer of 8096 bytes for buffering uncompressed bytes. It does no output buffering.

    Compression is just 1.5 faster than compress/flate, but decompression is up to 2-10 times faster.

    opened by funny-falcon 12
  • Improvements

    Improvements

    • use non-blocking select from single channel when possible such select doesn't need heap allocation nor complex logic
    • stop timer in client CallTimeout
    • detect empty requestsChan and responsesChan instead of timer to flush writers with default PendingRequests == PendingResponses == 32768 it is good both for throughput and latency (but i'll recomend to decrease default write buffers a bit).
    opened by funny-falcon 9
  • fix no delay writer loops

    fix no delay writer loops

    Benching before

    • with default flush delay
    PASS
    BenchmarkEchoInt10000Workers-4    300000          4327 ns/op
    BenchmarkEchoIntNocompress10000Workers-4      300000          3911 ns/op
    BenchmarkEchoString10000Workers-4     300000          4703 ns/op
    BenchmarkEchoStringNocompress10000Workers-4   300000          4380 ns/op
    BenchmarkEchoStruct10000Workers-4     200000          6127 ns/op
    BenchmarkEchoStructNocompress10000Workers-4   200000          5433 ns/op
    ok      _/home/yura/Projects/gorpc  13.575s
    
    • with disabled delay
    PASS
    BenchmarkEchoInt10000Workers-4    200000          8334 ns/op
    BenchmarkEchoIntNocompress10000Workers-4      300000          4466 ns/op
    BenchmarkEchoString10000Workers-4     200000          8551 ns/op
    BenchmarkEchoStringNocompress10000Workers-4   300000          4785 ns/op
    BenchmarkEchoStruct10000Workers-4     200000         10674 ns/op
    BenchmarkEchoStructNocompress10000Workers-4   200000          5675 ns/op
    ok      _/home/yura/Projects/gorpc  15.136s
    

    Benching after:

    • with default flush delay
    PASS
    BenchmarkEchoInt10000Workers-4    300000          4307 ns/op
    BenchmarkEchoIntNocompress10000Workers-4      300000          3856 ns/op
    BenchmarkEchoString10000Workers-4     300000          4643 ns/op
    BenchmarkEchoStringNocompress10000Workers-4   300000          4320 ns/op
    BenchmarkEchoStruct10000Workers-4     200000          6010 ns/op
    BenchmarkEchoStructNocompress10000Workers-4   200000          5348 ns/op
    ok      _/home/yura/Projects/gorpc  13.514s
    
    • with disabled flush delay
    PASS
    BenchmarkEchoInt10000Workers-4    300000          4230 ns/op
    BenchmarkEchoIntNocompress10000Workers-4      300000          3645 ns/op
    BenchmarkEchoString10000Workers-4     300000          4341 ns/op
    BenchmarkEchoStringNocompress10000Workers-4   300000          4029 ns/op
    BenchmarkEchoStruct10000Workers-4     200000          5884 ns/op
    BenchmarkEchoStructNocompress10000Workers-4   200000          5240 ns/op
    ok      _/home/yura/Projects/gorpc  13.057s
    

    edit: rerun bench with taskset -c 4,5,6,7

    opened by funny-falcon 7
  • windows call timeout

    windows call timeout

    go version go1.5.1 windows/amd64

    go test -run ExampleDispatcher_serviceCalls

    --- FAIL: ExampleDispatcher_serviceCalls (120.06s) got: Get=, gorpc.Client: [:7892]. Cannot obtain response during timeout=20s Get=, gorpc.Client: [:7892]. Cannot obtain response during timeout=20s Set=, gorpc.Client: [:7892]. Cannot obtain response during timeout=20s, 456

    GetError42=, gorpc.Client: [:7892]. Cannot obtain response during timeout=2 0s GetError42=, gorpc.Client: [:7892]. Cannot obtain response during timeout=2 0s DDDD=, gorpc.Client: [:7892]. Cannot obtain response during timeout=20s want: Get=123, Get=456, Set=, , 78 GetError42=78, GetError42=, error42 DDDD=13, FAIL exit status 1

    bug help wanted 
    opened by gyf19 3
  • Is this still a better choice than golang roc?

    Is this still a better choice than golang roc?

    I see the last commit is more than two years ago, so I would like to know whether this is better choice if I want to use it in high-load servers with large request size (~200K content per request on average).

    opened by 1a1a11a 2
  • Is it possible to add support of struct{}?

    Is it possible to add support of struct{}?

    In many cases sets implemented as map[T]struct{}. It would be nice to support it to avoid extra convertion on both server and client sides.

    Currently Dispatcher AddFunc panics:

    ... cannot contain struct without exported fields [struct {}] in the value of map...
    

    I could implement custom GobDecoder/GobDecoder for map[T]struct{}-type in my code, but it would be nice if gorpc supports it.

    opened by ikkeps 2
  • Dispatcher type validation is probably a bit too strict (bug)

    Dispatcher type validation is probably a bit too strict (bug)

    Hi,

    I was trying to pass a struct containing a time.Time object between the server and the client and I got the following panic just after running:

    gorpc.Dispatcher: response in the function [Foo.Bar] cannot contain struct without exported fields [time.Time] in the field [Start] of struct [foo.Result]

    It appears that the dispatcher, in the validateType function, checks for at least one exported field in each struct it sees and therefore reject time.Time which has none. This seems overly strict because time.Time implements the GobEncoder and GobDecoder interfaces even with no exported fields and therefore should be able to pass on the wire without problem.

    I am fairly new to Golang and therefore have no idea how this could be fixed but I am under the impression that it is a bug of gorpc and not an intended feature.

    Could you have a look, please?

    Best,

    HLG

    opened by hlegendre 2
  • Build is breaking under go older than 1.4

    Build is breaking under go older than 1.4

    as of a pull right now.

    ../../../valyala/gorpc/client.go:553: undefined: atomic.Value ../../../valyala/gorpc/server.go:187: undefined: atomic.Value ../../../valyala/gorpc/server.go:240: undefined: atomic.Value

    bug 
    opened by manishrjain 2
  • Dispatcher couples server and client

    Dispatcher couples server and client

    It seems that you need to register functions or services with the Dispatcher that are both used by the client and server. Because of this I'm not able to build the server and client code separately. I'd like to see the dispatcher decoupled from the server and client.

    I don't have any idea what is a nice way to implement this. Could you give directions, than I can create a pull request with the changes.

    opened by dwlnetnl 2
  • GoRPC on 32 bits platforms

    GoRPC on 32 bits platforms

    Hello,

    Since I ran into this issue and I cannot upgrade my machine for now, I am stuck with 32 bits mode. I think that the stats could just be made uint32 on 32 bits platforms and left as is on 64 bits ones.

    What do you think?

    Pierre

    opened by pierrec 2
  • Cannot decode request

    Cannot decode request

    Hi,

    When running example code from README:

    2015/08/11 12:31:48 Obtained request foobar from the client 127.0.0.1:62891
    2015/08/11 12:31:48 gorpc.Server: [127.0.0.1:62891]->[:12345]. Cannot decode request: [unexpected EOF]
    
    opened by krzysztofantczak 1
  • Add a conn != nil check

    Add a conn != nil check

    go version go1.13 linux/amd64 Ubuntu 18.04.3 LTS

    Т.к. в clientHandler() в рутине происходит conn, err = c.Dial(), то бывают такие случаи...

    panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x12ee814]

    goroutine 39 [running]: github.com/valyala/gorpc.clientHandleConnection(0xc0002d0000, 0x0, 0x0) /mnt/hdd/home/biter/go/src/github.com/valyala/gorpc/client.go:688 +0x2b4 github.com/valyala/gorpc.clientHandler(0xc0002d0000) /mnt/hdd/home/biter/go/src/github.com/valyala/gorpc/client.go:673 +0x4f6 created by github.com/valyala/gorpc.(*Client).Start /mnt/hdd/home/biter/go/src/github.com/valyala/gorpc/client.go:157 +0x241

    opened by biter777 0
  • the docs make me confused, is there any project for reference?

    the docs make me confused, is there any project for reference?

    I have seen the example_test.go, but I still feel confused about how to use gorpc. For exmaple, if my implementation of client and server is segregative, how can I use code like this d := NewDispatcher() s := NewTCPServer("127.0.0.1:12445", d.NewHandlerFunc()) c := NewTCPClient("127.0.0.1:12445") dc := d.NewFuncClient(c) And I feel confused that whether rpc call need a connection pool or not. Thanks!

    opened by beaulian 2
  • ConnStats.Snapshot example please

    ConnStats.Snapshot example please

    Just a bit of pseudo code would be great...

    d := gorpc.NewDispatcher()   
    s := gorpc.NewTCPServer("127.0.0.1:43216", d.NewHandlerFunc())
    s.Start()
    stats := ??? Snapshot()
    
    opened by goarchit 0
  • How to coordinate physically seperate servers and clients

    How to coordinate physically seperate servers and clients

    I'm sure I'm missing something extremely trivial here, but am stuck. I'd like to implement a server on one computer that has a registered service, call it "Ping", that can be called from a client on another server and respond with "Pong".

    Per https://godoc.org/github.com/valyala/gorpc#example-Dispatcher--FuncCalls the client wrapper is established as dc := d.NewFuncClient(c).

    My concern is that d is not defined in a client only applet.

    If I reference the NewTCPClient directly, with a call like c.Call("Ping") the server will error out with:

    gorpc.Dispatcher: unsupported request type received from the client: string

    Obviously I'm missing something basic, please clue me in.

    Server side pseudo code:

     d := gorpc.NewDispatcher()
     d.AddFunc("Ping", func () string {return "Pong"})
     s := gorpc.NewTCPServer("localhost:1234",d.NewHandlerFunc())
     s.Start()
    

    Client side pseudo code:

     c := gorpc.NewTCPClient("localhost:1234")
     c.Start()
     c.Call("Ping")
    

    I've also tried:

    d := gorpc.NewDispatcher()
    c := gorpc.NewTCPClient("localhost:1234")
    c.Start()
    dc := d.NewFuncClient(c)
    resp, err := dc.Call("Ping", nil)
    

    Which results in a " Error when reading handshake from client: [EOF]" error since no d.AddFunc() has been called.

    My brain is having problems groking that the client code would need any details beyond a function name that the server handles.

    I did find that if I used the 2nd approach and added a d.AddFunc("Ping", func() {}) prior to the dc assignment it works.

    Really not clear if that is the right way though or if I just hacked it... Comments please.

    opened by goarchit 2
Owner
Aliaksandr Valialkin
Working on @VictoriaMetrics
Aliaksandr Valialkin
A Go library for master-less peer-to-peer autodiscovery and RPC between HTTP services

sleuth sleuth is a Go library that provides master-less peer-to-peer autodiscovery and RPC between HTTP services that reside on the same network. It w

null 356 Sep 2, 2022
A simple go implementation of json rpc 2.0 client over http

JSON-RPC 2.0 Client for golang A go implementation of an rpc client using json as data format over http. The implementation is based on the JSON-RPC 2

Alexander Gehres 241 Sep 16, 2022
Flowgraph package for scalable asynchronous system development

flowgraph Getting Started go get -u github.com/vectaport/flowgraph go test Links Wiki Slides from Minneapolis Golang Meetup, May 22nd 2019 Overview F

Scott Johnston 52 Sep 5, 2022
Scalable, fault-tolerant application-layer sharding for Go applications

ringpop-go (This project is no longer under active development.) Ringpop is a library that brings cooperation and coordination to distributed applicat

Uber Open Source 747 Sep 15, 2022
Sandglass is a distributed, horizontally scalable, persistent, time sorted message queue.

Sandglass is a distributed, horizontally scalable, persistent, time ordered message queue. It was developed to support asynchronous tasks and message

Sandglass 1.5k Sep 23, 2022
Hprose is a cross-language RPC. This project is Hprose for Golang.

Hprose 3.0 for Golang Introduction Hprose is a High Performance Remote Object Service Engine. It is a modern, lightweight, cross-language, cross-platf

Hprose 1.2k Sep 13, 2022
The Go language implementation of gRPC. HTTP/2 based RPC

gRPC-Go The Go implementation of gRPC: A high performance, open source, general RPC framework that puts mobile and HTTP/2 first. For more information

grpc 16.8k Sep 22, 2022
The jsonrpc package helps implement of JSON-RPC 2.0

jsonrpc About Simple, Poetic, Pithy. No reflect package. But reflect package is used only when invoke the debug handler. Support GAE/Go Standard Envir

Osamu TONOMORI 169 Sep 26, 2022
Distributed Lab 2: RPC in Go

Distributed Lab 2: RPC in Go Using the lab sheet There are two ways to use the lab sheet, you can either: create a new repo from this template - this

null 0 Oct 18, 2021
A feature complete and high performance multi-group Raft library in Go.

Dragonboat - A Multi-Group Raft library in Go / 中文版 News 2021-01-20 Dragonboat v3.3 has been released, please check CHANGELOG for all changes. 2020-03

lni 4.4k Sep 18, 2022
High performance, distributed and low latency publish-subscribe platform.

Emitter: Distributed Publish-Subscribe Platform Emitter is a distributed, scalable and fault-tolerant publish-subscribe platform built with MQTT proto

emitter 3.4k Sep 22, 2022
💡 A Distributed and High-Performance Monitoring System. The next generation of Open-Falcon

夜莺简介 夜莺是一套分布式高可用的运维监控系统,最大的特点是混合云支持,既可以支持传统物理机虚拟机的场景,也可以支持K8S容器的场景。同时,夜莺也不只是监控,还有一部分CMDB的能力、自动化运维的能力,很多公司都基于夜莺开发自己公司的运维平台。开源的这部分功能模块也是商业版本的一部分,所以可靠性有保

DiDi 5.4k Sep 22, 2022
short-url distributed and high-performance

durl 是一个分布式的高性能短链服务,逻辑简单,并提供了相关api接口,开发人员可以快速接入,也可以作为go初学者练手项目.

宋昂 483 Sep 22, 2022
High-Performance server for NATS, the cloud native messaging system.

NATS is a simple, secure and performant communications system for digital systems, services and devices. NATS is part of the Cloud Native Computing Fo

NATS - The Cloud Native Messaging System 11.5k Sep 24, 2022
Collection of high performance, thread-safe, lock-free go data structures

Garr - Go libs in a Jar Collection of high performance, thread-safe, lock-free go data structures. adder - Data structure to perform highly-performant

LINE 348 Sep 11, 2022
Golang client library for adding support for interacting and monitoring Celery workers, tasks and events.

Celeriac Golang client library for adding support for interacting and monitoring Celery workers and tasks. It provides functionality to place tasks on

Stefan von Cavallar 73 Jul 19, 2022
AppsFlyer 497 Sep 26, 2022
dht is used by anacrolix/torrent, and is intended for use as a library in other projects both torrent related and otherwise

dht Installation Install the library package with go get github.com/anacrolix/dht, or the provided cmds with go get github.com/anacrolix/dht/cmd/....

Matt Joiner 244 Sep 8, 2022