Simple, fast and scalable golang rpc library for high load

Related tags

Network gorpc
Overview

gorpc

Simple, fast and scalable golang RPC library for high load and microservices.

Gorpc provides the following features useful for highly loaded projects with RPC:

  • It minimizes the number of connect() syscalls by pipelining request and response messages over a single connection.

  • It minimizes the number of send() syscalls by packing as much as possible pending requests and responses into a single compressed buffer before passing it into send() syscall.

  • It minimizes the number of recv() syscalls by reading and buffering as much as possible data from the network.

  • It supports RPC batching, which allows preparing multiple requests and sending them to the server in a single batch.

These features help the OS minimizing overhead (CPU load, the number of TCP connections in TIME_WAIT and CLOSE_WAIT states, the number of network packets and the amount of network bandwidth) required for RPC processing under high load.

Additionally gorpc provides the following features missing in net/rpc:

  • Client automatically manages connections and automatically reconnects to the server on connection errors.
  • Client supports response timeouts.
  • Client supports RPC batching.
  • Client supports async requests' canceling.
  • Client prioritizes new requests over old pending requests if server fails to handle the given load.
  • Client detects stuck servers and immediately returns error to the caller.
  • Client supports fast message passing to the Server, i.e. requests without responses.
  • Both Client and Server provide network stats and RPC stats out of the box.
  • Commonly used RPC transports such as TCP, TLS and unix socket are available out of the box.
  • RPC transport compression is provided out of the box.
  • Server provides graceful shutdown out of the box.
  • Server supports RPC handlers' councurrency throttling out of the box.
  • Server may pass client address to RPC handlers.
  • Server gracefully handles panic in RPC handlers.
  • Dispatcher accepts functions as RPC handlers.
  • Dispatcher supports registering multiple receiver objects of the same type under distinct names.
  • Dispatcher supports RPC handlers with zero, one (request) or two (client address and request) arguments and zero, one (either response or error) or two (response, error) return values.

Dispatcher API provided by gorpc allows easily converting usual functions and/or struct methods into RPC versions on both client and server sides. See Dispatcher examples for more details.

By default TCP connections are used as underlying gorpc transport. But it is possible using arbitrary underlying transport - just provide custom implementations for Client.Dial and Server.Listener. RPC authentication, authorization and encryption can be easily implemented via custom underlying transport and/or via OnConnect callbacks. Currently gorpc provides TCP, TLS and unix socket transport out of the box.

Currently gorpc with default settings is successfully used in highly loaded production environment serving up to 40K qps. Switching from http-based rpc to gorpc reduced required network bandwidth from 300 Mbit/s to 24 Mbit/s.

Docs

See http://godoc.org/github.com/valyala/gorpc .

Usage

Server:

s := &gorpc.Server{
	// Accept clients on this TCP address.
	Addr: ":12345",

	// Echo handler - just return back the message we received from the client
	Handler: func(clientAddr string, request interface{}) interface{} {
		log.Printf("Obtained request %+v from the client %s\n", request, clientAddr)
		return request
	},
}
if err := s.Serve(); err != nil {
	log.Fatalf("Cannot start rpc server: %s", err)
}

Client:

c := &gorpc.Client{
	// TCP address of the server.
	Addr: "rpc.server.addr:12345",
}
c.Start()

// All client methods issuing RPCs are thread-safe and goroutine-safe,
// i.e. it is safe to call them from multiple concurrently running goroutines.
resp, err := c.Call("foobar")
if err != nil {
	log.Fatalf("Error when sending request to server: %s", err)
}
if resp.(string) != "foobar" {
	log.Fatalf("Unexpected response from the server: %+v", resp)
}

Both client and server collect connection stats - the number of bytes read / written and the number of calls / errors to send(), recv(), connect() and accept(). This stats is available at Client.Stats and Server.Stats.

See tests for more usage examples.

Comments
  • Simpler compression library

    Simpler compression library

    I tried to create simple streaming compression library for this project

    https://github.com/funny-falcon/go-funlz

    Format derived from lzf. It is byte oriented, minimum compressed chunk is 4 bytes, backref window is 4096 bytes. It uses circular buffer of 8096 bytes for buffering uncompressed bytes. It does no output buffering.

    Compression is just 1.5 faster than compress/flate, but decompression is up to 2-10 times faster.

    opened by funny-falcon 12
  • Improvements

    Improvements

    • use non-blocking select from single channel when possible such select doesn't need heap allocation nor complex logic
    • stop timer in client CallTimeout
    • detect empty requestsChan and responsesChan instead of timer to flush writers with default PendingRequests == PendingResponses == 32768 it is good both for throughput and latency (but i'll recomend to decrease default write buffers a bit).
    opened by funny-falcon 9
  • fix no delay writer loops

    fix no delay writer loops

    Benching before

    • with default flush delay
    PASS
    BenchmarkEchoInt10000Workers-4    300000          4327 ns/op
    BenchmarkEchoIntNocompress10000Workers-4      300000          3911 ns/op
    BenchmarkEchoString10000Workers-4     300000          4703 ns/op
    BenchmarkEchoStringNocompress10000Workers-4   300000          4380 ns/op
    BenchmarkEchoStruct10000Workers-4     200000          6127 ns/op
    BenchmarkEchoStructNocompress10000Workers-4   200000          5433 ns/op
    ok      _/home/yura/Projects/gorpc  13.575s
    
    • with disabled delay
    PASS
    BenchmarkEchoInt10000Workers-4    200000          8334 ns/op
    BenchmarkEchoIntNocompress10000Workers-4      300000          4466 ns/op
    BenchmarkEchoString10000Workers-4     200000          8551 ns/op
    BenchmarkEchoStringNocompress10000Workers-4   300000          4785 ns/op
    BenchmarkEchoStruct10000Workers-4     200000         10674 ns/op
    BenchmarkEchoStructNocompress10000Workers-4   200000          5675 ns/op
    ok      _/home/yura/Projects/gorpc  15.136s
    

    Benching after:

    • with default flush delay
    PASS
    BenchmarkEchoInt10000Workers-4    300000          4307 ns/op
    BenchmarkEchoIntNocompress10000Workers-4      300000          3856 ns/op
    BenchmarkEchoString10000Workers-4     300000          4643 ns/op
    BenchmarkEchoStringNocompress10000Workers-4   300000          4320 ns/op
    BenchmarkEchoStruct10000Workers-4     200000          6010 ns/op
    BenchmarkEchoStructNocompress10000Workers-4   200000          5348 ns/op
    ok      _/home/yura/Projects/gorpc  13.514s
    
    • with disabled flush delay
    PASS
    BenchmarkEchoInt10000Workers-4    300000          4230 ns/op
    BenchmarkEchoIntNocompress10000Workers-4      300000          3645 ns/op
    BenchmarkEchoString10000Workers-4     300000          4341 ns/op
    BenchmarkEchoStringNocompress10000Workers-4   300000          4029 ns/op
    BenchmarkEchoStruct10000Workers-4     200000          5884 ns/op
    BenchmarkEchoStructNocompress10000Workers-4   200000          5240 ns/op
    ok      _/home/yura/Projects/gorpc  13.057s
    

    edit: rerun bench with taskset -c 4,5,6,7

    opened by funny-falcon 7
  • windows call timeout

    windows call timeout

    go version go1.5.1 windows/amd64

    go test -run ExampleDispatcher_serviceCalls

    --- FAIL: ExampleDispatcher_serviceCalls (120.06s) got: Get=, gorpc.Client: [:7892]. Cannot obtain response during timeout=20s Get=, gorpc.Client: [:7892]. Cannot obtain response during timeout=20s Set=, gorpc.Client: [:7892]. Cannot obtain response during timeout=20s, 456

    GetError42=, gorpc.Client: [:7892]. Cannot obtain response during timeout=2 0s GetError42=, gorpc.Client: [:7892]. Cannot obtain response during timeout=2 0s DDDD=, gorpc.Client: [:7892]. Cannot obtain response during timeout=20s want: Get=123, Get=456, Set=, , 78 GetError42=78, GetError42=, error42 DDDD=13, FAIL exit status 1

    bug help wanted 
    opened by gyf19 3
  • Is this still a better choice than golang roc?

    Is this still a better choice than golang roc?

    I see the last commit is more than two years ago, so I would like to know whether this is better choice if I want to use it in high-load servers with large request size (~200K content per request on average).

    opened by 1a1a11a 2
  • Is it possible to add support of struct{}?

    Is it possible to add support of struct{}?

    In many cases sets implemented as map[T]struct{}. It would be nice to support it to avoid extra convertion on both server and client sides.

    Currently Dispatcher AddFunc panics:

    ... cannot contain struct without exported fields [struct {}] in the value of map...
    

    I could implement custom GobDecoder/GobDecoder for map[T]struct{}-type in my code, but it would be nice if gorpc supports it.

    opened by ikkeps 2
  • Dispatcher type validation is probably a bit too strict (bug)

    Dispatcher type validation is probably a bit too strict (bug)

    Hi,

    I was trying to pass a struct containing a time.Time object between the server and the client and I got the following panic just after running:

    gorpc.Dispatcher: response in the function [Foo.Bar] cannot contain struct without exported fields [time.Time] in the field [Start] of struct [foo.Result]

    It appears that the dispatcher, in the validateType function, checks for at least one exported field in each struct it sees and therefore reject time.Time which has none. This seems overly strict because time.Time implements the GobEncoder and GobDecoder interfaces even with no exported fields and therefore should be able to pass on the wire without problem.

    I am fairly new to Golang and therefore have no idea how this could be fixed but I am under the impression that it is a bug of gorpc and not an intended feature.

    Could you have a look, please?

    Best,

    HLG

    opened by hlegendre 2
  • Build is breaking under go older than 1.4

    Build is breaking under go older than 1.4

    as of a pull right now.

    ../../../valyala/gorpc/client.go:553: undefined: atomic.Value ../../../valyala/gorpc/server.go:187: undefined: atomic.Value ../../../valyala/gorpc/server.go:240: undefined: atomic.Value

    bug 
    opened by manishrjain 2
  • Dispatcher couples server and client

    Dispatcher couples server and client

    It seems that you need to register functions or services with the Dispatcher that are both used by the client and server. Because of this I'm not able to build the server and client code separately. I'd like to see the dispatcher decoupled from the server and client.

    I don't have any idea what is a nice way to implement this. Could you give directions, than I can create a pull request with the changes.

    opened by dwlnetnl 2
  • GoRPC on 32 bits platforms

    GoRPC on 32 bits platforms

    Hello,

    Since I ran into this issue and I cannot upgrade my machine for now, I am stuck with 32 bits mode. I think that the stats could just be made uint32 on 32 bits platforms and left as is on 64 bits ones.

    What do you think?

    Pierre

    opened by pierrec 2
  • Cannot decode request

    Cannot decode request

    Hi,

    When running example code from README:

    2015/08/11 12:31:48 Obtained request foobar from the client 127.0.0.1:62891
    2015/08/11 12:31:48 gorpc.Server: [127.0.0.1:62891]->[:12345]. Cannot decode request: [unexpected EOF]
    
    opened by krzysztofantczak 1
  • Add a conn != nil check

    Add a conn != nil check

    go version go1.13 linux/amd64 Ubuntu 18.04.3 LTS

    Т.к. в clientHandler() в рутине происходит conn, err = c.Dial(), то бывают такие случаи...

    panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x12ee814]

    goroutine 39 [running]: github.com/valyala/gorpc.clientHandleConnection(0xc0002d0000, 0x0, 0x0) /mnt/hdd/home/biter/go/src/github.com/valyala/gorpc/client.go:688 +0x2b4 github.com/valyala/gorpc.clientHandler(0xc0002d0000) /mnt/hdd/home/biter/go/src/github.com/valyala/gorpc/client.go:673 +0x4f6 created by github.com/valyala/gorpc.(*Client).Start /mnt/hdd/home/biter/go/src/github.com/valyala/gorpc/client.go:157 +0x241

    opened by biter777 0
  • the docs make me confused, is there any project for reference?

    the docs make me confused, is there any project for reference?

    I have seen the example_test.go, but I still feel confused about how to use gorpc. For exmaple, if my implementation of client and server is segregative, how can I use code like this d := NewDispatcher() s := NewTCPServer("127.0.0.1:12445", d.NewHandlerFunc()) c := NewTCPClient("127.0.0.1:12445") dc := d.NewFuncClient(c) And I feel confused that whether rpc call need a connection pool or not. Thanks!

    opened by beaulian 2
  • ConnStats.Snapshot example please

    ConnStats.Snapshot example please

    Just a bit of pseudo code would be great...

    d := gorpc.NewDispatcher()   
    s := gorpc.NewTCPServer("127.0.0.1:43216", d.NewHandlerFunc())
    s.Start()
    stats := ??? Snapshot()
    
    opened by goarchit 0
  • How to coordinate physically seperate servers and clients

    How to coordinate physically seperate servers and clients

    I'm sure I'm missing something extremely trivial here, but am stuck. I'd like to implement a server on one computer that has a registered service, call it "Ping", that can be called from a client on another server and respond with "Pong".

    Per https://godoc.org/github.com/valyala/gorpc#example-Dispatcher--FuncCalls the client wrapper is established as dc := d.NewFuncClient(c).

    My concern is that d is not defined in a client only applet.

    If I reference the NewTCPClient directly, with a call like c.Call("Ping") the server will error out with:

    gorpc.Dispatcher: unsupported request type received from the client: string

    Obviously I'm missing something basic, please clue me in.

    Server side pseudo code:

     d := gorpc.NewDispatcher()
     d.AddFunc("Ping", func () string {return "Pong"})
     s := gorpc.NewTCPServer("localhost:1234",d.NewHandlerFunc())
     s.Start()
    

    Client side pseudo code:

     c := gorpc.NewTCPClient("localhost:1234")
     c.Start()
     c.Call("Ping")
    

    I've also tried:

    d := gorpc.NewDispatcher()
    c := gorpc.NewTCPClient("localhost:1234")
    c.Start()
    dc := d.NewFuncClient(c)
    resp, err := dc.Call("Ping", nil)
    

    Which results in a " Error when reading handshake from client: [EOF]" error since no d.AddFunc() has been called.

    My brain is having problems groking that the client code would need any details beyond a function name that the server handles.

    I did find that if I used the 2nd approach and added a d.AddFunc("Ping", func() {}) prior to the dc assignment it works.

    Really not clear if that is the right way though or if I just hacked it... Comments please.

    opened by goarchit 2
Owner
Aliaksandr Valialkin
Working on @VictoriaMetrics
Aliaksandr Valialkin
Antenna RPC is an RPC protocol for distributed computing, it's based on QUIC and Colfer. its currently an WIP.

aRPC - Antenna Remote Procedure Call Antenna remote procedure call (aRPC) is an RPC protocol focused on distributed processing and HPC. aRPC is implem

Raphael de Carvalho Almeida 3 Jun 16, 2021
the pluto is a gateway new time, high performance, high stable, high availability, easy to use

pluto the pluto is a gateway new time, high performance, high stable, high availability, easy to use Acknowledgments thanks nbio for providing low lev

mobus 2 Sep 19, 2021
rpc/v2 support for JSON-RPC 2.0 Specification.

rpc rpc/v2 support for JSON-RPC 2.0 Specification. gorilla/rpc is a foundation for RPC over HTTP services, providing access to the exported methods of

High Performance, Kubernetes Native Object Storage 3 Jul 4, 2021
Go Substrate RPC Client (GSRPC)Go Substrate RPC Client (GSRPC)

Go Substrate RPC Client (GSRPC) Substrate RPC client in Go. It provides APIs and types around Polkadot and any Substrate-based chain RPC calls. This c

Chino Chang 1 Nov 11, 2021
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Spiral Scout 6.9k Nov 29, 2022
High-performance PHP application server, load-balancer and process manager written in Golang

RoadRunner is an open-source (MIT licensed) high-performance PHP application server, load balancer, and process manager. It supports running as a serv

Spiral Scout 6.1k Dec 9, 2021
🧙 High-performance PHP-to-Golang IPC/RPC bridge

High-performance PHP-to-Golang IPC bridge Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/r

RoadRunner 1.1k Dec 6, 2022
A modern, fast and scalable websocket framework with elegant API written in Go

About neffos Neffos is a cross-platform real-time framework with expressive, elegant API written in Go. Neffos takes the pain out of development by ea

Gerasimos (Makis) Maropoulos 480 Dec 7, 2022
Go-random-chat - Fast and scalable real-time random chat written in go

Go Random Chat Fast and scalable real-time random chat written in go. Features:

Hao-Ming, Hsu 10 Nov 20, 2022
Netpoll is a high-performance non-blocking I/O networking framework, which focused on RPC scenarios, developed by ByteDance.

Netpoll is a high-performance non-blocking I/O networking framework, which focused on RPC scenarios, developed by ByteDance. RPC is usually heavy on processing logic and therefore cannot handle I/O serially. But Go's standard library net designed blocking I/O API, so that the RPC framework can only follow the One Conn One Goroutine design.

CloudWeGo 3.2k Dec 2, 2022
🚀Gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily build high-performance servers.

gev 中文 | English gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily bui

徐旭 1.5k Dec 5, 2022
Golang Super Simple Load Balance

SSLB (Super Simple Load Balancer) ver 0.1.0 It's a Super Simple Load Balancer, just a little project to achieve some kind of performance. Features Hig

Eduardo Pereira 143 Sep 27, 2022
`kawipiko` -- blazingly fast static HTTP server -- focused on low latency and high concurrency, by leveraging Go, `fasthttp` and the CDB embedded database

kawipiko -- blazingly fast static HTTP server kawipiko is a lightweight static HTTP server written in Go; focused on serving static content as fast an

Volution 315 Nov 26, 2022
llb - It's a very simple but quick backend for proxy servers. Can be useful for fast redirection to predefined domain with zero memory allocation and fast response.

llb What the f--k it is? It's a very simple but quick backend for proxy servers. You can setup redirect to your main domain or just show HTTP/1.1 404

Kirill Danshin 13 Sep 27, 2022
A blockchains platform with high throughput, and blazing fast transactions

Node implementation for the Avalanche network - a blockchains platform with high

null 0 Dec 18, 2021
Go-Web-Dev - Golang helps the developer to develop highly scalable applications

Go-Web-Dev Golang helps the developer to develop highly scalable applications. T

Mohamed Dhik 0 Feb 5, 2022
Laptop Booking Application in Golang and gRPC, load-balancing with NGINX, and fully compatible with HTTPS OpenAPI v3

Laptop Booking Application in Golang and gRPC Goals GitHub CI & Coverage Badge Serialize protobuf messages Create laptop unary gRPC Search laptop Serv

Tien La 3 Jun 17, 2022
A python comtrade load library accelerated by go

Comtrade-GRPC Code for python used is mainly from dparrini/python-comtrade. Just

Bo 1 Dec 27, 2021
Consul Load-Balancing made simple

Notes From release 1.5.15 onward, fabio changes the default GOGC from 800 back to the golang default of 100.

fabio 7.1k Dec 4, 2022