EasyTCP is a light-weight and less painful TCP server framework written in Go (Golang) based on the standard net package.

Overview

EasyTCP

Run Actions Go Report codecov

$ ./start

[EASYTCP ROUTE TABLE]:
+------------+-----------------------+
| Message ID |     Route Handler     |
+------------+-----------------------+
|       1000 | path/to/handler.Func1 |
+------------+-----------------------+
|       1002 | path/to/handler.Func2 |
+------------+-----------------------+
[EASYTCP] Serving at: tcp://[::]:10001

Introduction

EasyTCP is a light-weight and less painful TCP server framework written in Go (Golang) based on the standard net package.

Features:

  • Non-invasive design
  • Pipelined middlewares for route handler
  • Customizable message packer and codec, and logger
  • Handy functions to handle request data and send response
  • Common hooks

EasyTCP helps you build a TCP server easily and fast.

This package, so far, has been tested with

  • go1.14.x
  • go1.15.x
  • go1.16.x

on the latest Linux, Macos and Windows.

Install

Use the below Go command to install EasyTCP.

$ go get -u github.com/DarthPestilane/easytcp

Note: EasyTCP uses Go Modules to manage dependencies.

Quick start

package main

import (
    "fmt"
    "github.com/DarthPestilane/easytcp"
    "github.com/DarthPestilane/easytcp/message"
)

func main() {
    // Create a new server with default options.
    s := easytcp.NewServer(&easytcp.ServerOption{})

    // Register a route with message's ID.
    // The `DefaultPacker` treats id as uint32,
    // so when we add routes or return response, we should use uint32 or *uint32.
    s.AddRoute(uint32(1001), func(c *easytcp.Context) (*message.Entry, error) {
        fmt.Printf("[server] request received | id: %d; size: %d; data: %s\n", c.Message().ID, len(c.Message().Data), c.Message().Data)
        return c.Response(uint32(1002), []byte("copy that"))
    })

    // Set custom logger (optional).
    easytcp.SetLogger(lg)

    // Add global middlewares (optional).
    s.Use(recoverMiddleware)

    // Set hooks (optional).
    s.OnSessionCreate = func(sess *easytcp.Session) {}
    s.OnSessionClose = func(sess *easytcp.Session) {}

    // Set not-found route handler (optional).
    s.NotFoundHandler(handler)

    // Listen and serve.
    if err := s.Serve(":5896"); err != nil && err != server.ErrServerStopped {
        fmt.Println("serve error: ", err.Error())
    }
}

Above is the server side example. There are client and more detailed examples including:

in examples/tcp.

Benchmark

  • goos: darwin
  • goarch: amd64
Benchmark name (1) (2) (3) (4) remark
Benchmark_NoRoute-8 197164 8799 ns/op 159 B/op 5 allocs/op
Benchmark_NotFoundHandler-8 300993 7285 ns/op 811 B/op 13 allocs/op
Benchmark_OneHandler-8 201592 8907 ns/op 670 B/op 20 allocs/op
Benchmark_ManyHandlers-8 276344 8057 ns/op 796 B/op 15 allocs/op
Benchmark_OneRouteSet-8 248247 8245 ns/op 1002 B/op 19 allocs/op
Benchmark_OneRouteJsonCodec-8 176893 6413 ns/op 1618 B/op 32 allocs/op build with encoding/json
Benchmark_OneRouteJsonCodec-8 189723 5985 ns/op 1347 B/op 27 allocs/op build with json-jsoniter/go
Benchmark_OneRouteProtobufCodec-8 210532 6346 ns/op 708 B/op 19 allocs/op
Benchmark_OneRouteMsgpackCodec-8 181495 6196 ns/op 868 B/op 22 allocs/op

Architecture

accepting connection:

+------------+    +-------------------+    +----------------+
|            |    |                   |    |                |
|            |    |                   |    |                |
| tcp server |--->| accept connection |--->| create session |
|            |    |                   |    |                |
|            |    |                   |    |                |
+------------+    +-------------------+    +----------------+

in session:

+------------------+    +-----------------------+    +----------------------------------+
| read connection  |--->| unpack packet payload |--->|                                  |
+------------------+    +-----------------------+    |                                  |
                                                     | router (middlewares and handler) |
+------------------+    +-----------------------+    |                                  |
| write connection |<---| pack packet payload   |<---|                                  |
+------------------+    +-----------------------+    +----------------------------------+

in route handler:

+----------------------------+    +------------+
| codec decode request data  |--->|            |
+----------------------------+    |            |
                                  | user logic |
+----------------------------+    |            |
| codec encode response data |<---|            |
+----------------------------+    +------------+

Conception

Routing

EasyTCP considers every message has a ID segment to distinguish one another. A message will be routed, according to it's id, to the handler through middlewares.

request flow:

+----------+    +--------------+    +--------------+    +---------+
| request  |--->|              |--->|              |--->|         |
+----------+    |              |    |              |    |         |
                | middleware 1 |    | middleware 2 |    | handler |
+----------+    |              |    |              |    |         |
| response |<---|              |<---|              |<---|         |
+----------+    +--------------+    +--------------+    +---------+

Register a route

s.AddRoute(reqID, func(c *easytcp.Context) (*message.Entry, error) {
    // handle the request via ctx
    fmt.Printf("[server] request received | id: %d; size: %d; data: %s\n", c.Message().ID, len(c.Message().Data), c.Message().Data)

    // do things...

    // return response
    return c.Response(respID, []byte("copy that"))
})

Using middleware

// register global middlewares.
// global middlewares are prior than per-route middlewares, they will be invoked first
s.Use(recoverMiddleware, logMiddleware, ...)

// register middlewares for one route
s.AddRoute(reqID, handler, middleware1, middleware2)

// a middleware looks like:
var exampleMiddleware easytcp.MiddlewareFunc = func(next easytcp.HandlerFunc) easytcp.HandlerFunc {
    return func(c *easytcp.Context) (*message.Entry, error) {
        // do things before...
        resp, err := next(c)
        // do things after...
        return resp, err
    }
}

Packer

A packer is to pack and unpack packets' payload. We can set the Packer when creating the server.

s := easytcp.NewServer(&easytcp.ServerOption{
    Packer: new(MyPacker), // this is optional, the default one is DefaultPacker
})

We can set our own Packer or EasyTCP uses DefaultPacker.

The DefaultPacker considers packet's payload as a Size(4)|ID(4)|Data(n) format.

This may not covery some particular cases, but fortunately, we can create our own Packer.

// Packer16bit is a custom packer, implements Packer interafce.
// THe Packet format is `size[2]id[2]data`
type Packer16bit struct{}

func (p *Packer16bit) bytesOrder() binary.ByteOrder {
    return binary.BigEndian
}

func (p *Packer16bit) Pack(entry *message.Entry) ([]byte, error) {
    size := len(entry.Data) // without id
    buff := bytes.NewBuffer(make([]byte, 0, size+2+2))
    if err := binary.Write(buff, p.bytesOrder(), uint16(size)); err != nil {
        return nil, fmt.Errorf("write size err: %s", err)
    }
    if err := binary.Write(buff, p.bytesOrder(), entry.ID.(uint16)); err != nil {
        return nil, fmt.Errorf("write id err: %s", err)
    }
    if err := binary.Write(buff, p.bytesOrder(), entry.Data); err != nil {
        return nil, fmt.Errorf("write data err: %s", err)
    }
    return buff.Bytes(), nil
}

func (p *Packer16bit) Unpack(reader io.Reader) (*message.Entry, error) {
    sizeBuff := make([]byte, 2)
    if _, err := io.ReadFull(reader, sizeBuff); err != nil {
        return nil, fmt.Errorf("read size err: %s", err)
    }
    size := p.bytesOrder().Uint16(sizeBuff)

    idBuff := make([]byte, 2)
    if _, err := io.ReadFull(reader, idBuff); err != nil {
        return nil, fmt.Errorf("read id err: %s", err)
    }
    id := p.bytesOrder().Uint16(idBuff)
    // since id here is the type of uint16, we need to use a uint16 when adding routes.
    // eg: server.AddRoute(uint16(123), ...)

    data := make([]byte, size)
    if _, err := io.ReadFull(reader, data); err != nil {
        return nil, fmt.Errorf("read data err: %s", err)
    }

    entry := &message.Entry{ID: id, Data: data}
    entry.Set("theWholeLength", 2+2+size) // we can set our custom kv data here.
    // c.Message().Get("theWholeLength")  // and get them in route handler.
    return entry, nil
}

And see the custom packer here.

Codec

A Codec is to encode and decode message data. The Codec is optional, EasyTCP won't encode or decode message data if the Codec is not set.

We can set Codec when creating the server.

s := easytcp.NewServer(&easytcp.ServerOption{
    Codec: &easytcp.JsonCodec{}, // this is optional. The JsonCodec is a built-in codec
})

Since we set the codec, we may want to decode the request data in route handler.

s.AddRoute(reqID, func(c *easytcp.Context) (*message.Entry, error) {
    var reqData map[string]interface{}
    if err := c.Bind(&reqData); err != nil { // here we decode message data and bind to reqData
        // handle error...
    }
    fmt.Printf("[server] request received | id: %d; size: %d; data-decoded: %+v\n", c.Message().ID, len(c.Message().Data), reqData)
    respData := map[string]string{"key": "value"}
    return c.Response(respID, respData)
})

Codec's encoding will be invoked before message packed, and decoding should be invoked in the route handler which is after message unpacked.

NOTE:

If the Codec is not set (or is nil), EasyTCP will try to convert the respData (the second parameter of c.Response) into a []byte. So the type of respData should be one of string, []byte or fmt.Stringer.

JSON Codec

JsonCodec is an EasyTCP's built-in codec, which uses encoding/json as the default implementation. Can be changed by build from other tags.

jsoniter :

go build -tags=jsoniter .

Protobuf Codec

ProtobufCodec is an EasyTCP's built-in codec, which uses google.golang.org/protobuf as the implementation.

Msgpack Codec

MsgpackCodec is an EasyTCP's built-in codec, which uses github.com/vmihailenco/msgpack as the implementation.

Contribute

Check out a new branch for the job, and make sure github action passed.

Use issues for everything

  • For a small change, just send a PR.
  • For bigger changes open an issue for discussion before sending a PR.
  • PR should have:
    • Test case
    • Documentation
    • Example (If it makes sense)
  • You can also contribute by:
    • Reporting issues
    • Suggesting new features or enhancements
    • Improve/fix documentation
You might also like...
TCP output for beats to send events over TCP socket.

beats-tcp-output How To Use Clone this project to elastic/beats/libbeat/output/ Modify elastic/beats/libbeat/publisher/includes/includes.go : // add i

Go net wrappers that enable TCP Fast Open.

tfo-go tfo-go provides a series of wrappers around net.Listen, net.ListenTCP, net.DialContext, net.Dial, net.DialTCP that seamlessly enable TCP Fast O

Netkit - A type parameter(generics) net kit, support tcp kcp, customize packet

Netkit Netkit is a type parameter(generics) golang package Get Started Need Go i

A Light Golang RPC Framework

Glory Glory框架为一款Go语言的轻量级RPC框架,您可以使用它快速开发你的服务实例。如果您希望在微服务场景下使用gRPC进行网络通信,那么Glory会使您的开发、运维工作量减轻不少。 欢迎访问Glory主页: glory-go.github.io 示例仓库:github.com/glory

Client-Server tcp-based file transfer application in GoLang

Клиент-серверный файловый сервис на базе протокола TCP Клиент client.go шифрует свои файлы алгоритмом AES с режимом CBC и помещает их на сервер server

A TCP Server Framework with graceful shutdown, custom protocol.

xtcp A TCP Server Framework with graceful shutdown,custom protocol. Usage Define your protocol format: Before create server and client, you need defin

A socks5 server(tcp/udp) written in golang.

socks5-server A socks5 server(tcp/udp) written in golang. Usage Usage of /main: -l string local address (default "127.0.0.1:1080") -p stri

Develop, update, and restart your ESP32 applications in less than two seconds
Develop, update, and restart your ESP32 applications in less than two seconds

Jaguar Develop, update, and restart your ESP32 applications in less than two seconds. Use the really fast development cycle to iterate quickly and lea

“Dear Port80” is a zero-config TCP proxy server that hides SSH connection behind a HTTP server!

Dear Port80 About The Project: “Dear Port80” is a zero-config TCP proxy server that hides SSH connection behind a HTTP server! +---------------------

Comments
  • why don't close respQueue after the Session closed?

    why don't close respQueue after the Session closed?

    i found the respQueue chan was created in session struct

    type session struct {
    	id          interface{}   // session's ID.
    	conn        net.Conn      // tcp connection
    	closed      chan struct{} // to close()
    	closeOnce   sync.Once     // ensure one session only close once
    	respQueue   chan Context  // response queue channel, pushed in Send() and popped in writeOutbound()
    	packer      Packer        // to pack and unpack message
    	codec       Codec         // encode/decode message data
    	ctxPool     sync.Pool     // router context pool
    	asyncRouter bool          // calls router HandlerFunc in a goroutine if false
    }
    

    but it doesn't close when calling the session close function

    // Close closes the session, but doesn't close the connection.
    // The connection will be closed in the server once the session's closed.
    func (s *session) Close() {
    	s.closeOnce.Do(func() { close(s.closed) })
    }
    

    Could you tell me your design about it, many thanks.

    opened by cyheng 3
  • Help with install

    Help with install

    i got an error running my main.go after i run the command nothing changed

    error:

    main.go:6:5: no required module provides package github.com/DarthPestilane/easytcp/message; to add it: go get github.com/DarthPestilane/easytcp/message

    opened by AshKetchumPL 2
  • build: upgrade `go` directive in `go.mod` to 1.17

    build: upgrade `go` directive in `go.mod` to 1.17

    This PR upgrades the go directive in go.mod file by running go mod tidy -go=1.17 to enable module graph pruning and lazy module loading.

    Note 1: This does not prevent users with earlier Go versions from successfully building packages from this module.

    Note 2: The additional require directive is used to record indirect dependencies for Go 1.17 or higher, see https://go.dev/ref/mod#go-mod-file-go.

    opened by Juneezee 2
Releases(v0.3.0)
  • v0.3.0(Jun 1, 2022)

    What's Changed

    • d30427c chore: upgrade yaml.v3
    • 1613344 refactor: improve tls serve and examples
    • f478a26 refactor: remove message.Entry and introduce Message (#39)
    • 1ac862c feat: getter for underlined connection (#38)

    Full Changelog: https://github.com/DarthPestilane/easytcp/compare/v0.2.0...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Mar 28, 2022)

    75d4ad1...05ee572

    Changelog

    • 1cdcbb6 chore: add go1.18.x to github action workflow
    • c584038 feat: support serveTLS (#37)
    • c7f44ac feat: sort message-route table with message id (#35)
    Source code(tar.gz)
    Source code(zip)
  • v0.1.0(Jan 14, 2022)

    • 75d4ad1 feat: support async router (#33)
    • 4199499 refactor: remove the built-in session manager (#32)
    • f17b5fb refactor: reset ctx when session allocate one (#31)
    • 4768e76 feat: set session when call session.NewContext() (#30)
    • 7422342 feat: support context.Context in routeContext (#29)
    • 402c0bf refactor: close session channel politely (#28)
    Source code(tar.gz)
    Source code(zip)
  • v0.0.11(Dec 30, 2021)

    Changelog

    • 9681738 feat: the default logger is mute
    • 1643746 Merge pull request #26 from Juneezee/go1.17
    • b02bc88 feat: add more method to Context (#25)
    • 014e153 feat: add RawResponseData() to Context (#24)
    • 4b8e811 feat: add method NewContext() to return routeContext pointer
    • ec323d1 Merge pull request #23 from DarthPestilane/refactor/context-interface
    • 6ab835d Merge pull request #22 from DarthPestilane/refactor/session-interface
    • 83ec753 refactor: export Codec method to Session interface
    Source code(tar.gz)
    Source code(zip)
  • v0.0.10(Oct 27, 2021)

  • v0.0.9(Oct 11, 2021)

    Changelog

    feat: add Copy method to context (#18) refactor: abstract isStopped method refactor: export close method for session (#17) refactor: write attempt times (#16)

    Source code(tar.gz)
    Source code(zip)
  • v0.0.8(Sep 15, 2021)

    Changelog

    c9401a8 feat: write retry times (#15) 09dbcc3 chore: changelog filter and remove go1.14 in test (#14) 4de8792 fix: tryConnWrite delay sleep and exit with error (#13)

    Source code(tar.gz)
    Source code(zip)
  • v0.0.7(Sep 13, 2021)

    Changelog

    bcb3a25 Merge pull request #6 from DarthPestilane/improve/release-ci 1b2934a introduce const bd4f24a add remove method 929cbcf minor improvement

    Source code(tar.gz)
    Source code(zip)
  • v0.0.6(Sep 8, 2021)

    Features:

    • Non-invasive design
    • Pipelined middlewares for route handler
    • Customizable message packer and codec, and logger
    • Handy functions to handle request data and send response
    • Common hooks

    Changes:

    • simplify code
    • decrease allocs/op
    • refactor router
    Source code(tar.gz)
    Source code(zip)
  • v0.0.5(Aug 24, 2021)

    Features:

    • Non-invasive design
    • Pipelined middlewares for route handler
    • Customizable message packer and codec, and logger
    • Handy functions to handle request data and send response
    • Common hooks

    Changes:

    • simplify code
    • the default packer supports more types of entry.ID
    • improve logs
    Source code(tar.gz)
    Source code(zip)
  • v0.0.3(Aug 18, 2021)

    Features:

    • Non-invasive design
    • Pipelined middlewares for route handler
    • Customizable message packer and codec, and logger
    • Handy functions to handle request data and send response
    • Common hooks

    Changes:

    • simplify code
    Source code(tar.gz)
    Source code(zip)
  • v0.0.1(Aug 2, 2021)

    Features:

    • Non-invasive design
    • Pipelined middlewares for route handler
    • Customizable message packer and codec, and logger
    • Handy functions to handle request data and send response
    • Common hooks
    Source code(tar.gz)
    Source code(zip)
Owner
zxl
If you can understand the me, then I can understand the you.
zxl
Snugger is a light weight but fast network recon scanner that is written from pure golang

Snugger is a light weight but fast network recon scanner that is written from pure golang. with this scann you can ARP your network, port scan hosts and host lists, as well as scan for BSSId

RE43P3R 2 May 19, 2022
Light weight http rate limiting proxy

Introduction Light weight http rate limiting proxy. The proxy will perform rate limiting based on the rules defined in the configuration file. If no r

DHIS2 Platform Engineering 12 Aug 23, 2022
Go package to simulate bandwidth, latency and packet loss for net.PacketConn and net.Conn interfaces

lossy Go package to simulate bandwidth, latency and packet loss for net.PacketConn and net.Conn interfaces. Its main usage is to test robustness of ap

Cevat Barış Yılmaz 311 Sep 24, 2022
EasyNet - A light net library with epoll

easyNet - NON BLOCKING IO Examples echo-server package main import ( "fmt" "g

Dub Dub 7 Aug 24, 2022
🌍 Package tcplisten provides a customizable TCP net.Listener with various performance-related options

Package tcplisten provides customizable TCP net.Listener with various performance-related options: SO_REUSEPORT. This option allows linear scaling ser

Spiral Scout 1 Sep 7, 2022
Multiplexer over TCP. Useful if target server only allows you to create limited tcp connections concurrently.

tcp-multiplexer Use it in front of target server and let your client programs connect it, if target server only allows you to create limited tcp conne

许嘉华 3 May 27, 2021
Tcp chat go - Create tcp chat in golang

TCP chat in GO libs Go net package and goroutines and channels tcp tcp or transm

amirbahador 0 Feb 5, 2022
TcpRoute , TCP 层的路由器。对于 TCP 连接自动从多个线路(电信、联通、移动)、多个域名解析结果中选择最优线路。

TcpRoute2 TcpRoute , TCP 层的路由器。对于 TCP 连接自动从多个线路(允许任意嵌套)、多个域名解析结果中选择最优线路。 TcpRoute 使用激进的选路策略,对 DNS 解析获得的多个IP同时尝试连接,同时使用多个线路进行连接,最终使用最快建立的连接。支持 TcpRoute

GameXG 856 Sep 21, 2022