Extensible network application framework inspired by netty

Overview

GO-NETTY

GoDoc license-Apache 2 Go Report Card Build Status Coverage Status

中文介绍

Introduction

go-netty is heavily inspired by netty

Feature

  • Extensible transport support, default support TCP, UDP, QUIC, KCP, Websocket
  • Extensible codec support
  • Based on responsibility chain model
  • Zero-dependency

Documentation

Examples

Quick Start

package main

import (
	"fmt"
	"strings"

	"github.com/go-netty/go-netty"
	"github.com/go-netty/go-netty/codec/format"
	"github.com/go-netty/go-netty/codec/frame"
)

func main() {
	// child pipeline initializer
	var childInitializer = func(channel netty.Channel) {
		channel.Pipeline().
			// the maximum allowable packet length is 128 bytes,use \n to split, strip delimiter
			AddLast(frame.DelimiterCodec(128, "\n", true)).
			// convert to string
			AddLast(format.TextCodec()).
			// LoggerHandler, print connected/disconnected event and received messages
			AddLast(LoggerHandler{}).
			// UpperHandler (string to upper case)
			AddLast(UpperHandler{})
	}

	// create bootstrap & listening & accepting
	netty.NewBootstrap(netty.WithChildInitializer(childInitializer)).
		Listen(":9527").Sync()
}

type LoggerHandler struct {}

func (LoggerHandler) HandleActive(ctx netty.ActiveContext) {
	fmt.Println("go-netty:", "->", "active:", ctx.Channel().RemoteAddr())
	// write welcome message
	ctx.Write("Hello I'm " + "go-netty")
}

func (LoggerHandler) HandleRead(ctx netty.InboundContext, message netty.Message) {
	fmt.Println("go-netty:", "->", "handle read:", message)
	// leave it to the next handler(UpperHandler)
	ctx.HandleRead(message)
}

func (LoggerHandler) HandleInactive(ctx netty.InactiveContext, ex netty.Exception) {
	fmt.Println("go-netty:", "->", "inactive:", ctx.Channel().RemoteAddr(), ex)
	// disconnected,the default processing is to close the connection
	ctx.HandleInactive(ex)
}

type UpperHandler struct {}

func (UpperHandler) HandleRead(ctx netty.InboundContext, message netty.Message) {
	// text to upper case
	text := message.(string)
	upText := strings.ToUpper(text)
	// write the result to the client
	ctx.Write(text + " -> " + upText)
}

use Netcat to send message

$ echo -n -e "Hello Go-Netty\nhttps://go-netty.com\n" | nc 127.0.0.1 9527
Hello I'm go-netty
Hello Go-Netty -> HELLO GO-NETTY
https://go-netty.com -> HTTPS://GO-NETTY.COM
Comments
  • TCP收到数据后在哪里设置直接发送到业务处理器?

    TCP收到数据后在哪里设置直接发送到业务处理器?

    在运行测试代码时,第一个处理器总要定义为如何去拆这个包,如果第一个处理器没有拆这个包,那么后续处理器的HandleRead方法就不会被触发,直到这个连接被断开,此时会执行所有处理器的HandleRead函数,此时会收到缓冲区的所有数据。

    但是目前我的需求是框架收到数据,就立马发送到业务处理器,不管这一段数据是否完整,而不是缓存到缓冲区中。

    那么业务代码该怎么写?能提供一个代码片段么?

    opened by wanliofficial 6
  • 为什么用java netty client 发送byte数组 到 golang tcp server 接收不到呢?

    为什么用java netty client 发送byte数组 到 golang tcp server 接收不到呢?

    我的java netty client写法如下

    NioEventLoopGroup worker = new NioEventLoopGroup();   
            bootstrap.group(worker);   
            bootstrap.channel(NioSocketChannel.class);   
            bootstrap.handler(new ChannelInitializer<Channel>() {   
                @Override   
                protected void initChannel(Channel channel) throws Exception {   
                    //大端发送顺序   
                    channel.pipeline().addLast(new LengthFieldBasedFrameDecoder(ByteOrder.BIG_ENDIAN,1024*1024,2,2,-4,0,true));    
                    channel.pipeline().addLast(new ClientHandler());   
                }  
            });    
                ChannelFuture f = bootstrap.connect("127.0.0.1", 3000);     
                //发送消息  
                byte[] msg = "I am java netty client msg".getBytes();  
                //如果msg为单纯的字符串,go可以接受到,如果改成byte数组,go就一阻塞到  conn.Read(buffer)
                f.channel().writeAndFlush(msg)
    

    我的golang tcp server 写法如下

    func main() {
        listen, err := net.Listen("tcp", "127.0.0.1:3000")
        if err != nil {
            fmt.Println("listen failed, err:", err)
            return
        }
        defer listen.Close()
        for {
            conn, err := listen.Accept()
            if err != nil {
                fmt.Println("accept failed, err:", err)
                continue
            }
            go func() {
                defer conn.Close()
    
                var num int32
                buffer := make([]byte, 6)
                _, err := conn.Read(buffer)  //如果java netty 发的byte[] 字节数组的话,则一直阻塞在这一行
                if err != nil {
                    return
                }
                buf := bytes.NewReader(buffer)
                err = binary.Read(buf, binary.BigEndian, &num)
                fmt.Println(buf)
            }()
        }
    }
    
    opened by ltanme 6
  • [bug]客户端与服务器(几乎)同时断开连接时,异常的CPU占用

    [bug]客户端与服务器(几乎)同时断开连接时,异常的CPU占用

    【我又来提交bug了】 我的程序是这样的,当客户端想要断开连接时,会先发送断开消息,然后主动断开连接(可互动使用java的netty实现) 当服务器(go-netty实现)收到断开消息时,服务器也会主动断开连接(我不清楚这个操作是应该的还是不应该的,我之前认为这个是无所谓的)。但是当服务器放调用conn.Close(nil)时,该函数会返回,但是却会导致异常的cpu占用(从几乎百分之0升到快百分之30) 当我只选择一方断开连接,这个是正常的。

    opened by M1178475702 6
  • new requirement:  support UDP

    new requirement: support UDP

    Please add pure UDP transport. BTW, did you compare the performance of go-netty with netty? Is there any way to set SO_REUSEPORT socket option? Maybe with this option and gorutine process, the performance will be better.

    opened by freeasbird 5
  • 有计划使用 Epoll 实现吗?

    有计划使用 Epoll 实现吗?

    // start write & read routines
    func (c *channel) serveChannel() {
    	c.wait.Add(1)
    	go c.readLoop()
    	go c.writeLoop()
    	c.wait.Wait()
    }
    

    大致翻了 channel 和 transport 部分源码,如上,看起来依然是 conn-per-G 模型,貌似没有使用 Epoll?

    opened by AmourLinux 3
  • ctx.Channel().ID()是一个递增的int64,可能存在隐患

    ctx.Channel().ID()是一个递增的int64,可能存在隐患

    如题,隐患是,大量设备频繁断开与连接,这个自增ID是不会回收之前用过的数字,比如:设备A连接了netty,此时ID是1,设备A断开连接,下次再连接,ID就是2了。

    大量设备重复连接,int64只能表示到21亿左右,恐怕也不太够用。

    所以老哥设计这个ID字段时,为啥会用int64类型,而不是UUID或者雪花ID???

    opened by wanliofficial 2
  • 链接建立之后发送消息不成功

    链接建立之后发送消息不成功

    c, err := bootstrap.Connect("127.0.0.1:6565", nil)
    if err != nil {
    	fmt.Println(err)
    }
    c.Write([]byte("看好看好看好看好看"))
    

    客户端发送不成功,但是客户端断了 消息就发送过去了。为什么是传字节,不能是message netty.Message 这种类型么?

    opened by xiaohei88 2
  • frame.LengthFieldCodec的几个参数应该如何设置?

    frame.LengthFieldCodec的几个参数应该如何设置?

    我的需求是这样的:

    FF FF 01 [00 3D]【00 3D表示数据域的长度】 01 00 00 00 09 00 00 00 17 00 00 00 9F 03 E8 04 B0 00 00 00 00 0A C8 01 72 00 00 00 00 02 1D 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FE FE
    

    我需要从TCP数据包中取出这一段完整的数据包,我当前是这么设置的:

    bootstrap.ChildInitializer(func(channel netty.Channel) {
    	channel.Pipeline().
    		AddLast(frame.LengthFieldCodec(binary.LittleEndian, 1024, 2, 2, 3, 0)).
    		AddLast(EquipmentProtocolCodec()).
    		AddLast(EchoHandler{"Server"})
    })
    

    然后我在调试的时候,发现如下:

    Server -> active: 172.17.208.1:60782
    Server -> handle read: [255 255 1 0 61 1 0 0]  【①使用NetAssist网络调试助手发送数据,立马触发HandleRead方法】
    Server -> handle read: [0 9 0 0 0 23 0] 【②接上再次触发HandleRead方法,之后发送数据不再触发HandleRead方法】
    Server -> handle read: [0 0 159 3 232 4 176 0 0 0 0 10 200 1 114 0 0 0 0 2 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 254 254 255 255 1 0 61 1 0 0 0 9 0 0 0 23 0 0 0 159 3 232 4 176 0 0 0 0 10 200 1 114 0 0 0 0 2 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 254 254 255 255 1 0 61 1 0 0 0 9 0 0 0 23 0 0 0 159 3 232 4 176 0 0 0 0 10 200 1 114 0 0 0 0 2 29 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 254 254] 【③直到NetAssist网络调试助手断开和go-netty的连接,才会触发HandleRead方法】
    Server -> inactive: 172.17.208.1:60782 read header fail, headerLength: 4, read: 0, error: EOF
    

    我的需求就是需要go-netty从TCP数据包中一次性完成的提取出最上面那一段16进制的数据包,其中00 3D表示数据包的长度,可是我在代码这么写AddLast(frame.LengthFieldCodec(binary.LittleEndian, 1024, 2, 2, 3, 0))无法达到预期效果。

    EquipmentProtocolCodec()这个处理器的HandleRead方法目前啥也没干,只是原样把数据交给下一个处理器。

    opened by wanliofficial 2
  • I try to write a UDP client with your other library. Can you give me some idea

    I try to write a UDP client with your other library. Can you give me some idea

    I try to write a UDP client with go-netty and your other go-netty-transport. Can you give me some idea

    func main() {
    
    	var boot = netty.NewBootstrap().Transport(udp.New())
    	boot.ClientInitializer(func(channel netty.Channel) {
    		channel.Pipeline().AddLast(loggerHandler{})
    	})
    
    	con, err := boot.Connect("0.0.0.0:9090", nil)
    
    	utils.Assert(err)
    	select {
    	case <-con.Context().Done():
    		fmt.Println("con")
    	case <-boot.Context().Done():
    		fmt.Println("boot")
    	}
    
    }
    
    type loggerHandler struct{}
    
    func (loggerHandler) HandleRead(ctx netty.InboundContext, message netty.Message) {
    	fmt.Println("go-netty:", "->", "handle read:")
    	fmt.Println(message.(string))
    
    	read := message.(io.Reader)
    
    	b := make([]byte, 8)
    	for {
    		n, err := read.Read(b)
    		fmt.Printf("n = %v err = %v b = %v\n", n, err, b)
    		fmt.Printf("b[:n] = %q\n", b[:n])
    		if err == io.EOF {
    			break
    		}
    	}
    
    	// leave it to the next handler(upperHandler)
    	ctx.HandleRead(message)
    }
    
    
    func (loggerHandler) HandleInactive(ctx netty.InactiveContext, ex netty.Exception) {
    	fmt.Println("go-netty:", "->", "inactive:", ctx.Channel().RemoteAddr())
    	fmt.Println(ex)
    	// disconnected,the default processing is to close the connection
    	ctx.HandleInactive(ex)
    }
    
    interface conversion: netty.Message is *udp.udpClientTransport, not string
    
    
    opened by dengerhuan 2
  • 链路延时

    链路延时

    1. How to inject random latency in TCP connection?
    2. Can the library support 1000 client connection at the same tiime? Thanks. Want to use it to simulate a LAN on a local server.
    opened by Koll-Stone 1
  • [bug]关于lengthFieldCodec的initialBytesToStrip

    [bug]关于lengthFieldCodec的initialBytesToStrip

    我的应用层协议的header只有一个长度为4的长度字段,因此我的设置是这样的 image 因为这是我第一次用类netty的包(包括java的),initialBytesToStrip起初我并不知道是什么作用,因此我设置成了0. 起初他正常工作。 当我用java的netty写客户端的时候,问题发生了。 我对java的设置开始也是这样,initialBytesToStrip设置为0 image 但是我发现我收到的包总是会包括header(前四个长度字节) 通过查看java的netty的源码,我发现了,当交给下一个handler时, netty是将包的总长度减去initialBytesToStrip 因为我设置的是0,所以总是将包头也算了进去。 而您的代码中的实现是用包体的长度减去initialBytesToStrip,所以我设置成0反而成功了 java中的做法: image 这里是又将framelength(本来是包体的长度)还原成了整个frame的长度 image

    opened by M1178475702 1
  • Hello, can you listen to the connection events between client and server in go-netty?

    Hello, can you listen to the connection events between client and server in go-netty?

    Hello, can you listen to the connection events between client and server in go-netty? I looked at the source code and found that there was no handler that could monitor the connection between client and server. I want to monitor the connection to client in server so that it is convenient to count the number of real-time connections to client.

    opened by LJF2402901363 2
Owner
The Go-Netty Project
The Go-Netty Project
Fast, multithreaded, modular and extensible DHCP server written in Go

coredhcp Fast, multithreaded, modular and extensible DHCP server written in Go This is still a work-in-progress Example configuration In CoreDHCP almo

CoreDHCP 793 Jan 9, 2023
CoreRAD is an extensible and observable IPv6 Neighbor Discovery Protocol router advertisement daemon. Apache 2.0 Licensed.

CoreRAD CoreRAD is an extensible and observable IPv6 Neighbor Discovery Protocol router advertisement daemon. Apache 2.0 Licensed. To get started with

Matt Layher 122 Nov 14, 2022
Package socket provides a low-level network connection type which integrates with Go's runtime network poller to provide asynchronous I/O and deadline support. MIT Licensed.

socket Package socket provides a low-level network connection type which integrates with Go's runtime network poller to provide asynchronous I/O and d

Matt Layher 49 Dec 14, 2022
Magma is an open-source software platform that gives network operators an open, flexible and extendable mobile core network solution.

Connecting the Next Billion People Magma is an open-source software platform that gives network operators an open, flexible and extendable mobile core

Magma 1.5k Dec 31, 2022
Optimize Windows's network/NIC driver settings for NewTek's NDI(Network-Device-Interface).

windows-ndi-optimizer[WIP] Optimize Windows's network/NIC driver settings for NewTek's NDI(Network-Device-Interface). How it works This is batchfile d

Nil Hiiragi 3 Apr 15, 2022
A simple network analyzer that capture http network traffic

httpcap A simple network analyzer that captures http network traffic. support Windows/MacOS/Linux/OpenWrt(x64) https only capture clienthello colorful

null 2 Oct 25, 2022
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.

Thank you for your interest in ZASentinel ZASentinel helps organizations improve information security by providing a better and simpler way to protect

ZTALAB 8 Nov 1, 2022
Ipfs-retriever - An application that retrieves files from IPFS network

ipfs-retriever This is an application that retrieves files from IPFS network. It

Phat Nguyen Luu 0 Jan 5, 2022
Build - The axelar-core app based on the Cosmos SDK is the main application of the axelar network

⚠️ ⚠️ ⚠️ THIS IS A WORK IN PROGRESS ⚠️ ⚠️ ⚠️ axelar-core The axelar-core app bas

Kalidux 0 Feb 15, 2022
NFF-Go -Network Function Framework for GO (former YANFF)

Network Function Framework for Go (former YANFF) Wonderful news : we are now supporting AF_XDP and supporting(almost) getting packets directly from Li

Intel Go Team 1.3k Dec 28, 2022
Go network programming framework, supports multiplexing, synchronous and asynchronous IO mode, modular design, and provides flexible custom interfaces

Go network programming framework, supports multiplexing, synchronous and asynchronous IO mode, modular design, and provides flexible custom interfaces。The key is the transport layer, application layer protocol has nothing to do

rick.wu 11 Nov 7, 2022
Powerful golang network framework, supporting FFAX Protocol

X.NET framework Install $ go get github.com/RealFax/XNET This is a high-performance network framework, currently only supports tcp and FFAX protocol U

Realfax Messenger 0 Nov 19, 2021
YoMo 46 Nov 9, 2022
An application that uses gRPC Client Streaming framework to computes average.

compute-average An API that uses Client Streaming gRPC framework to capture all integer numbers on Requests that the client sent and returns just the

Lucas Thiago Batista 0 Dec 26, 2021
Inspired by go-socks5,This package provides full functionality of socks5 protocol.

The protocol described here is designed to provide a framework for client-server applications in both the TCP and UDP domains to conveniently and securely use the services of a network firewall.

Zhangliu 69 Dec 16, 2022
A terminal UI for tshark, inspired by Wireshark

Termshark A terminal user-interface for tshark, inspired by Wireshark. V2.2 is out now with vim keys, packet marks, a command-line and themes! See the

Graham Clark 7.7k Jan 9, 2023
An easy HTTP client for Go. Inspired by the immortal Requests.

rek An easy HTTP client for Go inspired by Requests, plus all the Go-specific goodies you'd hope for in a client. Here's an example: // GET request re

Luc Perkins 383 Sep 20, 2022
Inspired by Have I Been Pwnd

Have I Been Redised Check it out at RedisPwned and Have I Been Redised How it works We scan the internet for exposed Redis databases broadcasting to t

Michael Blum 4 Nov 14, 2021
MPD client inspired by ncmpcpp written in GO with builtin cover art previews.

goMP MPD client inspired by ncmpcpp written in GO demo.mp4 Roadmap Add Functionality to Sort out most played songs Add a config parser Image Previews

Aditya Kurdunkar 46 Jan 1, 2023