zMemif is a native golang based library for memif to interworking with dpdk.

Related tags

Network zmemif
Overview

zMemif

zMemif is a native golang based library for memif to interworking with dpdk. it can simply provide 20Mpps recv and 10Mpps xmit capability.

  • The golang lib is forked from vpp gomemif example code, and modified the "Interface" to "Port" which make golang developers more clear about the definition. it also change the "master/slave" to "server/client" to align with existing dpdk library.
  • we write a dpdk based light-weight forwarder to replace vpp on data path, provide simple virtio-user interface for kernel service and tcp based remote mgmt usage, all udp packet will redirect to memif which can be directly serve by native golang.

Motivation

Many applications like network telemetry analysis, video processing and other udp based application may require high performance I/O support in native-golang enviroment. DPDK may useful to accelerate packet proccessing in these case, intel open souce nff-go project few years ago, however the cgo and flow based abstraction may useful for NFV developer, but many application developer require a simple native go based solution.

This project wil be used for ruta-io and netDAM. We may integrate zMemif with go-quic in the future to provide quic based segment routing(quic-sr) or Segment Routing over UDP(SRoU).

Introduction

zmemif is a memory interface implementation for accelerate golang. You could build your application pure in golang mode, and use DPDK as memif server.

Usage

1. native golang server <----memif----> native golang client

example could be found /example/simple_echo and /example/bw_test

2. DPDK based server <----memif----> native golang client

dpdk forwarder server could be found /dpdk folder.

main.c defined the RX/TX core and queue per core, will add by cli args in the future

    unsigned int first_lcore = 25;
    char *ipaddr = "11.11.11.11";
    char *netmask = "255.255.255.0";
    int num_memif = 16;
    int num_rx_core = 4;
    int num_tx_core = 2;
    int server_mode = 1;

how to compile dpdk forwarder

  1. install and setup dpdk env could be found in the following url

https://github.com/zartbot/learn_dpdk/tree/main/a1_setup_mlx5_sriov_env

  1. compile and run
cd dpdk
make
sudo ./build/run
  1. about unix sock file by default, it will create /tmp/memif.sock file for unix socket to communicate with golang client, this unix socket is used to allocate memory region and ring buffer. DPDK forwarder need to run in root mode and auto create this file, so this file must provide access priviledge for golang client.
sudo chmod 777 /tmp/memif.sock 
  1. run native golang client
cd /example/dpdk_co_worker

run send/recv examples..

Roadmap

  1. reliable transmit on datapath
  2. simple udp level send/recv warpper for application migration.
  3. In the future , we will use the netDAM DPU hardware to replace the dpdk based forwarder and provide full native userspace memif access.

Reference

Owner
Mie~~~
Mie~~~
grobotstxt is a native Go port of Google's robots.txt parser and matcher library.

grobotstxt grobotstxt is a native Go port of Google's robots.txt parser and matcher C++ library. Direct function-for-function conversion/port Preserve

Jim Smart 86 Apr 7, 2022
Golang client for NATS, the cloud native messaging system.

NATS - Go Client A Go client for the NATS messaging system. Installation # Go client go get github.com/nats-io/nats.go/ # Server go get github.com/na

NATS - The Cloud Native Messaging System 4.1k Aug 17, 2022
🀘 The native golang ssh client to execute your commands over ssh connection. πŸš€πŸš€

Golang SSH Client. Fast and easy golang ssh client module. Goph is a lightweight Go SSH client focusing on simplicity! Installation ❘ Features ❘ Usage

Mohamed El Bahja 1.1k Aug 9, 2022
A cloud native distributed streaming network telemetry.

Panoptes Streaming Panoptes Streaming is a cloud native distributed streaming network telemetry. It can be installed as a single binary or clustered n

Yahoo 36 Jun 28, 2022
The Cloud Native Application Proxy

Traefik (pronounced traffic) is a modern HTTP reverse proxy and load balancer that makes deploying microservices easy. Traefik integrates with your ex

Traefik Labs 39.2k Aug 9, 2022
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io

δΈ­ζ–‡ MOSN is a network proxy written in Golang. It can be used as a cloud-native network data plane, providing services with the following proxy functio

MOSN 3.8k Aug 15, 2022
Cloud Native Tunnel

inlets is a Cloud Native Tunnel written in Go Expose your local endpoints to the Internet or within a remote network, without touching firewalls. Foll

inlets 8.4k Jan 4, 2022
A native Thrift package for Go

Thrift Package for Go API Documentation: http://godoc.org/github.com/samuel/go-thrift License 3-clause BSD. See LICENSE file. Overview Thrift is an ID

Samuel Stauffer 373 Jul 21, 2022
Native ZooKeeper client for Go. This project is no longer maintained. Please use https://github.com/go-zookeeper/zk instead.

Native Go Zookeeper Client Library License 3-clause BSD. See LICENSE file. This Repository is No Longer Maintained Please use https://github.com/go-zo

Samuel Stauffer 1.6k Aug 15, 2022
go implementation of fissions web-native file system

wnfs-go go language implementation of the fission web-native file system, using the typescript implementation as a reference. Development Status: Work

Qri.io 13 Mar 28, 2022
Package rsync contains a native Go rsync implementation.

gokrazy rsync Package rsync contains a native Go rsync implementation. ⚠ Beware: very fresh. Might eat your data. You have been warned! ⚠ The only com

null 285 Aug 12, 2022
Native macOS networking for QEMU using vmnet.framework and socket networking.

qemu-vmnet Native macOS networking for QEMU using vmnet.framework and socket networking. Getting started TODO -netdev socket,id=net0,udp=:1234,localad

Alessio Dionisi 25 Aug 1, 2022
Curtis Milo 2 May 8, 2022
πŸš€Gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily build high-performance servers.

gev δΈ­ζ–‡ | English gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily bui

徐旭 1.5k Aug 8, 2022
eBPF library for Go based on Linux libbpf

libbpfgo libbpfgo is a Go library for working with Linux's eBPF. It was created for Tracee, our open source Runtime Security and eBPF tracing tools wr

Aqua Security 335 Aug 16, 2022
Lux - A web library collection based on net/http

Lux - A web library collection based on net/http

serve-it-yourself 16 Aug 1, 2022
A library for the MIGP (Might I Get Pwned) protocolA library for the MIGP (Might I Get Pwned) protocol

MIGP library This contains a library for the MIGP (Might I Get Pwned) protocol. MIGP can be used to build privacy-preserving compromised credential ch

Cloudflare 21 Dec 16, 2021
V3IO Frames ("Frames") is a Golang based remote data frames access (over gRPC or HTTP stream)

V3IO Frames ("Frames") is a multi-model open-source data-access library that provides a unified high-performance DataFrame API for working with different types of data sources (backends). The library was developed by Iguazio to simplify working with data in the Iguazio Data Science Platform ("the platform"), but it can be extended to support additional backend types.

null 23 Mar 3, 2022