🔎Sniffing and parsing mysql,redis,http,mongodb etc protocol. 抓包截取项目中的数据库请求并解析成相应的语句。

Overview

go-sniffer

Capture mysql,redis,http,mongodb etc protocol... 抓包截取项目中的数据库请求并解析成相应的语句,如mysql协议会解析为sql语句,便于调试。 不要修改代码,直接嗅探项目中的数据请求。

GitHub license

中文使用说明

Support List:

Demo:

$ go-sniffer en0 mysql

image

Setup:

  • support : MacOS Linux Unix
  • not support : windows
  • If you encounter problems in the go get process, try upgrading the go version (如果go get 过程中遇到问题,请尝试升级go版本)

Centos

$ yum -y install libpcap-devel

Ubuntu

$ apt-get install libpcap-dev

MacOs


RUN

$ go get -v -u github.com/40t/go-sniffer
$ cp -rf $(go env GOPATH)/bin/go-sniffer /usr/local/bin
$ go-sniffer

Usage:

==================================================================================
[Usage]

    go-sniffer [device] [plug] [plug's params(optional)]

    [Example]
          go-sniffer en0 redis          Capture redis packet
          go-sniffer en0 mysql -p 3306  Capture mysql packet

    go-sniffer --[commend]
               --help "this page"
               --env  "environment variable"
               --list "Plug-in list"
               --ver  "version"
               --dev  "device"
    [Example]
          go-sniffer --list "show all plug-in"

==================================================================================
[device] : lo0 :   127.0.0.1
[device] : en0 : xx:xx:xx:xx:xx:xx  192.168.199.221
==================================================================================

中文使用说明

=======================================================================
[使用说明]

    go-sniffer [设备名] [插件名] [插件参数(可选)]

    [例子]
          go-sniffer en0 redis          抓取redis数据包
          go-sniffer en0 mysql -p 3306  抓取mysql数据包,端口3306

    go-sniffer --[命令]
               --help 帮助信息
               --env  环境变量
               --list 插件列表
               --ver  版本信息
               --dev  设备列表
    [例子]
          go-sniffer --list 查看可抓取的协议

=======================================================================
[设备名] : lo0 :   127.0.0.1
[设备名] : en0 : x:x:x:x:x5:x  192.168.1.3
[设备名] : utun2 :   1.1.11.1
=======================================================================

Example:

$ go-sniffer lo0 mysql 
$ go-sniffer en0 redis 
$ go-sniffer eth0 http -p 8080
$ go-sniffer eth1 mongodb

License:

MIT

Comments
  • centos抓mysql报错

    centos抓mysql报错

    2018-11-01 16:21:39| cli -> ser |Stm id[10]: 'SELECT domain FROM xxx WHERE ticket = ? LIMIT 1 '; set @p0 = 'xxxxx'; Execute stm id[10]: using @p0; Drop stm id[10];

    Start new stream: 127.0.0.1->127.0.0.1 33134->3306

    2018/11/01 16:21:39 ERR : Not found stm id 19 panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x7885cf]

    goroutine 44 [running]: github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveClientPacket(0xc420401590, 0xc42051f200, 0x1e, 0x600, 0x0) /Golang/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:314 +0x4ef github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc420401590) /Golang/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:192 +0x98 created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream /Golang/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x42e

    opened by lanybass 7
  • 异常退出,不给?

    异常退出,不给?

    panic: runtime error: index out of range

    goroutine 7 [running]: github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveServerPacket(0xc420056d80, 0xc4204cd000, 0x0, 0x200, 0xb) /data/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:219 +0x442 github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc420056d80) /data/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:194 +0xc0 created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream /data/go/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x3b8

    opened by zackzhan 3
  • fix affected rows conversoin bug when the number of row is large

    fix affected rows conversoin bug when the number of row is large

    related issue

    https://github.com/40t/go-sniffer/issues/29

    https://github.com/40t/go-sniffer/issues/16 ?

    how to reproduce error

    execute query which affected rows is over 2 byte-integer

    [email protected] [sbtest] > desc sbtest1;
    +-------+-----------+------+-----+---------+----------------+
    | Field | Type      | Null | Key | Default | Extra          |
    +-------+-----------+------+-----+---------+----------------+
    | id    | int(11)   | NO   | PRI | NULL    | auto_increment |
    | c     | char(120) | NO   |     |         |                |
    +-------+-----------+------+-----+---------+----------------+
    4 rows in set (0.00 sec)
    
    [email protected] [sbtest] > update sbtest1 set c='fuga' where id<100000;
    Query OK, 99999 rows affected (0.24 sec)
    Rows matched: 99999  Changed: 99999  Warnings: 0
    

    Note

    the type of 'affected rows' is packet-Protocol::LengthEncodedInteger, LengthEncodedInt is appropriate

    refs. https://dev.mysql.com/doc/internals/en/integer.html#packet-Protocol::LengthEncodedInteger https://dev.mysql.com/doc/internals/en/packet-OK_Packet.html

    opened by takaidohigasi 2
  • unrecognized import path

    unrecognized import path "plugin"

    [root❄anhk-pc:go-sniffer]☭ GOPATH=$(pwd)/vendor go get github.com/40t/go-sniffer package plugin: unrecognized import path "plugin" (import path does not begin with hostname) [root❄anhk-pc:go-sniffer]☭

    opened by anhk 2
  • 在Mac下安装报错

    在Mac下安装报错

    之前一直都在Centos下使用,没遇到过问题。这次装到自己的Macbook pro上,在安装时候报错了。

    go get -v -u github.com/40t/go-sniffer
    github.com/40t/go-sniffer (download)
    github.com/google/gopacket (download)
    github.com/google/gopacket
    github.com/40t/go-sniffer/plugSrc/mongodb/build/internal/json
    github.com/40t/go-sniffer/plugSrc/http/build
    github.com/40t/go-sniffer/plugSrc/redis/build
    github.com/40t/go-sniffer/plugSrc/mysql/build
    github.com/google/gopacket/layers
    github.com/40t/go-sniffer/plugSrc/mongodb/build/bson
    github.com/40t/go-sniffer/plugSrc/mongodb/build
    github.com/google/gopacket/pcap
    github.com/google/gopacket/tcpassembly
    github.com/google/gopacket/tcpassembly/tcpreader
    github.com/40t/go-sniffer/core
    github.com/40t/go-sniffer
    # github.com/40t/go-sniffer
    /usr/local/Cellar/go/1.8.3/libexec/pkg/tool/darwin_amd64/link: /usr/local/Cellar/go/1.8.3/libexec/pkg/tool/darwin_amd64/link: combining dwarf failed: Unknown load command 0x32 (50)
    

    这个是跟go的版本有关系吗?我的版本是:

    go version go1.8.3 darwin/amd64
    
    opened by chao 2
  • centos install error

    centos install error

    install error in centos, I think you could provide a binary file of multi system os.

    OS Version

    cat /etc/issue
    
    CentOS release 6.8 (Final)
    Kernel \r on an \m
    

    install

    yum -y install libcap
    yum -y install libcap-devel
    
    go get -v -u github.com/40t/go-sniffer
    github.com/40t/go-sniffer (download)
    github.com/google/gopacket (download)
    # runtime/debug
    gc 1 @0.005s 6%: 0.029+1.2+0.069 ms clock, 0.11+0.17/1.1/1.1+0.27 ms cpu, 4->4->3 MB, 5 MB goal, 4 P
    gc 2 @0.011s 6%: 0.004+1.4+0.048 ms clock, 0.019+0.057/1.3/2.6+0.19 ms cpu, 5->6->5 MB, 6 MB goal, 4 P
    github.com/google/gopacket/pcap
    # github.com/google/gopacket/pcap
    /devops/app/go/src/github.com/google/gopacket/pcap/pcap.go:22:18: fatal error: pcap.h: No such file or directory
     #include <pcap.h>
    
    opened by rfyiamcool 2
  • ./bin/go-sniffer eth0 mysql

    ./bin/go-sniffer eth0 mysql

    2018-12-05 14:41:24| ser -> cli |【Ok】 Effect Row:0 2018-12-05 14:41:24| ser -> cli |【Ok】 Effect Row:0 panic: runtime error: index out of range

    goroutine 34 [running]: github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveServerPacket(0xc420418000, 0xc420456200, 0x0, 0x200, 0xb) /gomeo2o/soft/go-sniffer/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:219 +0x50d github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc420418000) /gomeo2o/soft/go-sniffer/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:194 +0xc0 created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream /gomeo2o/soft/go-sniffer/src/github.com/40t/go-sniffer/plugSrc/mysql/build/entry.go:71 +0x3ff

    opened by runblood 1
  • 阿里云Centos7上下载报错

    阿里云Centos7上下载报错

    github.com/google/gopacket/pcap

    src/github.com/google/gopacket/pcap/pcap.go:22:18: fatal error: pcap.h: No such file or directory #include <pcap.h> ^ compilation terminated.

    opened by xulei131401 1
  • Resolve Crash

    Resolve Crash

    panic: runtime error: index out of range [0] with length 0

    goroutine 454 [running]: github.com/40t/go-sniffer/plugSrc/mysql/build.(*Stmt).BindArgs(0xc1884aa380, {0xc188a3320a, 0x1, 0xc18848fe88?}, {0x0, 0x0, 0x0?}, {0x0, 0x0, 0x0}) /root/go/pkg/mod/github.com/40t/[email protected]/plugSrc/mysql/build/stmt.go:79 +0xbd4 github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolveClientPacket(0xc00006b3b0?, {0xc188a33200, 0x14, 0x600}, 0x0?) /root/go/pkg/mod/github.com/40t/[email protected]/plugSrc/mysql/build/entry.go:344 +0x36d github.com/40t/go-sniffer/plugSrc/mysql/build.(*stream).resolve(0xc00006b3b0) /root/go/pkg/mod/github.com/40t/[email protected]/plugSrc/mysql/build/entry.go:193 +0x6b created by github.com/40t/go-sniffer/plugSrc/mysql/build.(*Mysql).ResolveStream /root/go/pkg/mod/github.com/40t/[email protected]/plugSrc/mysql/build/entry.go:71 +0x3f0

    opened by vzong 0
  •  mysql连接上去的无法被抓取

    mysql连接上去的无法被抓取

    两个虚拟机 [[email protected] ~]# go-sniffer ens33 mysql -p 3306 tcp and port 3306

    Start new stream: 192.168.137.151->192.168.137.137 40944->3306

    Start new stream: 192.168.137.151->192.168.137.137 41996->3306

    Start new stream: 192.168.137.137->192.168.137.151 3306->41996

    Start new stream: 192.168.137.151->192.168.137.137 41996->3306

    Start new stream: 192.168.137.151->192.168.137.137 43058->3306

    只能看到链接的信息,但是151操作137的相关操作,却看不到?为啥,

    opened by hope1234567hope 1
  • Optimize the output format of redis requests

    Optimize the output format of redis requests

    Before

    tcp and port 6379
    # Start new stream: 127.0.0.72->127.0.0.72 59640->6379
     INFO
    # Start new stream: 127.0.0.72->127.0.0.72 6379->59640
     CONFIG GET maxmemory
    # Start new stream: 127.0.0.72->127.0.0.72 60578->6379
     PING
    # Start new stream: 127.0.0.72->127.0.0.72 6379->60578
    # Start new stream: 127.0.0.1->127.0.0.1 41712->6379
     SLOWLOG GET 11
    # Start new stream: 127.0.0.1->127.0.0.1 6379->41712
     INFO
     CONFIG GET maxmemory
     SLOWLOG GET 11
     INFO
     CONFIG GET maxmemory
     SLOWLOG GET 11
    # Start new stream: 127.0.0.72->127.0.0.72 60580->6379
     PING
    # Start new stream: 127.0.0.72->127.0.0.72 6379->60580
     INFO
     CONFIG GET maxmemory
     SLOWLOG GET 11
     SLOWLOG GET 11
    # Start new stream: 127.0.0.1->127.0.0.1 44172->6379
    # Start new stream: 127.0.0.1->127.0.0.1 6379->44172
     PING
     CLIENT SETNAME redis_exporter
     CONFIG GET *
     INFO ALL
     INFO
     CONFIG GET maxmemory
     SLOWLOG LEN
     SLOWLOG GET 1
     PING
     SLOWLOG GET 11
     INFO
     CONFIG GET maxmemory
     SLOWLOG GET 11
     INFO
     CONFIG GET maxmemory
    

    After

    tcp and port 6379
    # Start new stream: 127.0.0.72->127.0.0.72 59640->6379
    "1612323922 [127.0.0.72:59640] INFO"
    # Start new stream: 127.0.0.72->127.0.0.72 6379->59640
    "1612323922 [127.0.0.72:59640] CONFIG GET maxmemory"
    # Start new stream: 127.0.0.1->127.0.0.1 41712->6379
    "1612323923 [127.0.0.1:41712] SLOWLOG GET 11"
    # Start new stream: 127.0.0.1->127.0.0.1 6379->41712
    # Start new stream: 127.0.0.72->127.0.0.72 60580->6379
    "1612323923 [127.0.0.72:60580] PING"
    # Start new stream: 127.0.0.72->127.0.0.72 6379->60580
    "1612323923 [127.0.0.72:59640] INFO"
    "1612323923 [127.0.0.72:59640] CONFIG GET maxmemory"
    "1612323924 [127.0.0.1:41712] SLOWLOG GET 11"
    "1612323924 [127.0.0.72:59640] INFO"
    "1612323924 [127.0.0.72:59640] CONFIG GET maxmemory"
    "1612323925 [127.0.0.1:41712] SLOWLOG GET 11"
    # Start new stream: 127.0.0.72->127.0.0.72 60578->6379
    "1612323925 [127.0.0.72:60578] PING"
    # Start new stream: 127.0.0.72->127.0.0.72 6379->60578
    "1612323926 [127.0.0.72:59640] INFO"
    "1612323926 [127.0.0.72:59640] CONFIG GET maxmemory"
    "1612323926 [127.0.0.1:41712] SLOWLOG GET 11"
    "1612323927 [127.0.0.1:41712] SLOWLOG GET 11"
    "1612323927 [127.0.0.72:59640] INFO"
    "1612323927 [127.0.0.72:59640] CONFIG GET maxmemory"
    "1612323928 [127.0.0.1:41712] SLOWLOG GET 11"
    "1612323928 [127.0.0.72:59640] INFO"
    "1612323928 [127.0.0.72:59640] CONFIG GET maxmemory"
    "1612323928 [127.0.0.72:60580] PING"
    "1612323929 [127.0.0.1:41712] SLOWLOG GET 11"
    "1612323929 [127.0.0.72:59640] INFO"
    "1612323929 [127.0.0.72:59640] CONFIG GET maxmemory"
    "1612323930 [127.0.0.1:41712] SLOWLOG GET 11"
    "1612323930 [127.0.0.72:59640] INFO"
    "1612323930 [127.0.0.72:59640] CONFIG GET maxmemory"
    "1612323930 [127.0.0.72:60578] PING"
    "1612323931 [127.0.0.1:41712] SLOWLOG GET 11"
    "1612323931 [127.0.0.72:59640] INFO"
    "1612323931 [127.0.0.72:59640] CONFIG GET maxmemory"
    "1612323932 [127.0.0.1:41712] SLOWLOG GET 11"
    "1612323932 [127.0.0.72:59640] INFO"
    
    opened by yz1509 0
Owner
Four
✉️ [email protected] 百度招人:核心项目,后端架构
Four
A simple tool to convert socket5 proxy protocol to http proxy protocol

Socket5 to HTTP 这是一个超简单的 Socket5 代理转换成 HTTP 代理的小工具。 如何安装? Golang 用户 # Required Go 1.17+ go install github.com/mritd/[email protected] Docker 用户 docker pull m

mritd 7 Sep 7, 2022
Um chat feito em go utilizando gorilla/websocket, go-redis/redis,golang-jwt/jwte labstack/echo.

go-chat Um chat feito em go utilizando gorilla/websocket, go-redis/redis,golang-jwt/jwte labstack/echo. Why Eu gostaria de aprender algumas ferramenta

Kevin Souza 0 Jul 14, 2022
wire protocol for multiplexing connections or streams into a single connection, based on a subset of the SSH Connection Protocol

qmux qmux is a wire protocol for multiplexing connections or streams into a single connection. It is based on the SSH Connection Protocol, which is th

Jeff Lindsay 202 Nov 30, 2022
go HTTP client that makes it plain simple to configure TLS, basic auth, retries on specific errors, keep-alive connections, logging, timeouts etc.

goat Goat, is an HTTP client built on top of a standard Go http package, that is extremely easy to configure; no googling required. The idea is simila

VSPAZ 1 Jun 25, 2022
PlanB: a HTTP and websocket proxy backed by Redis and inspired by Hipache.

PlanB: a distributed HTTP and websocket proxy What Is It? PlanB is a HTTP and websocket proxy backed by Redis and inspired by Hipache. It aims to be f

vinay badhan 1 Mar 20, 2022
Kick dropper is a very simple and leightweight demonstration of SQL querying, and injection by parsing URl's

__ __ __ __ _____ ______ | |/ |__|.----.| |--.______| \.----.| |.-----.-----.-----.----.

RE43P3R 2 Feb 6, 2022
File Collector is an application that uses HTTP protocol to collect homework and files.

File Collector File Collector is an application that uses HTTP protocol to collect homework and files. Usage When filecollector is run, it will automa

毛亚琛 1 Jun 16, 2022
A Go library for fetching, parsing, and updating RSS feeds.

rss RSS is a small library for simplifying the parsing of RSS and Atom feeds. The package could do with more testing, but it conforms to the RSS 1.0,

Jamie Hall 373 Nov 24, 2022
A plugin of protoc that for using a service of Protocol Buffers as http.Handler definition

protoc-gen-gohttp protoc-gen-gohttp is a plugin of protoc that for using a service of Protocol Buffers as http.Handler definition. The generated inter

John_Suu 1 Dec 9, 2021
TritonHTTP - A simple web server that implements a subset of the HTTP/1.1 protocol specification

TritonHTTP Spec Summary Here we provide a concise summary of the TritonHTTP spec. You should read the spec doc for more details and clarifications. HT

Boyu Chen 1 Nov 5, 2022
IPIP.net officially supported IP database ipdb format parsing library

IPIP.net officially supported IP database ipdb format parsing library

null 240 Nov 15, 2022
Peg, Parsing Expression Grammar, is an implementation of a Packrat parser generator.

PEG, an Implementation of a Packrat Parsing Expression Grammar in Go A Parsing Expression Grammar ( hence peg) is a way to create grammars similar in

Andrew Snodgrass 879 Nov 16, 2022
scrapligo -- is a Go library focused on connecting to devices, specifically network devices (routers/switches/firewalls/etc.) via SSH and NETCONF.

scrapligo -- scrap(e c)li (but in go!) -- is a Go library focused on connecting to devices, specifically network devices (routers/switches/firewalls/etc.) via SSH and NETCONF.

null 150 Nov 4, 2022
Service registration and discovery, support etcd, zookeeper, consul, etc.

discox 支持类型 zookeeper etcd consul 示例 zookeeper server package main import ( "fmt" "github.com/goeasya/discox" "os" ) func main() { cfg := discox

goeasya 5 Aug 31, 2022
Http-server - A HTTP server and can be accessed via TLS and non-TLS mode

Application server.go runs a HTTP/HTTPS server on the port 9090. It gives you 4

Vedant Pareek 0 Feb 3, 2022
Gos: Armed Golang 💪 ( solutions for go module, goproxy, cross compilation, etc.)

The current gos is still an alpha version, welcome more heroes to comment and improve it ?? , you can add more commands to it, or modify something to make it perform better.

storyicon 373 Sep 15, 2022
Control your legacy Reciva based internet radios (Crane, Grace Digital, Tangent, etc.) via REST api or web browser.

reciva-web-remote Control your legacy Reciva based internet radios (Crane, Grace Digital, Tangent, etc.) via REST api or web browser. Usage This progr

null 6 May 3, 2022
Todo-app-grpc - Go/GRPC codebase containing RealWorld examples (CRUD, auth, advanced patterns, etc)

Go/GRPC codebase containing RealWorld examples (CRUD, auth, advanced patterns, e

Sammi Aldhi Yanto 5 Oct 12, 2022