Using Wireshark to decrypt TLS gRPC Client-Server protobuf messages

Overview

Using Wireshark to decrypt TLS gRPC Client-Server protobuf messages

Sample client server in golang that demonstrates how to decode protobuf messages for a gRPC Client->Server over TLS.

This is nothing new and is described here in wireshark's documentation (How to Export TLS Master keys of gRPC)

What drove me to write this sample was a co-woker who as interested in viewing gRPC protbuf message in java for Google Cloud Pubsub.

Since i earlier worked on similar stuff in other repos like

i figured i'd give it a shot....fast forward, while i could decrypt the pubsub TLS traffic, wasn't able to get the correct .proto Google Cloud uses to properly parse inside wireshark's dissectors.

I'm not sure what the issue with wireshark is but i suspect its a mismatch of the proto version or annotations google added in.

Anyway, what this repo does show is a very simple go-based client/server gRPC system that does unary, client streaming, server streaming and full bidi. All along the tls traffic is seen by wireshark, decrypted and after that, parsed using the .proto files

Lets get started

Sample gRPC Client/Server

First thing is to setup the client server and wireshark.

For wireshark, we will save the TLS encryption keys to /tmp/keylog.log. This file contains the TLS keys in NSS Key Log Format which golang can output and wireshark can consume.

First tell wireshark where to find it

images/wireshark_keylog.png

Now we need to tell whreshark how to decode the proto after TLS is done. For that, we will configure wireshark to look at src/echo/echo.proto

images/wireshark_proto.png

Now we're ready to start the gRPC Server

# optionally, if you really want to compile from source...
# protoc --go_out=. --go_opt=paths=source_relative --go-grpc_opt=require_unimplemented_servers=false --go-grpc_out=. --go-grpc_opt=paths=source_relative src/echo/echo.proto

  go run src/grpc_server.go \
    --grpcport 0.0.0.0:50051 \
    --tlsCert=certs/grpc_server_crt.pem \
    --tlsKey=certs/grpc_server_key.pem

Now start wireshark. In a new window run

sudo wireshark

Once you start wireshark, select the 'lo' (local interface)

Run the client but first tell it where to log the keys

export SSLKEYLOGFILE=/tmp/keylog.log

  go run src/grpc_client.go \
    --host 127.0.0.1:50051 \
    --tlsCert=certs/CA_crt.pem \
    --servername=grpc.domain.com

Note, to use go, you need to first ask it to dump the keylog per crypto.tls.Config.KeyLogWriter. Please take careful note of the dangers described there...

The output of the various modes

Unary

images/unary.png

Client Streaming

images/grpc_client_stream.png

Server Streaming

images/grpc_server_stream.png

BiDi Streaming

client->server:

images/grpc_bidi_req.png

server->client:

images/grpc_bidi_response.png

note the Stream ID 15 is used in both paths

I've also left a sample keylog and wireshark file for you to load and test decoding

google PubSub

Ok, so now we're back to what i really wanted to do...decode PubSub.

As mentioned, i was unsuccessful (partially)...i could decrypt but not decode

I think it should be something like this

# first get all the protos
git clone https://github.com/googleapis/googleapis.git
git clone https://github.com/protocolbuffers/protobuf.git

Then tell wireshark to load the proto files << this is the point where i'm stuck. Wireshark keeps complaining about annotations that appear as syntax errors..

Anyway, if you ran the grpc client in this repo, and prior to that set the SSLKEYLOGFILE parameter, you should be able to see the pubsub traffic if you use the following filter in wireshark:

tcp.port == 443 and tls.handshake.extensions_server_name=="pubsub.googleapis.com"

However, even if you cant' see the traffic, you can still see the bytes...presumably, you can dump the DATA to a file from wireshark, then use go to read each payload as protoMessage

images/pubsub.png


JAVA

I tried to use jSSLKeyLog with Cloud PubSub in Java and while it did dump the keys to a file, I wasn't able to use wireshark to decrypt the data

I've left off as far as i got with that here in he repo

YMMV

# run standalone
mvn clean install exec:java

# create fatjar
mvn clean package

# or as package
java -jar target/TestApp-1.0-SNAPSHOT-jar-with-dependencies.jar
A terminal UI for tshark, inspired by Wireshark

Termshark A terminal user-interface for tshark, inspired by Wireshark. V2.2 is out now with vim keys, packet marks, a command-line and themes! See the

Graham Clark 7.2k May 20, 2022
Server and client implementation of the grpc go libraries to perform unary, client streaming, server streaming and full duplex RPCs from gRPC go introduction

Description This is an implementation of a gRPC client and server that provides route guidance from gRPC Basics: Go tutorial. It demonstrates how to u

Joram Wambugu 0 Nov 24, 2021
protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript clients that connect the web frontend and golang backend fronted by grpc-gateway.

protoc-gen-grpc-gateway-ts protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript

gRPC Ecosystem 63 May 9, 2022
GRPC - A client-server mockup, using gRPC to expose functionality.

gRPC This is a mockup application that I built to help me visualise and understand the basic concepts of gRPC. In this exchange, the client can use a

Fergal Bittles 0 Jan 4, 2022
Go-grpc-tutorial - Simple gRPC server/client using go

Simple gRPC server/client using go Run server go run usermgmt_server/usermgmt_

Renner Poveda 0 Feb 14, 2022
Ephemeral One Time/Build-Time gRPC TLS PKI system.

PkiSauce Ephemeral Build Time TLS PKI saucing for your intra services GRPC (or not) communications. Description A simple attempt to avoid deploying co

Ermites.IO 3 Dec 14, 2021
Orion - a small lightweight framework written around grpc/protobuf with the aim to shorten time to build microservices at Carousell.

Orion Orion is a small lightweight framework written around grpc/protobuf with the aim to shorten time to build microservices at Carousell. It is deri

Carousell 139 May 8, 2022
Go-grpc - This is grpc server for golang.

go-grpc This is grpc server for golang. protocのインストール brew install protoc Golang用のプラグインのインストール go install google.golang.org/protobuf/cmd/protoc-gen-go

jotaro yuza 1 Jan 2, 2022
Grpc-gateway-map-null - gRPC Gateway test using nullable values in map

Demonstrate gRPC gateway behavior with nullable values in maps Using grpc-gatewa

null 1 Jan 6, 2022
Grpc-train - Train booking demo using gRPC

gRPC Demo: Train Booking Service Description Usage Contributing Development Tool

Fadi Asfour 0 Feb 6, 2022
Client - Server TCP Chat For String Messages And Random Files

GoChat Client - Server TCP Chat For String Messages And Random Files GoChat is a chat for string messages and random files using Golorem by Derek A. R

Brian GL 0 Sep 29, 2021
go HTTP client that makes it plain simple to configure TLS, basic auth, retries on specific errors, keep-alive connections, logging, timeouts etc.

goat Goat, is an HTTP client built on top of a standard Go http package, that is extremely easy to configure; no googling required. The idea is simila

VSPAZ 1 Nov 18, 2021
Experiments with pulsar go client and protobuf

How to use proto schema with pulsar go client library Define a .proto file Generate Code using below command: protoc -I=. -I=$GOPATH/src -I=$GOPATH/sr

Deepak Sah 1 Nov 15, 2021
A suite of gRPC debugging tools. Like Fiddler/Charles but for gRPC.

grpc-tools A suite of tools for gRPC debugging and development. Like Fiddler/Charles but for gRPC! The main tool is grpc-dump which transparently inte

Bradley Kemp 1.1k Apr 27, 2022
grpc-http1: A gRPC via HTTP/1 Enabling Library for Go

grpc-http1: A gRPC via HTTP/1 Enabling Library for Go This library enables using all the functionality of a gRPC server even if it is exposed behind a

StackRox 76 May 2, 2022
Go based grpc - grpc gateway micro service example

go-grpc-gateway-server This repository provides an example for go based microservice. Go micro services developed based on gRPC protobuf's and also us

Suresh Yekasiri 0 Dec 8, 2021
Simple grpc web and grpc transcoding with Envoy

gRPC Web and gRPC Transcoding with Envoy This is a simple stand-alone set of con

null 0 Dec 25, 2021
GRPC - Creating a gRPC service from scratch

#Go gRPC services course Creating a gRPC service from scratch Command line colle

Rafael Diaz Miles 1 Jan 2, 2022