Simple grpc web and grpc transcoding with Envoy

Overview

gRPC Web and gRPC Transcoding with Envoy

This is a simple stand-alone set of configurations demonstrating both gRPC Transcoding and gRPC-Web using Envoy Proxy.

This is actually nothing new and you can find a fair amount of references for each already pretty readily. I decided to write it up all within one repository as a way to refresh my knowledge since the last time i made separate samples of each years ago: grpc-transcoding-envoy, grpc_web_with_gke.

What this tutorial sets up is two independent configurations for envoy all using different mechanisms from the browser to talk to the gRPC server.

In the first, this tutorial demonstrates gRPC Transcoding where a simple gRPC client's api calls are made through REST

In the second, gRPC Web is used to show how to compile a .proto into javascript that runs on the browser.

In both Envoy is used as a proxy for the transcoding and in the other, as an intermediary to convert grpc-web to grpc (and also to do CORS)


To use this, at a minimum you will need

  • Docker
  • golang

If you want to run all of this manually, you'll also need node (10) and protoc (3.19.1)

Setup

Since this tutorial runs everything locally, we will need to setup some overrides to reference everything by DNS locally

edit /etc/hosts

127.0.0.1 frontend.domain.com grpc.domain.com transcoding.domain.com grpcweb.domain.com

The full layout of this tutorial is:

  • grpcServer: this is a simple grpc server that will run at grpc.domain.com:50051

  • envoy: for grpc_transcoding, envoy will run at transcoding.domain.com:8080

  • envoy for grpc_web will run at grpcweb.domain.com:18080

  • frontend: this is the frontend browser application for grpc_web and runs at frontend.domain.com:8080

First get Envoy

docker cp `docker create envoyproxy/envoy-dev:latest`:/usr/local/bin/envoy .

gRPC Client Server application

To get a sense of the gRPC app, lets run just the client-server by itself.

We will be using the gRPC server for the duration of this tutorial so keep that running

cd grpc_app/

## optionally compile (you don't need to do this since the repo contains this already)
# /usr/local/bin/protoc -I ./echo    --include_imports --include_source_info   --descriptor_set_out=echo/echo.proto.pb   --go_opt=paths=source_relative   --go_out=plugins=grpc:./echo/ echo/echo.proto

# run the client/server in different shells without TLS
go run greeter_server/grpc_server.go --grpcport :50051 --insecure  
go run greeter_client/grpc_client.go --host grpc.domain.com:50051 --insecure
# now run the client/server with TLS
go run greeter_server/grpc_server.go --grpcport :50051 --tlsCert certs/grpc_server_crt.pem --tlsKey certs/grpc_server_key.pem
go run greeter_client/grpc_client.go --host grpc.domain.com:50051 --cacert certs/tls-ca.pem --servername grpc.domain.com -skipHealthCheck

Build and run the docker images locally

docker build -t grpc_app .

docker run -p 50051:50051  -t grpc_app /grpc_server  \
    --grpcport :50051
    --tlsCert /certs/grpc_server_crt.pem  \
    --tlsKey /certs/grpc_server_key.pem

docker run --net=host --add-host grpc.domain.com:127.0.0.1    -t grpc_app /grpc_client  \
    --host=grpc.domain.com:50051 --cacert /certs/tls-ca.pem  \
    --servername grpc.domain.com

gRPC Transcoding

For gRPC transcoding, please see the official documentation as well as the published specification HTTP and gRPC Transcoding.

The layout of what we are running here is

images/transcoding.png

so, create the transcoding image and run it

cd grpc_transcoding/
docker build -t envoy_transcoding .

docker run --net=host  -p 18080:18080 -t envoy_transcoding

Run the grpc_server

cd grpc_app
go run greeter_server/grpc_server.go --grpcport :50051 --tlsCert certs/grpc_server_crt.pem --tlsKey certs/grpc_server_key.pem

In a new shell, use curl and REST api calls to call the grpc server:

$ curl -s --cacert certs/tls-ca.pem -H "host: transcoding.domain.com" https://transcoding.domain.com:18080/echo.EchoServer/SayHelloUnary/hi
{
 "message": "SayHelloUnary Response cf466c3f-6509-11ec-bae1-8c8caa692a4f"
}
$ curl -s --cacert certs/tls-ca.pem -H "host: transcoding.domain.com" https://transcoding.domain.com:18080/echo.EchoServer/SayHelloServerStream/hi
[{
 "message": "SayHelloServerStream Response"
}
,{
 "message": "SayHelloServerStream Response"
}
,{
 "message": "SayHelloServerStream Response"
}
,{
 "message": "SayHelloServerStream Response"
}
,{
 "message": "SayHelloServerStream Response"
}
]

for POST, edit echo.proto, as shown,

  rpc SayHelloUnary (EchoRequest) returns (EchoReply) {
    option (google.api.http) = {
      post: "/echo.EchoServer/SayHelloUnary"
    };  
  }

recompile the docker image and run

curl -vv --cacert certs/tls-ca.pem  \
   -H "host: transcoding.domain.com"     -H 'Content-Type: application/json'  \
   -d '{"name": "foo"}'  \
   https://transcoding.domain.com:18080/echo.EchoServer/SayHelloUnary

gRPC Web

For grpc-web, there are more components involved

images/grpc_web.png

So create the frontend image that will include the protoc compiled javascript for grpc-web:

cd frontend/
docker build -t frontend  .

# now run the image
docker run --net=host -p 8080:8080 -t frontend

# run envoy
./envoy -c proxy.yaml

If you want, check if envoy is correctly returning the CORS headers:

curl -vk --cacert CA_crt.pem -H "Origin: https://frontend.domain.com:8080"     -H "Access-Control-Request-Method: GET"     -H "Access-Control-Request-Headers: Authorization, X-grpc-web"     -H "host: grpcweb.domain.com" -X OPTIONS  https://grpcweb.domain.com:18080/echo.EchoServer

Since we are running all this locally with our own custom certificates, we will need to include the CA that singed the certs into your browser.

So, in firefox, import grpc_app/certs/tls-ca.pem file into the trust store images/cert_trust.png

Now, open up your browser and go to https://frontend.doman.com:8080/...note you should not see a certificate warning (if you do, please import the CA)

Once you go to the site, you'll see a simple text box.

What that textbox does is simply makes an CORS request to the envoy server and then makes a grpc-web call once its authorized:

images/grpc_browser.png

The request and payload for the grpc-web calls are then made through envoy

images/grpc_web_request.png

images/grpc_request_payload.png


References

You might also like...
Go based grpc - grpc gateway micro service example

go-grpc-gateway-server This repository provides an example for go based microservice. Go micro services developed based on gRPC protobuf's and also us

Go-grpc - This is grpc server for golang.

go-grpc This is grpc server for golang. protocのインストール brew install protoc Golang用のプラグインのインストール go install google.golang.org/protobuf/cmd/protoc-gen-go

GRPC - Creating a gRPC service from scratch

#Go gRPC services course Creating a gRPC service from scratch Command line colle

Totem - A Go library that can turn a single gRPC stream into bidirectional unary gRPC servers

Totem is a Go library that can turn a single gRPC stream into bidirectional unar

Grpc-gateway-map-null - gRPC Gateway test using nullable values in map

Demonstrate gRPC gateway behavior with nullable values in maps Using grpc-gatewa

Todo-app-grpc - Go/GRPC codebase containing RealWorld examples (CRUD, auth, advanced patterns, etc)

Go/GRPC codebase containing RealWorld examples (CRUD, auth, advanced patterns, e

GRPC - A client-server mockup, using gRPC to expose functionality.

gRPC This is a mockup application that I built to help me visualise and understand the basic concepts of gRPC. In this exchange, the client can use a

Raft-grpc-demo - Some example code for how to use Hashicorp's Raft implementation with gRPC

raft-grpc-example This is some example code for how to use Hashicorp's Raft impl

Benthos-input-grpc - gRPC custom benthos input

gRPC custom benthos input Create a custom benthos input that receives messages f

Owner
null
Json to rpc example with envoy, go, grpc, nats

grpc-nats-envoy json to rpc example with envoy, go, grpc, redis This repo is a mirror of https://github.com/charlesonunze/grpc-redis-envoy-example It

Charles Onunze 1 Dec 7, 2021
Using Envoy Proxy to load-balance gRPC services on GKE with header value based Session Affinity

Using Envoy Proxy to load-balance gRPC services on GKE with header value based S

Daniel William Clarke 2 Aug 24, 2022
protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript clients that connect the web frontend and golang backend fronted by grpc-gateway.

protoc-gen-grpc-gateway-ts protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript

gRPC Ecosystem 82 Sep 28, 2022
Let's implement some basic ZeroMQ publisher and subscriber in Golang. Utilize Envoy as a proxy.

Envy proxy with ZeroMQ Solution tested on DigitalOcean Droplet. In case of re-creation VM follow this article. Introduction Let's implement some basic

Jakub Wołynko 0 Jan 25, 2022
Envoy filters in Go

EGo-Demo This is a demo of how to build a Golang filter for Envoy, based on the Envoy Filter Example project, by using Go's CGo feature. It is still a

Grab 34 Jun 10, 2022
Envoy introspection for golang

Envoy introspection Demo Build the extension (.wasm file): make wasm Start the upstream service: docker run -d -p 3030:80 kennethreitz/httpbin Run th

Peter Jausovec 1 Nov 30, 2021
Go-grpc-tutorial - Simple gRPC server/client using go

Simple gRPC server/client using go Run server go run usermgmt_server/usermgmt_

Renner Poveda 0 Feb 14, 2022
Server and client implementation of the grpc go libraries to perform unary, client streaming, server streaming and full duplex RPCs from gRPC go introduction

Description This is an implementation of a gRPC client and server that provides route guidance from gRPC Basics: Go tutorial. It demonstrates how to u

Joram Wambugu 0 Nov 24, 2021
A suite of gRPC debugging tools. Like Fiddler/Charles but for gRPC.

grpc-tools A suite of tools for gRPC debugging and development. Like Fiddler/Charles but for gRPC! The main tool is grpc-dump which transparently inte

Bradley Kemp 1.1k Sep 22, 2022
grpc-http1: A gRPC via HTTP/1 Enabling Library for Go

grpc-http1: A gRPC via HTTP/1 Enabling Library for Go This library enables using all the functionality of a gRPC server even if it is exposed behind a

StackRox 85 Sep 30, 2022