Hi folks,

I'm a bit new to the proto and trying to replace the existing grpc with drpc in my project.

I look into the doc and it says "Place protoc-gen-go-drpc in your $PATH and have protoc generate your definitions", but find nowhere to get the binary protoc-gen-go-drpc. There is only a simple main.go and useless README under storj.io/drpc/cmd/protoc-gen-go-drpc.

May I know how to generate the drpc go code based on the proto I already have and where to get the required tool?

Appreciate for your help!

i need to transfer error message from drpc server to drpc client, nor error type is not exportable. Do you plan to export Error type and ability to get error message ? also it will be very useful to copy codes from grpc ( https://pkg.go.dev/google.golang.org/grpc/codes?utm_source=godoc ) via const to drpc

https://github.com/storj/drpc/blob/41e8fa5f513fe0893d3bef72310d0ccbba66cc74/drpcwire/reader.go#L119

It appears the maximum payload size is 4MB right now and isn't configurable. Is this an intentional design choice?

if you change generator and generate for each method const variable name like

const (
  accountAccontServiceCreate = "/account.AccountService/Create"
)

func (c *drpcAccountServiceClient) Create(ctx context.Context, in *AccountCreateReq) (*Account, error) {
        out := new(Account)
        err := c.cc.Invoke(ctx, accountAccontServiceCreate, drpcEncoding_File_account_proto{}, in, out)
        if err != nil {
                return nil, err
        }
        return out, nil
}

we completely can avoid new string allocation for each request and minimize garbage. what you think ?

The previous behavior left a potential "CloseSend" frame pending on the manager's read queue. This delayed detection of a potential connection loss until the next RPC is started on the connection, which discards the old frame because of the stream ID mismatch.

I'm sure the test can be improved; if only by simplifying testTransport.

This change is a requirement for potential solutions to #11.

grpc has grpc.MaxRecvMsgSize and grpc.MaxSendMsgSize which restrict the maximum amount of data which can be sent or received by a grpc client/server.

What is the strategy that drpc uses? I did some tests with sending large amounts of data with the defaults. 1024 * 1024 bytes works fine. 1024 * 1024 * 100 bytes causes a data overflow protocol error from the server.

Looking at the code I can see some various

type ReaderOptions struct {
	// MaximumBufferSize controls the maximum size of buffered
	// packet data.
	MaximumBufferSize int
}

and

// Options controls configuration settings for a stream.
type Options struct {
	// SplitSize controls the default size we split packets into frames.
	SplitSize int

	// ManualFlush controls if the stream will automatically flush after every
	// message send. Note that flushing is not part of the drpc.Stream
	// interface, so if you use this you must be ready to type assert and
	// call RawFlush dynamically.
	ManualFlush bool

	// MaximumBufferSize causes the Stream to drop any internal buffers that
	// are larger than this amount to control maximum memory usage at the
	// expense of more allocations. 0 is unlimited.
	MaximumBufferSize int

	// Internal contains options that are for internal use only.
	Internal drpcopts.Stream
}

However, it isn't really clear the strategy or what I should be setting. Can anyone help here?

Some things that could use explicit documentation if one were to write a second, compatible implementation:

• [x] The stream state machine with associated expected errors during different states. (https://github.com/storj/drpc/wiki/Docs:-Stream-state-machine)
• [x] The wire format (https://github.com/storj/drpc/wiki/Docs:-Wire-protocol)
• [x] Other decisions that were made around multiplexing, frame assembly, stream liveness, etc. (https://github.com/storj/drpc/wiki/Docs:-Project-goals)

A good test would be to have someone other than those who have already worked on the code try to implement it in another language using only the documentation.

Hi guys,

I am trying to replace gRPC with dRPC in my testing project.

There are a few proto files in multiple directories and under the same directory "pkg", and some basic protos like "github.com.google/protobuf" under "vendor" directory. When I try to generate the pb.go files, it keeps returning the error:

protoc-gen-go-drpc: invalid Go import path "descriptor" for "google/protobuf/descriptor.proto"
The import path must contain at least one period ('.') or forward slash ('/') character.
See https://developers.google.com/protocol-buffers/docs/reference/go-generated#package for more information.

Here is my gen cmd:

for dir in ./pkg/abc/ ./pkg/efg; do \
  build/werror.sh /go_path/native/x86_64-pc-linux-gnu/protobuf/protoc
-Ipkg:./vendor/github.com:./vendor/github.com/gogo/protobuf:./vendor/github.com/gogo/protobuf/protobuf:./vendor/go.etcd.io:./vendor/github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis 
--go-drpc_out=paths=source_relative,protolib=github.com/gogo/protobuf $dir/*.proto;

There indeed a file "./vendor/github.com/gogo/protobuf/protobuf/google/protobuf/descriptor.proto" with "option go_package = descriptor". But it's pulled automatically and already exists in my local repo for a long time. I doubt the option "-I" not work with drpc-gen, so it try to locate the basic gogo proto from a wrong location?

I've beed debugging for this issue for more than a week and look up all docs I could find but got no help.

Appreciate for any help!

https://pkg.go.dev/google.golang.org/grpc?utm_source=godoc#UnknownServiceHandler

i'm really need this to be able to dynamically add handlers and also have ability to send custom response for unknown handler. can you implements something like this in drpc ?

i'm maintain a fork of go-micro ( github.com/unistack-org/micro ) and want to add drpc client/server support to it. grpc does not have native ability to pass context to server and also does not allow adding handler timeouts easy (via context with timeout or something like this) What you think about graceful shutdown of dprc handlers ? Do you plan to add option for server to specify handler timeout?

We've started rolling out drpc into production, and so far its been great! However, one issue we've run into is what appears to be memory leaks in the server. After looking at the memory profiles for a bit we see a bunch of memory being held by drpcwire reader ReadPacketUsing. My conjecture is that there are one of these per connection on the server, and each is holding a buffer which grows to the maximum message size that has ever been read over that connection.

Is there a way to limit the maximum number of connections a server will allow? I see the InactivityTimeout is configurable, but no parameter for max connections.

The debug logging is pretty inflexible. It would be nice to be able to override the logging function, but at a minimum using log.Output vs creating a new standard logger would at least enable me to capture the data into our standard logging vs redirecting stdout.

I'd like to log address data on both the client & the server. Is that data currently accessible?

On the server side drpc.Stream doesn't seem to expose any method to do that.

On the client side I'm logging right now by wrapping a drpcconn.Conn, and at that point the server isn't known.

Hi,

I'm looking into replacing grpc with drpc in one of my projects and initial tests show some nice performance improvements!

Switching to drpc was fairly painless. The only snag was that context cancelling seemingly closes the connection. This behavior is different from grpc, see my tweaked drpc/grpc examples in https://github.com/tomyl/drpc/commit/acb08bd89225fe5db68ae4dd9fa82805d28b7364.

Is this behavior intentional? I can't see that the documentation mentions it.

Thanks, Tommy

Hello everyone, I ran into a reconnect problem, if the server crashes in a panic and recovers, the client via tcp, of course, cannot reach it, except for creating a new client, I did not find a way to solve the problem. Maybe someone has a ready-made reconnect practice in the background?

i'm try to create api gateway that support http rest, grpc and drpc on the same port

i'm create grpc server, drpc http handler, and plain http handler and use the in http2 server as handler.

plain http and grpc works fine, but drpc request does not goes to http2 server =( does it possible to get this worked with native drpc clients ?

gsrv := grpc.NewServer(grpc.UnknownServiceHandler(h.ServeGRPC))
comboHandler := newComboMux(h, gsrv, drpchttp.New(h))
http2Server := &http2.Server{}
hs := &http.Server{Handler: h2c.NewHandler(comboHandler, http2Server)}

func (h *Handler) ServeDRPC(stream drpc.Stream, rpc string) error {
	ctx := stream.Context()
	logger.Infof(ctx, "drpc: %#+v", rpc)
	return nil
}

func (h *Handler) HandleRPC(stream drpc.Stream, rpc string) error {
	return h.ServeDRPC(stream, rpc)
}

func newComboMux(httph http.Handler, grpch http.Handler, drpch http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if r.ProtoMajor == 2 {
			ct := r.Header.Get("content-type")
			switch {
			case strings.HasPrefix(ct, "application/grpc"):
				grpch.ServeHTTP(w, r)
				return
			case strings.HasPrefix(ct, "application/drpc"):
				drpch.ServeHTTP(w, r)
				return
			}
		}
		httph.ServeHTTP(w, r)
	})
}