Protocol Buffers for Go with Gadgets

Overview

GoGo Protobuf looking for new ownership

Protocol Buffers for Go with Gadgets

Build Status GoDoc

gogoprotobuf is a fork of golang/protobuf with extra code generation features.

This code generation is used to achieve:

  • fast marshalling and unmarshalling
  • more canonical Go structures
  • goprotobuf compatibility
  • less typing by optionally generating extra helper code
  • peace of mind by optionally generating test and benchmark code
  • other serialization formats

Keeping track of how up to date gogoprotobuf is relative to golang/protobuf is done in this issue

Release v1.3.0

The project has updated to release v1.3.0. Check out the release notes here.

With this new release comes a new internal library version. This means any newly generated *pb.go files generated with the v1.3.0 library will not be compatible with the old library version (v1.2.1). However, current *pb.go files (generated with v1.2.1) should still work with the new library.

Please make sure you manage your dependencies correctly when upgrading your project. If you are still using v1.2.1 and you update your dependencies, one of which could include a new *pb.go (generated with v1.3.0), you could get a compile time error.

Our upstream repo, golang/protobuf, also had to go through this process in order to update their library version. Here is a link explaining hermetic builds.

Users

These projects use gogoprotobuf:

Please let us know if you are using gogoprotobuf by posting on our GoogleGroup.

Mentioned

Getting Started

There are several ways to use gogoprotobuf, but for all you need to install go and protoc. After that you can choose:

  • Speed
  • More Speed and more generated code
  • Most Speed and most customization

Installation

To install it, you must first have Go (at least version 1.6.3 or 1.9 if you are using gRPC) installed (see http://golang.org/doc/install). Latest patch versions of 1.12 and 1.15 are continuously tested.

Next, install the standard protocol buffer implementation from https://github.com/google/protobuf. Most versions from 2.3.1 should not give any problems, but 2.6.1, 3.0.2 and 3.14.0 are continuously tested.

Speed

Install the protoc-gen-gofast binary

go get github.com/gogo/protobuf/protoc-gen-gofast

Use it to generate faster marshaling and unmarshaling go code for your protocol buffers.

protoc --gofast_out=. myproto.proto

This does not allow you to use any of the other gogoprotobuf extensions.

More Speed and more generated code

Fields without pointers cause less time in the garbage collector. More code generation results in more convenient methods.

Other binaries are also included:

protoc-gen-gogofast (same as gofast, but imports gogoprotobuf)
protoc-gen-gogofaster (same as gogofast, without XXX_unrecognized, less pointer fields)
protoc-gen-gogoslick (same as gogofaster, but with generated string, gostring and equal methods)

Installing any of these binaries is easy. Simply run:

go get github.com/gogo/protobuf/proto
go get github.com/gogo/protobuf/{binary}
go get github.com/gogo/protobuf/gogoproto

These binaries allow you to use gogoprotobuf extensions. You can also use your own binary.

To generate the code, you also need to set the include path properly.

protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=. myproto.proto

To use proto files from "google/protobuf" you need to add additional args to protoc.

protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=\
Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/struct.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,\
Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types:. \
myproto.proto

Note that in the protoc command, {binary} does not contain the initial prefix of "protoc-gen".

Most Speed and most customization

Customizing the fields of the messages to be the fields that you actually want to use removes the need to copy between the structs you use and structs you use to serialize. gogoprotobuf also offers more serialization formats and generation of tests and even more methods.

Please visit the extensions page for more documentation.

Install protoc-gen-gogo:

go get github.com/gogo/protobuf/proto
go get github.com/gogo/protobuf/jsonpb
go get github.com/gogo/protobuf/protoc-gen-gogo
go get github.com/gogo/protobuf/gogoproto

GRPC

It works the same as golang/protobuf, simply specify the plugin. Here is an example using gofast:

protoc --gofast_out=plugins=grpc:. my.proto

See https://github.com/gogo/grpc-example for an example of using gRPC with gogoprotobuf and the wider grpc-ecosystem.

License

This software is licensed under the 3-Clause BSD License ("BSD License 2.0", "Revised BSD License", "New BSD License", or "Modified BSD License").

Comments
  • grpc-gateway calls golang/protobuf/jsonpb

    grpc-gateway calls golang/protobuf/jsonpb

    This particular bug bit us:

    To properly serialize/deserailize enums in jsonpb you need to call RegisterEnum. gogo/protobuf-generated messages register with gogo/protobuf/properties.go. However, if you have a library that is not under your control (e.g. gRPC Gateway), it will be using standard golang/jsonpb and thus the golang/protobuf/properties registration. Thus, any library using golang/jsonpb will not know about enums field maps.

    Can we have a fix for this? Proposals include:

    • having gogo register by only with golang/protobuf
    • having gogo register with both, or passing it up.
    question 
    opened by mwitkow 62
  • sdrawkcab fubotorp lahsraM

    sdrawkcab fubotorp lahsraM

    .)(eziS gnitupmoc-erp )yledetaeper dna( ylevisrucer diova ot ,)elbissop nehw( drawkcab dezilaires era stcejbo taht os cigol eht lla esreveR

    Fixes #559

    opened by apelisse 48
  • customtype test failures on 32-bit (and Big Endian) architectures

    customtype test failures on 32-bit (and Big Endian) architectures

    As reported in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=819994

    Builds of golang-gogoprotobuf on 32-bit architectures such as i386 and on ppc64 (the only big-endian architecture on which its build dependencies were available) all failed with test suite errors because various results weren't as expected. The details vary by architecture, per

    https://buildd.debian.org/status/logs.php?pkg=golang-gogoprotobuf&ver=0.2-1

    Could you please take a look?

    See full build logs on armel, armhf, i386 and ppc64. Tests do not fail only on amd64, arm64 and ppc64el.

    bug 
    opened by onlyjob 40
  • Work In Progress: Separate binary with default code generation behaviour for normal proto files.

    Work In Progress: Separate binary with default code generation behaviour for normal proto files.

    There are a bunch of dropbox internal customization which I'm interested in upstreaming. In particular:

    1. We've rename Size() to ProtoSize()
    2. We've enable marshaller, unmarshaller, and sizer by default

    To avoid breaking the rest of the world, I was thinking of creating a struct which would holds the default values (and we'll only patch this struct within dropbox). something like

    type DefaultParams struct {
       sizeMethodName string
       isMarshaller bool
       isUnmarshaller bool
       isSizer bool
    }
    

    the world would use:

    params := DefaultParams{"Size", false, false, false}
    

    while dropbox would override the defaults to:

    params := DefaultParams{"ProtoSize", true, true, true}
    

    Is this acceptable?

    Also, on the experimental side, I want to backport plugins from https://github.com/dropbox/goprotoc to generate proper mutator APIs (we've got burn by goproto's lack of proper mutator api a few of time recently, especially around repeated field). Is this something you would be interested in?

    Thanks, Patrick

    opened by pattyshack 36
  • Fix cases where the fix for issue260 was not enough

    Fix cases where the fix for issue260 was not enough

    The previous fix had some corner cases not covered by the useTime fix, since time.Time might only be used in the unmarhsaler plugin.

    This PR makes useTime public and adds it to the lines of unmarshaler where it is needed.

    Sorry, for having to do this fix.

    opened by Serabe 35
  • Unable to use google.rpc.Status

    Unable to use google.rpc.Status

    In our code at https://github.com/zvelo/msg/ we're using google.rpc.Status in some of our own protobufs. Unfortunately, this results in errors of the following form when building:

    ./dataset.pb.go:270:16: this.Error.Equal undefined (type
        *"zvelo.io/msg/vendor/google.golang.org/genproto/googleapis/rpc/status".Status
        has no field or method Equal)
    ./dataset.pb.go:308:16: this.Error.Equal undefined (type
        *"zvelo.io/msg/vendor/google.golang.org/genproto/googleapis/rpc/status".Status
        has no field or method Equal)
    

    Our team are temporarily working around the issue by forking our own copies of google.golang.org/genproto & github.com/golang/protobuf, but are curious if modifying our gogo generator somehow would be a better solution. Any thoughts?

    opened by robert-uhl 34
  • Known Issue: proto.MarshalTextString cannot handle custom types with Bytes method

    Known Issue: proto.MarshalTextString cannot handle custom types with Bytes method

    If a custom type implements proto.raw (i.e., has a Bytes() []byte method) proto.TextMarshaler.writeStruct will call it, causing invalid bytes to be written.

    For example, I have a UUID type with a Bytes() []byte method which returns the UUID's text format as a slice of bytes (it reduces allocations for other methods like MarshalText).

    I use this as a custom type and when I do I get the errors seen here.

    I'm not sure what your take is on this. Either there should be some docs (I know you said before custom type has issues, so if I could help let me know) or instead of Bytes() []byte the interface should be type raw interface { bytes() []byte } so no external types can implement it.

    wontfix 
    opened by ericlagergren 33
  • Merge ptypes from golang/goprotobuf

    Merge ptypes from golang/goprotobuf

    I am not sure what your policy on back-porting changes from goprotobuf, but it looks like ptypes has landed. Using these has been a pain point up until now.

    https://github.com/golang/protobuf/commit/750c6bd0827ae5f3ebc8f49c6dedacc7c5f555b4

    bug 
    opened by stevvooe 31
  • WIP: Memory pooling

    WIP: Memory pooling

    This is the initial work for memory pooling, wanted to put it up before I spent too much time on it so we could look it over. Tldr for a message with a few fields and a repeated field of size 10, 1664B and 33 allocs/op -> 0B and 0 allocs/op.

    Check out the test/pool directory.

    Note there is no change for existing users. You have to enable the (gogoproto.pool) option, and even if you enable it, there is no functional change in in terms of current usage - messages can still be allocated with or without a backing Pool.

    Some caveats:

    • String and byte slice fields are not pooled. Strings basically never could be since they are immutable, byte slices in theory could, but it would cause a world of pain and not be backwards compatible to try to pool them when they are fields. I added a MarshalPool method that does do byte slice pooling using a seg list with a wrapper *mem.Bytes type, however.

    A lot has changed since this PR was first put up. See the discussion below for the current state of this PR.

    opened by bufdev 30
  • Work In Progress: casttype (new type of customtype)

    Work In Progress: casttype (new type of customtype)

    main.go:

    package main
    
    //go:generate protoc -I. -I$GOPATH/src -I$GOPATH/src/github.com/gogo/protobuf/protobuf --gogo_out=. repro.proto
    
    type T uint32
    
    func main() {
    }
    

    repro.proto:

    package repro;
    
    option go_package = "main";
    
    import "github.com/gogo/protobuf/gogoproto/gogo.proto";
    
    option (gogoproto.unmarshaler_all) = true;
    
    message M {
      required uint32 X = 1 [(gogoproto.nullable) = false, (gogoproto.customtype) = "T"];
    }
    
    $ go generate && go build
    # _/home/tv/src/2014/protobuf-bug-type
    ./repro.pb.go:69: invalid operation: m.X |= uint32(b) & 127 << shift (mismatched types T and uint32)
    
    opened by tv42 30
  • vanity.NotInPackageGoogleProtobuf overly restrictive?

    vanity.NotInPackageGoogleProtobuf overly restrictive?

    I'm building go.pedge.io/pb, which is go.pedge.io/google-protobuf + go.pedge.io/googleapis + my own standard protobuf library, but for both golang and gogo this time. I am trying to use gogofast, but I got a ton of errors along the lines of:

    gogo/google/type/color.pb.go:202: m.Alpha.Size undefined (type *google_protobuf.FloatValue has no field or method Size)
    gogo/google/type/color.pb.go:203: m.Alpha.MarshalTo undefined (type *google_protobuf.FloatValue has no field or method MarshalTo)
    gogo/google/type/color.pb.go:252: m.Alpha.Size undefined (type *google_protobuf.FloatValue has no field or method Size)
    gogo/google/type/color.pb.go:371: m.Alpha.Unmarshal undefined (type *google_protobuf.FloatValue has no field or method Unmarshal)
    

    This is because of https://github.com/gogo/protobuf/commit/c948aac0178a73864b46b0dd098cf9075947570c, which if I understand, is basically meant to block out google/protobuf/descriptor.proto types, not the well-known types.

    Can I send a PR that makes this less restrictive? google/protobuf/descriptor.proto is a special case in google/protobuf in general, so I'd want to special case those specifically. Without changing this, using well-known types with gogofast/gogofaster/gogoslick is not possible, and people should use github.com/gogo/protobuf/protoc-gen-gogo/descriptor for google/protobuf/descriptor.proto anyways.

    question 
    opened by bufdev 29
  • How to customize the name of an enumeration value, using the extension `enumvalue_customname ` seems unable to complete.

    How to customize the name of an enumeration value, using the extension `enumvalue_customname ` seems unable to complete.

    This is the official test case: https://github.com/gogo/protobuf/blob/b03c65ea87cdc3521ede29f62fe3ce239267c1bc/test/enumcustomname/enumcustomname.proto#L38-L45 https://github.com/gogo/protobuf/blob/b03c65ea87cdc3521ede29f62fe3ce239267c1bc/test/enumcustomname/enumcustomname.pb.go#L31-L47

    In the above example, line 43 in the "enumcustomname.proto" file uses the "enumvalue_customname" extension to customize the name of the enumeration value. In the 34th page of "enumcustomname.pb.go", the variable name is the name of a custom enumeration value. However, lines 39 and 44 have not been replaced with custom enumeration names. But I think the generated code should be like this:

    const (
    	// The following field will take on the custom name and the prefix, joined
    	// by an underscore.
    	MyCustomEnum_MyBetterNameA MyCustomEnum = 0
    	MyCustomEnum_B             MyCustomEnum = 1
    )
    
    var MyCustomEnum_name = map[int32]string{
    	0: "MyBetterNameA",
    	1: "B",
    }
    
    var MyCustomEnum_value = map[string]int32{
    	"MyBetterNameA": 0,
    	"B": 1,
    }
    
    opened by MicroOps-cn 0
  • Panic: reflect: Elem of invalid type

    Panic: reflect: Elem of invalid type

    Hi,

    I'm using the following versions:

    • github.com/gogo/protobuf v1.3.2
    • github.com/golang/protobuf v1.4.3
    • go version 1.17.1
    • macOS Catalina 10.15.7

    The .proto file:

    message ServiceAttestation {
      string field1 = 1;
      string field2 = 2;
    }
    
    message ServiceAttestationRequest {
        repeated ServiceAttestation request = 1 [(gogoproto.nullable) = false];
    }
    

    The request (PUT) to the service was a json like this:

    {
        "request": [
            {
                "field1": "some_string",
                "field2": "aother_string"
            }
        ]
    }
    

    Panic message:

    2022/06/06 14:57:13 http: panic serving [::1]:64968: reflect: Elem of invalid type packagepb.ServiceAttestation
    goroutine 148 [running]:
    net/http.(*conn).serve.func1()
            GOROOT/src/net/http/server.go:1802 +0xb9
    panic({0x27de9c0, 0xc003abc760})
            GOROOT/src/runtime/panic.go:1047 +0x266
    reflect.(*rtype).Elem(0xc00013a7e0)
            GOROOT/src/reflect/type.go:954 +0x134
    google.golang.org/protobuf/internal/impl.(*messageConverter).New(0xc00149a6b0)
            external/org_golang_google_protobuf/internal/impl/convert.go:462 +0x29
    google.golang.org/protobuf/internal/impl.(*listReflect).NewElement(0x2a360e0)
            external/org_golang_google_protobuf/internal/impl/convert_list.go:134 +0x22
    github.com/golang/protobuf/jsonpb.(*Unmarshaler).unmarshalValue(0x286e820, {{}, 0x2a360e0, 0xc0035e9230, 0x7}, {0xc0001a61e0, 0xea, 0xf0}, {0x2f01c20, 0xc0009262a0})
            external/com_github_golang_protobuf/jsonpb/decode.go:402 +0x6ef
    github.com/golang/protobuf/jsonpb.(*Unmarshaler).unmarshalMessage(0xc003ab62b8, {0x2ef11b0, 0xc003abc740}, {0xc00062a800, 0x0, 0x100})
            external/com_github_golang_protobuf/jsonpb/decode.go:339 +0x1c8c
    github.com/golang/protobuf/jsonpb.(*Unmarshaler).UnmarshalNext(0xc000189180, 0x2a53ac0, {0xc79e7c0, 0xc003abee80})
            external/com_github_golang_protobuf/jsonpb/decode.go:127 +0x14e
    github.com/grpc-ecosystem/grpc-gateway/runtime.decodeJSONPb(0xfd, {0x2a53ac0, 0xc003abee80})
            external/com_github_grpc_ecosystem_grpc_gateway/runtime/marshal_jsonpb.go:176 +0x70
    github.com/grpc-ecosystem/grpc-gateway/runtime.DecoderWrapper.Decode({0xc79e7a0}, {0x2a53ac0, 0xc003abee80})
            external/com_github_grpc_ecosystem_grpc_gateway/runtime/marshal_jsonpb.go:149 +0x25
    gitlab.gs.com/sre/sre/slo-repo/proto/manualattestation.local_request_ManualAttestationService_Insert_0({0x2ea34b0, 0xc0035e91a0}, {0x2ec1f60, 0xc0014920f0}, {0x2e96a08, 0xc001492180}, 0xc0035e4f00, 0x16)
            bazel-out/darwin-fastbuild-ST-3891b650e255/bin/proto/manualattestation/manualattestationpb_go_proto_/gitlab.gs.com/sre/sre/slo-repo/proto/manualattestation/manualattestation.pb.gw.go:61 +0x112
    gitlab.gs.com/sre/sre/slo-repo/proto/manualattestation.RegisterManualAttestationServiceHandlerServer.func1({0x2e962e8, 0xc0009922a0}, 0xc0035e4f00, 0xc000191680)
    

    Thanks,

    opened by abaruchi 1
  • oom

    oom

    fatal error: runtime: out of memory

    runtime stack: runtime.throw(0x7e6913, 0x16) /usr/lib/golang/src/runtime/panic.go:608 +0x72 runtime.sysMap(0xc96c000000, 0x35c000000, 0xb389b8) /usr/lib/golang/src/runtime/mem_linux.go:156 +0xc7 runtime.(*mheap).sysAlloc(0xb20260, 0x35c000000, 0x80, 0xc000a33e30) /usr/lib/golang/src/runtime/malloc.go:619 +0x1c7 runtime.(*mheap).grow(0xb20260, 0x1ac19a, 0x0) /usr/lib/golang/src/runtime/mheap.go:920 +0x42 runtime.(*mheap).allocSpanLocked(0xb20260, 0x1ac19a, 0xb389c8, 0xc000a33ed8) /usr/lib/golang/src/runtime/mheap.go:848 +0x337 runtime.(*mheap).alloc_m(0xb20260, 0x1ac19a, 0xc001550101, 0x1) /usr/lib/golang/src/runtime/mheap.go:692 +0x119 runtime.(*mheap).alloc.func1() /usr/lib/golang/src/runtime/mheap.go:759 +0x4c runtime.(*mheap).alloc(0xb20260, 0x1ac19a, 0x300000101, 0xc000000300) /usr/lib/golang/src/runtime/mheap.go:758 +0x8a runtime.largeAlloc(0x358334000, 0xc000510100, 0xc000534f01) /usr/lib/golang/src/runtime/malloc.go:1019 +0x97 runtime.mallocgc.func1() /usr/lib/golang/src/runtime/malloc.go:914 +0x46 runtime.systemstack(0x0) /usr/lib/golang/src/runtime/asm_amd64.s:351 +0x66 runtime.mstart() /usr/lib/golang/src/runtime/proc.go:1229

    goroutine 1 [running]: runtime.systemstack_switch() /usr/lib/golang/src/runtime/asm_amd64.s:311 fp=0xc00ca1e640 sp=0xc00ca1e638 pc=0x453d90 runtime.mallocgc(0x358334000, 0x0, 0xc00ca1e700, 0x4c6478) /usr/lib/golang/src/runtime/malloc.go:913 +0x896 fp=0xc00ca1e6e0 sp=0xc00ca1e640 pc=0x40b916 runtime.growslice(0x770c00, 0xc6a7dd8000, 0x2acf5bfe3, 0x2acf5c000, 0x2acf5c013, 0xc00ca1e7a8, 0x5b32b1, 0xc0010469c0) /usr/lib/golang/src/runtime/slice.go:197 +0x219 fp=0xc00ca1e748 sp=0xc00ca1e6e0 pc=0x43f9c9 go/printer.(*printer).writeString(0xc00ca217e8, 0x0, 0x0, 0xd77a74b, 0x206f9, 0x2, 0xc0010469c0, 0x30, 0x20601) /usr/lib/golang/src/go/printer/printer.go:310 +0x356 fp=0xc00ca1e7b8 sp=0xc00ca1e748 pc=0x5b2d06 go/printer.(*printer).writeComment(0xc00ca217e8, 0xc00639ba60) /usr/lib/golang/src/go/printer/printer.go:652 +0x47a fp=0xc00ca1e890 sp=0xc00ca1e7b8 pc=0x5b3f6a go/printer.(*printer).intersperseComments(0xc00ca217e8, 0x0, 0x0, 0x40000c51, 0x7ac9f, 0x1a72, 0x38, 0x1a72) /usr/lib/golang/src/go/printer/printer.go:748 +0x11b fp=0xc00ca1e988 sp=0xc00ca1e890 pc=0x5b42bb go/printer.(*printer).flush(0xc00ca217e8, 0x0, 0x0, 0x40000c51, 0x7ac9f, 0x1a72, 0x38, 0x40000c51) /usr/lib/golang/src/go/printer/printer.go:1022 +0xb7 fp=0xc00ca1ea00 sp=0xc00ca1e988 pc=0x5b50a7 go/printer.(*printer).print(0xc00ca217e8, 0xc00ca1eb30, 0x2, 0x2) /usr/lib/golang/src/go/printer/printer.go:982 +0x274 fp=0xc00ca1eb00 sp=0xc00ca1ea00 pc=0x5b49c4 go/printer.(*printer).block(0xc00ca217e8, 0xc11e7c37d0, 0x1) /usr/lib/golang/src/go/printer/nodes.go:1052 +0x1c1 fp=0xc00ca1eb80 sp=0xc00ca1eb00 pc=0x5ac4b1 go/printer.(*printer).stmt(0xc00ca217e8, 0x882340, 0xc11e7c1700, 0x1) /usr/lib/golang/src/go/printer/nodes.go:1260 +0x22f3 fp=0xc00ca1f158 sp=0xc00ca1eb80 pc=0x5aefa3 go/printer.(*printer).stmtList(0xc00ca217e8, 0xc11e791c00, 0x6, 0x8, 0x1, 0xb03201) /usr/lib/golang/src/go/printer/nodes.go:1027 +0x170 fp=0xc00ca1f1e0 sp=0xc00ca1f158 pc=0x5ac140 go/printer.(*printer).block(0xc00ca217e8, 0xc11e7c3800, 0x1) /usr/lib/golang/src/go/printer/nodes.go:1050 +0xed fp=0xc00ca1f260 sp=0xc00ca1f1e0 pc=0x5ac3dd go/printer.(*printer).stmt(0xc00ca217e8, 0x882180, 0xc11e7c1740, 0x0) /usr/lib/golang/src/go/printer/nodes.go:1326 +0x60f fp=0xc00ca1f838 sp=0xc00ca1f260 pc=0x5ad2bf go/printer.(*printer).stmtList(0xc00ca217e8, 0xc11e7c1800, 0x3, 0x4, 0x1, 0xb03201) /usr/lib/golang/src/go/printer/nodes.go:1027 +0x170 fp=0xc00ca1f8c0 sp=0xc00ca1f838 pc=0x5ac140 go/printer.(*printer).block(0xc00ca217e8, 0xc11e7c3830, 0x1) /usr/lib/golang/src/go/printer/nodes.go:1050 +0xed fp=0xc00ca1f940 sp=0xc00ca1f8c0 pc=0x5ac3dd go/printer.(*printer).stmt(0xc00ca217e8, 0x882340, 0xc11e7cc380, 0x100) /usr/lib/golang/src/go/printer/nodes.go:1260 +0x22f3 fp=0xc00ca1ff18 sp=0xc00ca1f940 pc=0x5aefa3 go/printer.(*printer).stmtList(0xc00ca217e8, 0xc11e7c84f0, 0x1, 0x1, 0x1, 0x0) /usr/lib/golang/src/go/printer/nodes.go:1027 +0x170 fp=0xc00ca1ffa0 sp=0xc00ca1ff18 pc=0x5ac140 go/printer.(*printer).stmt(0xc00ca217e8, 0x881f40, 0xc11e7cc3c0, 0x7fb2c0cc0100) /usr/lib/golang/src/go/printer/nodes.go:1284 +0x20e8 fp=0xc00ca20578 sp=0xc00ca1ffa0 pc=0x5aed98 go/printer.(*printer).stmtList(0xc00ca217e8, 0xc11e791e00, 0x6, 0x8, 0x0, 0xb03201) /usr/lib/golang/src/go/printer/nodes.go:1027 +0x170 fp=0xc00ca20600 sp=0xc00ca20578 pc=0x5ac140 go/printer.(*printer).block(0xc00ca217e8, 0xc11e7ca9c0, 0x0) /usr/lib/golang/src/go/printer/nodes.go:1050 +0xed fp=0xc00ca20680 sp=0xc00ca20600 pc=0x5ac3dd go/printer.(*printer).stmt(0xc00ca217e8, 0x882780, 0xc11e7ca9f0, 0x101) /usr/lib/golang/src/go/printer/nodes.go:1289 +0xdce fp=0xc00ca20c58 sp=0xc00ca20680 pc=0x5ada7e go/printer.(*printer).stmtList(0xc00ca217e8, 0xc11e791880, 0x8, 0x8, 0x1, 0xb03201) /usr/lib/golang/src/go/printer/nodes.go:1027 +0x170 fp=0xc00ca20ce0 sp=0xc00ca20c58 pc=0x5ac140 go/printer.(*printer).block(0xc00ca217e8, 0xc11e7caa20, 0x1) /usr/lib/golang/src/go/printer/nodes.go:1050 +0xed fp=0xc00ca20d60 sp=0xc00ca20ce0 pc=0x5ac3dd go/printer.(*printer).stmt(0xc00ca217e8, 0x882180, 0xc11e7cc680, 0x0) /usr/lib/golang/src/go/printer/nodes.go:1326 +0x60f fp=0xc00ca21338 sp=0xc00ca20d60 pc=0x5ad2bf go/printer.(*printer).stmtList(0xc00ca217e8, 0xc11e791f00, 0x5, 0x8, 0x1, 0xc00ca21401) /usr/lib/golang/src/go/printer/nodes.go:1027 +0x170 fp=0xc00ca213c0 sp=0xc00ca21338 pc=0x5ac140 go/printer.(*printer).block(0xc00ca217e8, 0xc11e7cab10, 0x1) /usr/lib/golang/src/go/printer/nodes.go:1050 +0xed fp=0xc00ca21440 sp=0xc00ca213c0 pc=0x5ac3dd go/printer.(*printer).funcBody(0xc00ca217e8, 0x3e, 0xb, 0xc11e7cab10) /usr/lib/golang/src/go/printer/nodes.go:1687 +0x4b4 fp=0xc00ca21558 sp=0xc00ca21440 pc=0x5b13d4 go/printer.(*printer).funcDecl(0xc00ca217e8, 0xc11e7cab40) /usr/lib/golang/src/go/printer/nodes.go:1711 +0x193 fp=0xc00ca215c8 sp=0xc00ca21558 pc=0x5b16d3 go/printer.(*printer).decl(0xc00ca217e8, 0x8821c0, 0xc11e7cab40) /usr/lib/golang/src/go/printer/nodes.go:1721 +0x160 fp=0xc00ca21618 sp=0xc00ca215c8 pc=0x5b18c0 go/printer.(*printer).declList(0xc00ca217e8, 0xc03d1e4000, 0x6a99, 0x7800) /usr/lib/golang/src/go/printer/nodes.go:1764 +0x12c fp=0xc00ca21690 sp=0xc00ca21618 pc=0x5b1a8c go/printer.(*printer).file(0xc00ca217e8, 0xc11e8e4600) /usr/lib/golang/src/go/printer/nodes.go:1772 +0x13a fp=0xc00ca21700 sp=0xc00ca21690 pc=0x5b1cba go/printer.(*printer).printNode(0xc0079717e8, 0x787140, 0xc11e8e4600, 0xc11e8c9750, 0x0) /usr/lib/golang/src/go/printer/printer.go:1152 +0x59c fp=0xc00ca217a8 sp=0xc00ca21700 pc=0x5b5a2c go/printer.(*Config).fprint(0xc00ca21a68, 0x87f160, 0xc00010e000, 0xc005143140, 0x787140, 0xc11e8e4600, 0xc11e8e9950, 0xd, 0x1acf) /usr/lib/golang/src/go/printer/printer.go:1293 +0x170 fp=0xc00ca21950 sp=0xc00ca217a8 pc=0x5b64b0 go/printer.(*Config).Fprint(0xc007971a68, 0x87f160, 0xc00010e000, 0xc005143140, 0x787140, 0xc11e8e4600, 0xc11e8e4600, 0x0) /usr/lib/golang/src/go/printer/printer.go:1351 +0x72 fp=0xc00ca219a8 sp=0xc00ca21950 pc=0x5b6752 github.com/gogo/protobuf/protoc-gen-gogo/generator.(*Generator).generate(0xc00010e000, 0xc004a79b80) /data/home/user00/github.com/gogo/protobuf/protoc-gen-gogo/generator/generator.go:1277 +0xb97 fp=0xc00ca21d58 sp=0xc00ca219a8 pc=0x60c617 github.com/gogo/protobuf/protoc-gen-gogo/generator.(*Generator).GenerateAllFiles(0xc00010e000) /data/home/user00/github.com/gogo/protobuf/protoc-gen-gogo/generator/generator.go:1160 +0x27e fp=0xc00ca21ea8 sp=0xc00ca21d58 pc=0x60b47e github.com/gogo/protobuf/vanity/command.Generate(0xc00009e420, 0x5) /data/home/user00/github.com/gogo/protobuf/vanity/command/command.go:131 +0x17f fp=0xc00ca21f30 sp=0xc00ca21ea8 pc=0x73c99f main.main() /data/home/user00/github.com/gogo/protobuf/protoc-gen-gogofaster/main.go:50 +0x1e4 fp=0xc00ca21f98 sp=0xc00ca21f30 pc=0x73d194 runtime.main() /usr/lib/golang/src/runtime/proc.go:201 +0x207 fp=0xc00ca21fe0 sp=0xc00ca21f98 pc=0x42bb07 runtime.goexit() /usr/lib/golang/src/runtime/asm_amd64.s:1333 +0x1 fp=0xc00ca21fe8 sp=0xc00ca21fe0 pc=0x455cf1 --gogofaster_out: protoc-gen-gogofaster: Plugin failed with status code 2.

    opened by niupanyu 0
  • string time and duration

    string time and duration

    I would like to say this:

    message X {
       optional string foo = 1 [default = "2m", (gogoproto.strduration) = true];
    }
    

    in text proto, and binary representation it would be a string, but a struct in Go will be a time.Duration that properly calls time.ParseDuration / dur.String().

    Similar for time.Time (with possibility to specify the format to use).

    Yes, this is less efficient than WKT Duration and Timestamp, for transmit over network or storage, but if using protocol buffers for configuration maintained by humans, it is way easier. Also there is no way to provide default values for WKT Duration and Timestamp.

    opened by baryluk 0
  • Improper Input Validation in GoGo Protobuf

    Improper Input Validation in GoGo Protobuf

    Hi team @gpaul @gburt @gaffneyc @AlekSi @temoto thank you for developing awesome Protocol Buffers for Go with Gadgets

    I am using github.com/gogo/[email protected] for my project. I am seeing security waring as following by github Dependabot alerts

    An issue was discovered in GoGo Protobuf before 1.3.2. plugin/unmarshal/unmarshal.go lacks certain index validation, aka the "skippy peanut butter" issue.

    can you guide me how to work on this security issue so that I can fix and create pull request ?

    Thank you

    opened by pratikdhanavesearce 1
Releases(v1.3.2)
  • v1.3.2(Jan 10, 2021)

  • v1.3.1(Oct 14, 2019)

    Tested versions:

    go 1.12.10 protoc 3.9.1

    Bug fixes:

    • proto/buffer: fix proto.Buffer marshaling.
      • Thanks: https://github.com/apelisse
    • plugin/gostring: generate values instead of pointers when a field is repeated and non-nullable.
      • Thanks https://github.com/godfried
    • protoc-gen-gogo/generator: Generate json and custom tags for oneof
      • Thanks: https://github.com/krhubert
    • plugin/marshalto: Use ProtoSize() in MarshalTo when enabled for oneof fields.
      • Thanks: https://github.com/gaffneyc

    Upstream commits:

    • 4c88cc3f1a34ffade77b79abc53335d1e511f25b - all: fix reflect.Value.Interface races.
    • 6c65a5562fc06764971b7c5d05c76c75e84bdbf7 - jsonpb: fix marshaling of Duration
    • b285ee9cfc6c881bb20c0d8dc73370ea9b9ec90f - Log parsing errors using log pkg

    Misc:

    • add github workflow config
    • protoc update - Updated to protoc 3.9.1
    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(Sep 1, 2019)

    Tested versions:

    go 1.12.9 protoc 3.7.1

    Improvements:

    • plugin/stringer - Handle repeated and/or nullable types a bit better now.
    • plugin/size - Remove the loop in sovXXX by using bit twiddling.
      • Thanks: https://github.com/apelisse
    • plugin/marshalto - Implemented a reverse marshal strategy which allows for faster marshalling. This now avoids a recursive (and repeated) call to Size().
      • Thanks: https://github.com/apelisse
    • plugin/compare - Added support for for oneof types.

    Bug fixes:

    • protoc-gen-gogo/generator - Fix assignment to entry in nil map.
      • Thanks: https://github.com/tgulacsi
    • protoc-gen-gogo/generator - Allows plugins to call RecordTypeUse without panicking.
      • Thanks: https://github.com/fedenusy
    • proto/extensions - Fixed set extension regression. We did not clear the extensions before setting.
    • io/uint32 - fix uint32reader bug that causes ReadMsg to recreate buffer when lengths are the same.
      • Thanks: https://github.com/SebiSujar
    • proto/table_merge: Fix merge of non-nullable slices.
      • Thanks: https://github.com/euroelessar

    Upstream commits:

    • merged in golang/protobuf commit 318d17de72747ed1c16502681db4b2bb709a92d0 - Add UnimplementedServer for server interface
    • merged in golang/protobuf commit b85cd75de734650db18a99a943fe351d41387800 - protoc-gen-go/grpc: inline errUnimplemented function
    • merged in golang/protobuf commit d3c38a4eb4970272b87a425ae00ccc4548e2f9bb - protoc-gen-go/grpc: use status and code packages only if needed
    • merged in golang/protobuf commit e91709a02e0e8ff8b86b7aa913fdc9ae9498e825 - fix indentation in jsonpb with Any messages
    • merged in golang/protobuf commit 8d0c54c1246661d9a51ca0ba455d22116d485eaa - protoc-gen-go: generate XXX_OneofWrappers instead of XXX_OneofFuncs

    Misc:

    • extensions.md - Markdown update.
      • Thanks: https://github.com/TennyZhuang
    • Readme.md - Added user.
    • go/protoc update - Updated to go1.12.x and protoc 3.7.1
    • Makefile update - fix go vet shadow tool reference
    • test/mixbench - Update mixbench tool. Expose runnable benchmarks via flags.
    Source code(tar.gz)
    Source code(zip)
  • v1.2.1(Feb 18, 2019)

  • v1.2.0(Dec 11, 2018)

  • v1.1.0(Jul 15, 2018)

  • v1.0.0(Feb 2, 2018)

    This is the first tag of golang/protobuf and gogo/protobuf is syncing with this process. This release also includes:

    • minor fixes
    • compatibility with go 1.10 beta
    • compatibility with protoc 3.5.1
    Source code(tar.gz)
    Source code(zip)
  • v0.5(Oct 7, 2017)

    Code generation includes no more unsafe code. See https://github.com/gogo/protobuf/pull/343 for details. Well known types have been added, so proto3 supports seems to be full. Compatibility up to go 1.8.3 and 1.9 is tested. Protoc compatibility has been tested up to version protoc version 3.4.

    Source code(tar.gz)
    Source code(zip)
  • v0.4(Mar 9, 2017)

    This is quite a stable point for gogoprotobuf. proto 3 support is limited, since it does not include support for well known types. Compatibility with go 1.7 up to 1.8 is tested.

    Source code(tar.gz)
    Source code(zip)
  • v0.3(Aug 24, 2016)

    This is quite a stable point for gogoprotobuf. proto 3 support is limited, since it does not include support for well known types. Compatibility with go 1.4 up to 1.7 is tested.

    Source code(tar.gz)
    Source code(zip)
  • v0.2(Mar 24, 2016)

    This is quite a stable point for gogoprotobuf. proto 3 support is limited, since it does not include support for well known types. Compatibility with go 1.3.3 up to 1.6 is tested.

    Source code(tar.gz)
    Source code(zip)
  • v0.1(Aug 31, 2015)

    This is quite a stable point for gogoprotobuf including many gadgets. proto 2.6.1 and 3 support are limited. Compatibility with go 1.2 up to 1.5 is tested.

    Source code(tar.gz)
    Source code(zip)
Owner
gogoprotobuf
Protocol Buffers for Go with Gadgets
gogoprotobuf
Encode and decode Go (golang) struct types via protocol buffers.

protostructure protostructure is a Go library for encoding and decoding a struct type over the wire. This library is useful when you want to send arbi

Mitchell Hashimoto 172 Sep 9, 2022
Create a Protocol Buffers (Protobuf) plugin, which is executed with the protoc compileCreate a Protocol Buffers (Protobuf) plugin, which is executed with the protoc compile

Interview Assignment Overview You assignment is to create a Protocol Buffers (Protobuf) plugin, which is executed with the protoc compiler. In this ex

Patrick Valle 0 Nov 19, 2021
A scanner/exploitation tool written in GO, which leverages Prototype Pollution to XSS by exploiting known gadgets.

ppmap A simple scanner/exploitation tool written in GO which automatically exploits known and existing gadgets (checks for specific variables in the g

kleiton0x00 343 Sep 13, 2022
Go support for Google's protocol buffers

Go support for Protocol Buffers This module (github.com/golang/protobuf) contains Go bindings for protocol buffers. It has been superseded by the goog

Go 8.7k Sep 23, 2022
A proxy server than converts JSON request bodies to protocol buffers

Welcome to Protoxy ?? What is Protoxy? Protoxy allows you to test your REST APIs that use Protocol Buffer serialization through Postman and other API

null 25 Sep 9, 2022
A new way of working with Protocol Buffers.

Buf All documentation is hosted at https://buf.build. Please head over there for more details. Goal Buf’s long-term goal is to enable schema-driven de

null 5.4k Sep 26, 2022
Encode and decode Go (golang) struct types via protocol buffers.

protostructure protostructure is a Go library for encoding and decoding a struct type over the wire. This library is useful when you want to send arbi

Mitchell Hashimoto 172 Sep 9, 2022
A Protocol Buffers compiler that generates optimized marshaling & unmarshaling Go code for ProtoBuf APIv2

vtprotobuf, the Vitess Protocol Buffers compiler This repository provides the protoc-gen-go-vtproto plug-in for protoc, which is used by Vitess to gen

PlanetScale 508 Sep 25, 2022
🔄 A command-line utility to export Protocol Buffers (proto) files to YAML, and JSON

proto2yaml ?? A command-line utility to export Protocol Buffers (proto) files to YAML, and JSON. Currently supported exports are for: Packages Service

Kristof Kowalski 8 Jul 25, 2022
Protocol Buffers - Google's data interchange format

Protocol Buffers - Google's data interchange format Copyright 2008 Google Inc. https://developers.google.com/protocol-buffers/ Overview Protocol Buffe

Protocol Buffers 56.4k Sep 27, 2022
Package pbpgx provides a toolkit for easier Protocol Buffers interaction with PostgreSQL databases.

PBPGX Package pbpgx provides a toolkit for easier Protocol Buffers interaction with PostgreSQL databases. Pbpgx supports the Protocol Buffer types gen

Tim Möhlmann 11 Jun 27, 2022
A plugin of protoc that for using a service of Protocol Buffers as http.Handler definition

protoc-gen-gohttp protoc-gen-gohttp is a plugin of protoc that for using a service of Protocol Buffers as http.Handler definition. The generated inter

John_Suu 1 Dec 9, 2021
Go support for Protocol Buffers - Google's data interchange format

Go support for Protocol Buffers - Google's data interchange format Google's data interchange format. Copyright 2010 The Go Authors. https://github.com

Tinker Board - Android 0 Dec 15, 2021
Estudos com Golang, GRPC e Protocol Buffers

Golang, GRPC e Protocol Buffers Estudos com Golang, GRPC e Protocol Buffers Projeto feito para fins de estudos. Para rodar basta seguir os passos abai

Marcos Daniel 1 Feb 10, 2022
This is a golang C2 + Implant that communicates via Protocol Buffers (aka. protobufs).

Br4vo6ix DISCLAIMER: This tool is for educational, competition, and training purposes only. I am in no way responsible for any abuse of this tool This

RITSEC Redteam 5 May 15, 2022
Protocol Buffers to HTTP client code generator/converter

Proto2http Proto2http provides a code generation tool to convert your protocol buffers (.proto) files into invokable HTTP request. Usage proto2http -p

Kodiiing 6 Aug 18, 2022
wire protocol for multiplexing connections or streams into a single connection, based on a subset of the SSH Connection Protocol

qmux qmux is a wire protocol for multiplexing connections or streams into a single connection. It is based on the SSH Connection Protocol, which is th

Jeff Lindsay 191 Sep 25, 2022
Eunomia is a distributed application framework that support Gossip protocol, QuorumNWR algorithm, PBFT algorithm, PoW algorithm, and ZAB protocol and so on.

Introduction Eunomia is a distributed application framework that facilitates developers to quickly develop distributed applications and supports distr

Cong 2 Sep 28, 2021
A simple tool to convert socket5 proxy protocol to http proxy protocol

Socket5 to HTTP 这是一个超简单的 Socket5 代理转换成 HTTP 代理的小工具。 如何安装? Golang 用户 # Required Go 1.17+ go install github.com/mritd/[email protected] Docker 用户 docker pull m

mritd 7 Sep 7, 2022
A project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol formatA project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol format

Intro This project outputs Bluetooth Low Energy (BLE) sensors data in InfluxDB line protocol format. It integrates nicely with the Telegraf execd inpu

Marc Venturini 1 Apr 15, 2022