idiomatic codec and rpc lib for msgpack, cbor, json, etc. msgpack.org[Go]

Related tags

Serialization go
Overview

Sourcegraph Build Status codecov PkgGoDev rcard License

go-codec

This repository contains the go-codec library, the codecgen tool and benchmarks for comparing against other libraries.

This is a High Performance, Feature-Rich Idiomatic Go 1.4+ codec/encoding library for binary and text formats: binc, msgpack, cbor, json and simple.

It fully supports the legacy GOPATH and the new go modules modes.

Code Organization and Module Support

This repository consists of 4 modules:

  • github.com/ugorji/go (requires github.com/ugorji/go/codec)
  • github.com/ugorji/go/codec (requires github.com/ugorji/go) README
  • github.com/ugorji/go/codec/codecgen (requires github.com/ugorji/go/codec) README
  • github.com/ugorji/go/codec/bench (requires github.com/ugorji/go/codec) README

For encoding and decoding, the github.com/ugorji/go/codec module is sufficient.

To install:

go get github.com/ugorji/go/codec

The other modules exist for specific uses, and all require github.com/ugorji/go/codec

Issues
  • Fix module issues once and for all

    Fix module issues once and for all

    The use of modules in go-codec introduced more headaches than necessary.

    It has been a disaster, and I really wish I had waited until the dust settled before making this module aware, especially since so many people depended on this package and it has the atypical "package name differs from directory name" issue.

    I regret jumping on modules that early, and having to constantly try to correct my mistakes.

    It seems the dust has settled now and we have the tools to:

    • support the simulation of a multi-module repository

    We have 2 options:

    1. Make go/codec its own module using a tag: codec/vX.Y.Z
    2. Keep go as the only module, using a tag: vX.Y.Z
    3. Have both as modules using "same'ish" parallel tags (vX.Y.Z and codec/vX.Y.Z) and corresponding go.mod files

    I am not sure if 3. above is supported by go modules. I read that a module doesn't include modules within it.

    I will appreciate any thoughts on how best to resolve this. I am including references to places where I have seen folks have issues. I hope that you folks can respond and help me settle this once and for all with the best solution.

    https://github.com/golang/go/wiki/Modules#publishing-a-release https://golang.org/wiki/Modules#faqs--multi-module-repositories

    https://github.com/gin-gonic/gin/issues/1897

    @dmitshur @arizvisa @bvisness @HZ89 @bcmills @amarox @thepudds

    Urgent 
    opened by ugorji 32
  • Elements ignored when encoding struct embedding a Selfer

    Elements ignored when encoding struct embedding a Selfer

    Here's a simplified repro:

    package main
    
    import (
            "bytes"
            "fmt"
    
            "github.com/ugorji/go/codec"
    )
    
    // We have defined Selfers for a type (either with codecgen or manually)
    type Embedded struct {
            Embedded1 string
            Embedded2 string
    }
    
    func (e Embedded) CodecEncodeSelf(encoder *codec.Encoder) {
            // phon4y output, in a real example this would encode the struct properly
            encoder.Encode("['phony']")
    }
    
    func (e *Embedded) CodecDecodeSelf(decoder *codec.Decoder) {
    }
    
    // We use embedded in a different structure
    type Foo struct {
            Bar string
            Embedded
    }
    
    func main() {
            foo := Foo{
                    Bar: "bar",
                    Embedded: Embedded{
                            Embedded1: "embedded1",
                            Embedded2: "embedded2",
                    },
            }
    
            // The Decoder only encodes the Embedded struct
            // and ignores Bar!
            buffer := &bytes.Buffer{}
            codec.NewEncoder(buffer, &codec.JsonHandle{}).Encode(foo)
    
             fmt.Printf("Output: %s\n", buffer.String())
    }
    

    The program above outputs:

    Output: "['phony']"
    

    completely ignoring the Bar element.

    I would expect something in the lines of:

    Output: { Bar: "Bar", Embedded: "phony" }
    

    Or, with a properly defined selfer:

    Output: { Bar: "Bar", Embedded1: "embedded1", Embedded2: "embedded2"  }
    

    I guess that, since the embedded Selfer makes the full struct a Selfer too, the encoder is blindly applying (e Embedded) CodecEncodeSelf(), ignoring other fields. I think the Encoder must somehow distinguish between the case in which the struct embeds a Selfer and when the struct itself implements a Selfer directly.

    I haven't tried it yet it but, as a workaround, I guess one can define a Selfer for Foo.

    I haven't tested the decoding, but maybe that's broken too.

    WorkAsDesigned 
    opened by 2opremio 29
  • Change import path to github.com/ugorji/go-codec

    Change import path to github.com/ugorji/go-codec

    It would be nice if the import path mapped directly to a URL to the source code.

    Currently, it doesn't, and for no good reason. Initially, we used a repository name of go, with the hope of putting multiple packages into the same repository. However, in retrospect, that is not a good idea; one repository should only hold one package, with possible sub-packages which relate directly to that package.

    The plan of action is:

    • Use github to change repository name from go to go-codec
    • Move all files from the codec sub-directory of the repository to the top-level directory
    • Keep the codec subdirectory there, and create a codec/fail.go file

    The new codec/fail.go file will contain

    • package docs that tell users the new import path and where to find docs
    • init method that panics with an appropriate error message
    • possibly use the // import label to allow go get fail early

    We hope to do this soon, likely by the end of january.

    enhancement wontfix BreakingChange 
    opened by ugorji 29
  • Kubernetes: Failure when codecgen is run on anonymous type, but not on wrapping type.

    Kubernetes: Failure when codecgen is run on anonymous type, but not on wrapping type.

    To reproduce I'm using: https://github.com/kubernetes/kubernetes/compare/master...wojtek-t:use_ugorij#diff-22207499f9ab0d60831b9c388397f034R1

    Input data: https://github.com/kubernetes/kubernetes/compare/master...wojtek-t:use_ugorij#diff-29f2371dad30835b344a6ca05b473eaeR1

    I did NOT run codecgen for the file that contains definition of that jsons: https://github.com/kubernetes/kubernetes/blob/master/plugin/pkg/scheduler/api/types.go

    IIUC, this should result in falling back to encoding/json (which works fine). However this is not the case (i.e. the output is empty)

    @ugorji

    WorkAsDesigned 
    opened by wojtek-t 19
  • Please tag releases

    Please tag releases

    Formal releases are useful to define dependency relationships. 3rd party developers would be able to bundle somewhat "stable" version of your software instead of just top of "master". Also it is helpful for downstream maintainers for numerous reasons.

    Please consider tagging. Thanks.

    enhancement 
    opened by onlyjob 17
  • Slower than encoding/json

    Slower than encoding/json

    Here's my test:

    n := 5
    
    log.Print("encoding/json")
    for i := 0; i < n; i++ {
      t := time.Now()
      var v interface{}
      for j := 0; j < 20; j++ {
        if err = json.Unmarshal(b, &v); err != nil {
          panic(err)
        }
      }
      log.Print(time.Now().Sub(t))
    }
    
    log.Print("go-codec")
    for i := 0; i < n; i++ {
      t := time.Now()
      for j := 0; j < 20; j++ {
        var dec *codec.Decoder = codec.NewDecoderBytes(b, new(codec.JsonHandle))
        var v interface{}
        if err := dec.Decode(&v); err != nil {
          panic(err)
        }
      }
      log.Print(time.Now().Sub(t))
    }
    

    For a large, minified file with long keys (4.7MB), go-codec is ~ 31% slower than encoding/json:

    encoding/json
    1.617601942s
    1.576523967s
    1.610622356s
    1.608594081s
    1.647141938s
    go-codec
    2.129585437s
    2.166288614s
    2.1520013s
    2.061147463s
    2.102718168s
    

    For a file containing the same values, but with every key shortened to just 2 characters (970KB), it's 80% slower:

    encoding/json
    474.933239ms
    451.410297ms
    452.822768ms
    462.508272ms
    456.032676ms
    go-codec
    852.508337ms
    849.04073ms
    830.897711ms
    852.40816ms
    826.848954ms
    

    This is from byte arrays into interface{} values. Is there anything I can tweak to make go-codec be faster? Is this by design, somehow?

    Go 1.5.1 on OS X.

    opened by atombender 17
  • "reflect.Value.Addr of unaddressable value" error while marshaling

    When I was trying to use Ugorij for Kubernetes: https://github.com/kubernetes/kubernetes/blob/master/pkg/api/types.go I received the following error: "reflect.Value.Addr of unaddressable value"

    How to reproduce it:

    • I used the following code: https://github.com/kubernetes/kubernetes/compare/master...wojtek-t:use_ugorij#diff-a880abe356116a717967d2246a1ffc36R29

    • This diff also contains a file that was generated for our structures.

      Please let me know if you need more information on reproducing it

    UserError 
    opened by wojtek-t 16
  • bug: MapValueReset has no effect

    bug: MapValueReset has no effect

    The sample code below doesn't seem to work as it should:

    package main
    
    import (
    	"bytes"
    	"fmt"
    
    	"github.com/ugorji/go/codec"
    )
    
    func main() {
    
    	h := &codec.JsonHandle{}
    	h.MapValueReset = true
    
    	d := bytes.NewBuffer([]byte(`{"map": {"a": 1}}`))
    	dec := codec.NewDecoder(d, h)
    
    	o := struct {
    		Map map[string]interface{} `json:"map"`
    	}{}
    
    	o.Map = map[string]interface{}{"b": 1}
    
    	if err := dec.Decode(&o); err != nil {
    		panic(err)
    	}
    
    	fmt.Println(o.Map)
    }
    

    This prints:

    map[a:1 b:1]
    

    Since I set MapValueReset = true, I would expect it to print:

    map[a:1]
    
    question WorkAsDesigned 
    opened by primalmotion 15
  • mod: justify mod style

    mod: justify mod style

    In this way, we can get the right tag like v1.1.3, instead of pseudo-version

    A good example would be shown below

    https://github.com/stretchr/testify/blob/master/go.mod https://github.com/andrewstuart/go-robinhood/blob/master/go.mod#L8

    go-robinhood imports github.com/stretchr/testify/assert and got the right version

    github.com/stretchr/testify v1.2.2
    
    opened by hnlq715 14
  • register extension without custom en/decoding

    register extension without custom en/decoding

    As far as I've managed to dig into the docs, registering an extension (cbor tag) requires you to also manually en/decode the data yourself. Both the InterfaceExt and Ext interfaces assume you implement the en/decoding.

    I was wondering if there is a way to register a struct as an extension, and let the normal un/marshalling of the struct to take place for its payload.

    eg.

    type Something struct {
    	Foo string `json:"foo"`
    }
    

    I'd like for example to be able to register Foo as extension 123 and have the handler marshal the tag, followed by a map with a string element with key foo.

    Is there a way to do something simple like this in the current state of the lib that I am missing? If not, could point me to the right direction of what could be done to add it?

    Thank you for an amazing library.

    enhancement 
    opened by geoah 14
  • Kubernetes: codecgen: produce consistent output on repeated runs

    Kubernetes: codecgen: produce consistent output on repeated runs

    This is usability issue (not correctness one). I observed that when I'm using codecgen, the output produced for a given file (I was using: https://github.com/kubernetes/kubernetes/blob/master/pkg/api/types.go is non deterministic. Few things that change:

    1. constants "codecSelferC*" has different names, e.g.:
    codecSelferC_UTF85477         = 1
    

    vs

     codecSelferC_UTF85165         = 1
    
    1. ordering of generated function (and thus the names of variables they are using, because IIUC, you're using consecutive integers for variable names)

    It would be very convenient if the output would be deterministic.

    opened by wojtek-t 13
  • Incorrect codecgen code is generated

    Incorrect codecgen code is generated

    Hi, I attached a sample project in order to show the issue.

    The problem appears when a package tries to use code generated in other package.

    To reproduce the issue follow this steps:

    1. Run go mod tidy and go mod vendor to download the dependencies. (vendoring is not required but used here to encapsulate the test)
    2. Run codecgen -nx -o test2_codec.go test2.go in the test2 subdirectory.
    3. Run codecgen -nx -o test_codec.go test.go in the "root" directory.
    4. Run go test

    You will get something like this:

    # test.com/test
    .\test_codec.go:77:11: yysf5.CodecEncodeSelf undefined (type **test2.A has no field or method CodecEncodeSelf)
    .\test_codec.go:101:11: yysf7.CodecEncodeSelf undefined (type **test2.A has no field or method CodecEncodeSelf)
    .\test_codec.go:227:12: yysf4.CodecEncodeSelf undefined (type **test2.A has no field or method CodecEncodeSelf)
    .\test_codec.go:254:12: yysf5.CodecEncodeSelf undefined (type **test2.A has no field or method CodecEncodeSelf)
    FAIL    test.com/test [build failed]
    

    I was able to patch it by replacing this line:

    if ti2.flagSelfer {
    

    with:

    if ti2.flagSelfer || (isptr && ti2.flagSelferPtr) {
    

    but not sure if the right thing to do.

    Kind regards, Mauro.

    opened by mxmauro 1
  • Skip type aliases

    Skip type aliases

    Hi, when you define type A = B, A becomes an alias of B.

    Instead, using type A B, B is an underlying object of A.

    For the former, the generator incorrectly generates duplicated selfer methods causing the go code to be invalid.

    Regards, Mauro

    opened by mxmauro 0
  • XML support

    XML support

    go-codec currently supports

    • Binary: Msgpack
    • Binary: CBOR
    • Text: JSON

    For completeness, it should support XML also.

    Though XML has fallen out-of-favor, it is still widely used. The encoding/xml in the standard library is not efficient, and does not conform to the simplicity used in encoding/json. We have a chance to add a performant XML encoder/decoder as just another format.

    We are attempting this due to perceived issues with encoding/xml:

    • Complicated. It tried to do too much, and is not as simple to use as json.
    • Due to over-engineering, reflection is over-used AND performance suffers: java is 6X faster:http://fabsk.eu/blog/category/informatique/dev/golang/ even PYTHON performs better: http://outgoing.typepad.com/outgoing/2014/07/exploring-golang.html

    codec framework will offer the following benefits

    • VASTLY improved performance (when using reflection-mode or codecgen)
    • simplicity and consistency: with the rest of the supported formats
    • all other benefits of codec framework (streaming, codegeneration, etc)

    go-codec is not a drop-in replacement for encoding/xml, as the tags and style will not transfer over. Having said that, it is a replacement, based on the simplicity and performance of codec.

    The goal is for this to be like JAXB for Go.

    enhancement 
    opened by ugorji 1
Releases(v1.2.7)
  • v1.2.7(Feb 26, 2022)

    This is a production release of go-codec.

    Changes include:

    • support registering extension functions for time.Time
    • fix bugs with encoding/decoding very-nested slices in specific cases

    Please try it out and share your experiences.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.6(May 15, 2021)

    This is a production release of go-codec.

    Changes include:

    • support EncodeOptions.NoAddressableReadonly option
    • streamline/reduce size of helper types
    • optimize side(En|De)code and (En|De)codeExt calls
    • decode a number from any number in the stream i.e. you can decode a float/integer/unsigned integer from any number in stream
    • clean up tests so they work completely regardless if we recover from panics
    • improve inlining of common functions

    Please try it out and share your experiences.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.5(Mar 30, 2021)

    This is a production release of go-codec.

    This is a very important release, building upon the optimization introduced in v1.2.4.

    Changes include:

    • support gollvm in addition to gccgo in default high-performance mode
    • support codec.safe and codec.notfastpath as preferred build tags (replacing safe and notfastpath)
    • support Canonical mode with MissingFielder
    • robust handling of codecgen caveats wrt Canonical|CheckCircularRef flags and MissingFielder implementations
    • robust handling of transient values, where we track if a value has internal pointers and handle appropriately
    • reduce use of global values: all in-use values are scoped to an Encoder or Decoder
    • implement stateManager for Encoder/Decoder, supporting capture, reset and restore functions.
    • side encoding or decoding can now reuse an Encoder/Decoder, leveraging state management to capture its state, do something else, then restore its state and continue
    • support running all tests in parallel, so we can shake out any concurrency issues
    • Fix issue in rpc where writer was not being flushed, causing a hang where reader was waiting on an encoded value
    • Support shallow copies of a Handle, to support parallel execution where we need to temporarily change some Handler fields

    This is the best release yet. Please try it out and share your experiences.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.4(Feb 4, 2021)

    Release 1.2.4

    This is a production release of go-codec.

    This is a very important release, which focused on optimization across the board to ensure that go-codec is the most performant library for popular encoding formats.

    Optimizations across the board include:

    • optimization by eliding allocation where not necessary e.g.
      • when encoding non-addressable value with ptr receiver
      • map iteration/access
      • decoding floats
      • creating slices or sub-slices (removing temporary slice allocated on heap)
      • decoding map keys if both key and value kinds are scalars
      • decoding map values if value kind is a scalar
    • better reuse of pool'ed []byte(s)
    • leveraging mapXXX_fastXXX calls where possible
    • better inlining (where applicable) e.g. arrayStart, mapStart, etc
    • support complex numbers (where we treat complex numbers as floats with imaginary part = 0)
    • optimize encoding and decoding arrays, including using fast-paths for slices as applicable
    • use runtime's growslice algorithm when growing arrays for better performance
    • optimize nextValueBytes and swallow so they do not allocate when applicable e.g. do not allocate/copy bytes if swallow is called or you were decoding from a []byte (just return a sub-slice)
    • optimize isEmptyStruct to just compare if value == zero'es (instead of field by field), simulating comparing to zero value.

    To guarantee better fairness in benchmarking, we did the following:

    • use common types in benchmarks and common methods to ensure fairness in benchmarks
    • run benchmarks with ZeroCopy=true and (MapValue|Interface|SliceElement)Reset = true, aligning with how other libraries operate by default

    With these changes, we have reflection-based mode performing within 25% of codecgen, with similar allocation numbers as codecgen. For example, encoding typically has just 1 allocation without codecgen (just as in codecgen).

    This is the best release yet. Please try it out and share your experiences.

    Source code(tar.gz)
    Source code(zip)
  • v1.2.3(Jan 10, 2021)

    This is a production release of go-codec.

    This optimizes decoding for zero-copy mode when decoding from a string or []byte.

    With this mode, we see 50% reduction in allocation in both codecgen and normal execution, and this gives performance better than other libraries (easyjson, json-iterator, etc) while providing much more features and supported formats (json, cbor, msgpack, simple, etc).

    Source code(tar.gz)
    Source code(zip)
  • v1.2.2(Dec 20, 2020)

  • v1.2.1(Dec 3, 2020)

  • v1.2.0(Nov 11, 2020)

    This is a production release of go-codec.

    Today marks 1 full year after the 11th day of the 11th month of last year, a very very special day in my life.

    On this very special day, we release the latest patch release of v1.1 and it matches with the first release of v1.2.0 of go-codec.

    This release should be about 10% faster during decode than previous releases.

    Beyond performance, the focus was on cleaning up the codebase while maintaining support for much earlier releases of go, and supporting gccgo.

    Changes

    • support gccgo
    • improve performance in looking up fast-path functions or cached struct information
    • optimize json's encoding and decoding of numbers (integers and floats) and strings
    • optimize base reading primitive functions, to reduce pointer-chasing
    • optimize map iteration and lookup

    See benchmark results run on a very complicated struct full of embedded types, with multiple nested levels. Each struct value encodes as 47K of tightly packed json or 37K of binary msgpack or cbor.

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we continue to achieve that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.14(Nov 11, 2020)

    This is a production release of go-codec.

    Today marks 1 full year after the 11th day of the 11th month of last year, a very very special day in my life.

    On this very special day, we release the latest patch release of v1.1 and it matches with the first release of v1.2 of go-codec.

    This release should be about 10% faster during decode than previous releases.

    Beyond performance, the focus was on cleaning up the codebase while maintaining support for much earlier releases of go, and supporting gccgo.

    Changes

    • support gccgo
    • improve performance in looking up fast-path functions or cached struct information
    • optimize json's encoding and decoding of numbers (integers and floats) and strings
    • optimize base reading primitive functions, to reduce pointer-chasing
    • optimize map iteration and lookup

    See benchmark results run on a very complicated struct full of embedded types, with multiple nested levels. Each struct value encodes as 47K of tightly packed json or 37K of binary msgpack or cbor.

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we continue to achieve that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.13(Oct 19, 2020)

    This is a production release of go-codec.

    This release should be about 10% faster during decode than previous releases.

    Changes

    • 10% faster on decode and encoding if omitempty is configured
    • improved performance in looking up struct field values when decoding structs
    • codecgen: support generating reduced code for json only, omitempty support, or to_array support
    • omitempty: support for IsCodecEmpty, a custom way of checking if a value is empty
    • omitempty: optimize comparing to zero'es to check if a value is empty
    • stack overflow/circular references: optimize check by push/popping a stack

    See benchmark results run on a very complicated struct full of embedded types, with multiple nested levels. Each struct value encodes as 47K of tightly packed json or 37K of binary msgpack or cbor.

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we continue to achieve that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.12(Oct 14, 2020)

    UPDATE: v1.1.12 includes fix for a race detector issue #336.

    This is a production release of go-codec.

    Today marks 11 months after the 11th day of the 11th month of last year.

    On this very special day, we release v1.1.11 of go-codec.

    This day holds a special place in my heart on a very personal level.

    I hope you enjoy the immense effort put into refining and cleaning out the edges go-codec, delivered in v1.1.11

    The quality and performance should shine across:

    • binary size
    • performance/speed
    • memory use
    • feature rich (more than any other library in the go ecosystem)

    See benchmark results run on a very complicated struct full of embedded types, with multiple nested levels. Each struct value encodes as 47K of tightly packed json or 37K of binary msgpack or cbor.

    High level benefits include

    • significantly expanded test suite, bringing code coverage to 92%
    • remove limits on struct embedding
    • optimize struct field access, json parsing, free list usage, etc
    • multiple performance improvements across the board

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we continue to achieve that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.11(Oct 11, 2020)

    UPDATE: v1.1.1 introduced a race detector issue #336, which has now been fixed in v1.1.12. Please upgrade.

    This is a production release of go-codec.

    Today marks 11 months after the 11th day of the 11th month of last year.

    On this very special day, we release v1.1.11 of go-codec.

    This day holds a special place in my heart on a very personal level.

    I hope you enjoy the immense effort put into refining and cleaning out the edges go-codec, delivered in v1.1.11

    The quality and performance should shine across:

    • binary size
    • performance/speed
    • memory use
    • feature rich (more than any other library in the go ecosystem)

    See benchmark results run on a very complicated struct full of embedded types, with multiple nested levels. Each struct value encodes as 47K of tightly packed json or 37K of binary msgpack or cbor.

    High level benefits include:

    • significantly expanded test suite, bringing code coverage to 92%
    • remove limits on struct embedding
    • optimize struct field access, json parsing, free list usage, etc
    • multiple performance improvements across the board

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we continue to achieve that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.10(Oct 3, 2020)

    This is a production release of go-codec.

    The main benefits are much expanded testing and associated code fixes, re-factoring and performance improvements:

    • removed some code cruft
    • expand tests and fix issues around extensions support in codecgen
    • clean up codebase for easier maintenance
    • and many general performance improvements ...

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we continue to achieve that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.9(Sep 27, 2020)

    This is a production release of go-codec.

    The main benefits are streamlined go modules support, re-factoring and performance improvements:

    • removed some code cruft
    • moved codec/bench into its own module, so it's requirements do not pollute the core library dependencies
    • implement reading the next value in the stream natively, provided by each format's handle
    • remove mode of reading the next value in the stream by decoding into a throwaway value and tracking bytes read
    • clean up code to enable better inlining for better performance
    • fix and expand tests to get richer code coverage; go test -cover reports > 90%
    • and many general performance improvements ...

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we continue to achieve that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.8(Sep 14, 2020)

    This is a production release of go-codec.

    The main benefits are a number of bug fixes and some performance improvements:

    • reduce binary size by 30% by reducing types for which we generate fast-path functions
    • support OptimumSize encode option: cbor uses to encode floats in smallest size e.g. binary16
    • decode stream nil value in mapping as a nil pointer in memory (previously set to zero value)
    • validate that malformed streams can be decoded without causing stack overflow or out of memory errors
    • json: return EOF in all scenarios when end of stream is reached (previously returned other error sometimes)
    • json: handle malformed unicode correctly in corner cases
    • and some general performance improvements ...

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we have achieved that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.7(Jul 2, 2019)

    This is a production release of go-codec.

    First, it resolves the go.mod impasse where we had different import paths (github.com/ugorji/go and github.com/ugorji/go/codec) causing the ambiguous import error.

    This is now fixed by leveraging import cycles to ensure that either one works well and resolves to the same bits.

    The other benefits are a clean up and refactoring of the codebase:

    • provide better and more consistent performance
    • streamline the use of sync.Pool vs free lists
    • Use more optimized helpers when not in safe mode
    • Smoothen rough edges across the board
    • and many more ...

    The main goal has always been to provide the most feature-rich and performant package for encoding and decoding of multiple popular binary and text formats in a consistent way.

    We believe we have achieved that!

    Source code(tar.gz)
    Source code(zip)
  • v1.1.5-pre(Oct 14, 2020)

  • v1.1.4(Apr 8, 2019)

    This is a production release of go-codec.

    It enforces a stable import path for releases of go which may or may not be aware of modules.

    It fixes the incorrect go.mod in the (broken and now removed) v1.1.3 tag and 1.1.3 release.

    It also includes fixes and optimizations since the last release, including clearer handling of UTF-8 and strings (readonly raw bytes).

    Source code(tar.gz)
    Source code(zip)
  • v1.1.2(Jan 26, 2019)

    This is a production release of go-codec.

    It includes fixes and optimizations over the last 9 months, including:

    • better use of resource pooling and lazy initialization
    • support for mid-stack inlining and lock-free optimizations
    • better support for go modules
    • feature support for MissingFielder, MaxDepth, etc
    • general performance improvements across the board
    Source code(tar.gz)
    Source code(zip)
  • v1.1.1(Apr 9, 2018)

    This is a production release of go-codec.

    It includes fixes including:

    • 32-bit OS support
    • more robust chan encoding/decoding
    • at end of Encode(), call Flush() method of passed io.Reader if available
    • handle struct field of type *time.Time appropriately
    • improved error messages
    Source code(tar.gz)
    Source code(zip)
  • v1.1(Jan 21, 2018)

    This is a production release of go-codec.

    It includes much more tests (with coverage > 90%), bug fixes, and exceptional performance improvements.

    Out of the box, you get performance that rivals code generation. codecgen only gives 20-50% performance improvement, not the 2X-5X performance improvements it gave before.

    This should fulfill the commitment made in #113 to give better confidence in using master.

    This package is very feature-rich, seamlessly supports multiple formats and is highly performant compared to the competition. Please see https://github.com/ugorji/go-codec-bench for details on performance comparison vs other libraries.

    Source code(tar.gz)
    Source code(zip)
Owner
Ugorji Nwoke
Ugorji Nwoke
msgpack.org[Go] MessagePack encoding for Golang

MessagePack encoding for Golang ❤️ Uptrace.dev - All-in-one tool to optimize performance and monitor errors & logs Join Discord to ask questions. Docu

Vladimir Mihailenco 1.8k Jun 28, 2022
Cap'n Proto library and parser for go. This is go-capnproto-1.0, and does not have rpc. See https://github.com/zombiezen/go-capnproto2 for 2.0 which has rpc and capabilities.

Version 1.0 vs 2.0 Update 2015 Sept 20: Big news! Version 2.0 of the go-bindings, authored by Ross Light, is now released and newly available! It feat

Jason E. Aten, Ph.D. 282 Jun 9, 2022
csvutil provides fast and idiomatic mapping between CSV and Go (golang) values.

csvutil Package csvutil provides fast and idiomatic mapping between CSV and Go (golang) values. This package does not provide a CSV parser itself, it

Jacek Szwec 720 Jun 24, 2022
Generate TypeScript interfaces from Go structs/interfaces - useful for JSON RPC

bel Generate TypeScript interfaces from Go structs/interfaces - useful for JSON RPC bel is used in production in https://gitpod.io. Getting started be

Christian Weichel 24 Jun 13, 2022
Go package for dealing with maps, slices, JSON and other data.

Objx Objx - Go package for dealing with maps, slices, JSON and other data. Get started: Install Objx with one line of code, or update it with another

Stretchr, Inc. 523 Jun 21, 2022
A high-performance 100% compatible drop-in replacement of "encoding/json"

A high-performance 100% compatible drop-in replacement of "encoding/json" You can also use thrift like JSON using thrift-iterator Benchmark Source cod

Jsoniter 11k Jun 30, 2022
Encode and decode binary message and file formats in Go

Encode and Decode Binary Formats in Go This module wraps the package encoding/binary of the Go standard library and provides the missing Marshal() and

Joel Ling 7 Jun 3, 2022
Some Golang types based on builtin. Implements interfaces Value / Scan and MarshalJSON / UnmarshalJSON for simple working with database NULL-values and Base64 encoding / decoding.

gotypes Some simple types based on builtin Golang types that implement interfaces for working with DB (Scan / Value) and JSON (Marshal / Unmarshal). N

null 0 Feb 12, 2022
Asn.1 BER and DER encoding library for golang.

WARNING This repo has been archived! NO further developement will be made in the foreseen future. asn1 -- import "github.com/PromonLogicalis/asn1" Pac

Logicalis 50 Apr 8, 2022
Go library for decoding generic map values into native Go structures and vice versa.

mapstructure mapstructure is a Go library for decoding generic map values to structures and vice versa, while providing helpful error handling. This l

Mitchell Hashimoto 5.8k Jun 26, 2022
Easily and dynamically generate maps from Go static structures

structomap This package helps you to transform your struct into map easily. It provides a structomap.Serializer interface implemented by the structoma

Daniel Perez 132 Jun 12, 2022
Encode and decode Go (golang) struct types via protocol buffers.

protostructure protostructure is a Go library for encoding and decoding a struct type over the wire. This library is useful when you want to send arbi

Mitchell Hashimoto 171 Mar 24, 2022
Simple, specialised, and efficient binary marshaling

?? surge Documentation A library for fast binary (un)marshaling. Designed to be used in Byzantine networks, ?? surge never explicitly panics, protects

Ren 38 Mar 3, 2022
An optimal, byte-aligned, LZ+RLE hybrid encoder, designed to maximize decoding speed on NMOS 6502 and derived CPUs

TSCrunch TSCrunch is an optimal, byte-aligned, LZ+RLE hybrid encoder, designed to maximize decoding speed on NMOS 6502 and derived CPUs, while keeping

null 26 Jun 10, 2022
CBOR RFC 7049 (Go/Golang) - safe & fast with standard API + toarray & keyasint, CBOR tags, float64/32/16, fuzz tested.

CBOR library in Go fxamacker/cbor is a CBOR encoder & decoder in Go. It has a standard API, CBOR tags, options for duplicate map keys, float64→32→16,

Faye Amacker 455 Jun 24, 2022
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.

Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package. The library allows you to call Go service methods from PHP with a minimal footprint, structures and []byte support.

Spiral Scout 1.1k Jun 24, 2022
msgpack.org[Go] MessagePack encoding for Golang

MessagePack encoding for Golang ❤️ Uptrace.dev - distributed traces, logs, and errors in one place Join Discord to ask questions. Documentation Refere

Vladimir Mihailenco 1.8k Jun 29, 2022
msgpack.org[Go] MessagePack encoding for Golang

MessagePack encoding for Golang ❤️ Uptrace.dev - All-in-one tool to optimize performance and monitor errors & logs Join Discord to ask questions. Docu

Vladimir Mihailenco 1.8k Jun 28, 2022
An golang log lib, supports tracking and level, wrap by standard log lib

Logex An golang log lib, supports tracing and level, wrap by standard log lib How To Get shell go get gopkg.in/logex.v1 source code import "gopkg.in/

chzyer 39 Apr 15, 2022
rpc/v2 support for JSON-RPC 2.0 Specification.

rpc rpc/v2 support for JSON-RPC 2.0 Specification. gorilla/rpc is a foundation for RPC over HTTP services, providing access to the exported methods of

High Performance, Kubernetes Native Object Storage 3 Jul 4, 2021
Package flac provides access to FLAC (Free Lossless Audio Codec) streams.

flac This package provides access to FLAC (Free Lossless Audio Codec) streams. Documentation Documentation provided by GoDoc. flac: provides access to

null 215 Jun 27, 2022
H265/HEVC HM Video Codec in Go

GoHM H.265/HEVC HM Video Codec in Go Release Note: +v1.0.b03092013 -fix multiple bugs in decoder +v1.0.b03032013 -Initial release candidate for GoHM 1

Rain Liu 58 Dec 22, 2021
A codec for Go structs with support for chainable encoding/decoding hooks.

structool A codec for Go structs with support for chainable encoding/decoding hooks. Features Provide a uniform codec by combining mapstructure and st

Luo Peng 11 Jan 15, 2022
Image compression codec for 16 bit medical images

MIC - Medical Image Codec This library introduces a lossless medical image compression codec MIC for 16 bit images which provides compression ratio si

Kuldeep S 0 Dec 26, 2021
Github-org-diff - Simple CLI tool to check a diff between 2 branches of all org repos

github-org-diff Simple CLI tool to list org repos that have diff between dev and

Alex Pliutau 1 Jan 25, 2022
Library to integrate github.com/google/uuid with gopkg.in/vmihailenco/msgpack

Library to integrate github.com/google/uuid with gopkg.in/vmihailenco/msgpack

Citilink LLC 73 Apr 26, 2022
Cap'n Proto library and parser for go. This is go-capnproto-1.0, and does not have rpc. See https://github.com/zombiezen/go-capnproto2 for 2.0 which has rpc and capabilities.

Version 1.0 vs 2.0 Update 2015 Sept 20: Big news! Version 2.0 of the go-bindings, authored by Ross Light, is now released and newly available! It feat

Jason E. Aten, Ph.D. 282 Jun 9, 2022
Antenna RPC is an RPC protocol for distributed computing, it's based on QUIC and Colfer. its currently an WIP.

aRPC - Antenna Remote Procedure Call Antenna remote procedure call (aRPC) is an RPC protocol focused on distributed processing and HPC. aRPC is implem

Raphael de Carvalho Almeida 3 Jun 16, 2021
go-opa-validate is an open-source lib that evaluates OPA (open policy agent) policy against JSON or YAML data.

go-opa-validate go-opa-validate is an open-source lib that evaluates OPA (open policy agent) policy against JSON or YAML data. Installation Usage Cont

chenk 5 Feb 5, 2022