Fast JSON encoder/decoder compatible with encoding/json for Go

Overview

go-json

Go GoDoc codecov

Fast JSON encoder/decoder compatible with encoding/json for Go

Installation

go get github.com/goccy/go-json

How to use

Replace import statement from encoding/json to github.com/goccy/go-json

-import "encoding/json"
+import "github.com/goccy/go-json"

JSON library comparison

name encoder decoder compatible with encoding/json
encoding/json yes yes N/A
json-iterator/go yes yes partial
easyjson yes yes no
gojay yes yes no
segmentio/encoding/json yes yes yes
jettison yes no no
simdjson-go no yes no
goccy/go-json yes yes yes
  • json-iterator/go isn't compatible with encoding/json in many ways, but it hasn't been supported for a long time.

Benchmarks

$ cd benchmarks
$ go test -bench .

Encode

Decode

How it works

go-json is very fast in both encoding and decoding compared to other libraries. It's easier to implement by using automatic code generation for performance or by using a dedicated interface, but go-json dares to stick to compatibility with encoding/json and is the simple interface. Despite this, we are developing with the aim of being the fastest library.

Here, we explain the various speed-up techniques implemented by go-json.

Basic technique

The techniques listed here are the ones used by most of the libraries listed above.

Buffer reuse

Since the only value required for the result of json.Marshal(interface{}) ([]byte, error) is []byte, the only value that must be allocated during encoding is the return value []byte .

Also, as the number of allocations increases, the performance will be affected, so the number of allocations should be kept as low as possible when creating []byte.

Therefore, there is a technique to reduce the number of times a new buffer must be allocated by reusing the buffer used for the previous encoding by using sync.Pool.

Finally, you allocate a buffer that is as long as the resulting buffer and copy the contents into it, you only need to allocate the buffer once in theory.

type buffer struct {
    data []byte
}

var bufPool = sync.Pool{
    New: func() interface{} {
        return &buffer{data: make([]byte, 0, 1024)}
    },
}

buf := bufPool.Get().(*buffer)
data := encode(buf.data) // reuse buf.data

newBuf := make([]byte, len(data))
copy(newBuf, buf)

buf.data = data
bufPool.Put(buf)

Elimination of reflection

As you know, the reflection operation is very slow.

Therefore, using the fact that the address position where the type information is stored is fixed for each binary ( we call this typeptr ), we can use the address in the type information to call a pre-built optimized process.

For example, you can get the address to the type information from interface{} as follows and you can use that information to call a process that does not have reflection.

To process without reflection, pass a pointer (unsafe.Pointer) to the value is stored.

type emptyInterface struct {
    typ unsafe.Pointer
    ptr unsafe.Pointer
}

var typeToEncoder = map[uintptr]func(unsafe.Pointer)([]byte, error){}

func Marshal(v interface{}) ([]byte, error) {
    iface := (*emptyInterface)(unsafe.Pointer(&v)
    typeptr := uintptr(iface.typ)
    if enc, exists := typeToEncoder[typeptr]; exists {
        return enc(iface.ptr)
    }
    ...
}

※ In reality, typeToEncoder can be referenced by multiple goroutines, so exclusive control is required.

Unique speed-up technique

Encoder

Do not escape arguments of Marshal

json.Marshal and json.Unmarshal receive interface{} value and they perform type determination dynamically to process. In normal case, you need to use the reflect library to determine the type dynamically, but since reflect.Type is defined as interface, when you call the method of reflect.Type, The reflect's argument is escaped.

Therefore, the arguments for Marshal and Unmarshal are always escape to the heap. However, go-json can use the feature of reflect.Type while avoiding escaping.

reflect.Type is defined as interface, but in reality reflect.Type is implemented only by the structure rtype defined in the reflect package. For this reason, to date reflect.Type is the same as *reflect.rtype.

Therefore, by directly handling *reflect.rtype, which is an implementation of reflect.Type, it is possible to avoid escaping because it changes from interface to using struct.

The technique for working with *reflect.rtype directly from go-json is implemented at https://github.com/goccy/go-json/blob/master/rtype.go.

Also, the same technique is cut out as a library ( https://github.com/goccy/go-reflect )

Initially this feature was the default behavior of go-json. But after careful testing, I found that I passed a large value to json.Marshal() and if the argument could not be assigned to the stack, it could not be properly escaped to the heap (a bug in the Go compiler).

Therefore, this feature will be provided as an optional until this issue is resolved.

To use it, add NoEscape like MarshalNoEscape()

Encoding using opcode sequence

I explained that you can use typeptr to call a pre-built process from type information.

In other libraries, this dedicated process is processed by making it an function calling like anonymous function, but function calls are inherently slow processes and should be avoided as much as possible.

Therefore, go-json adopted the Instruction-based execution processing system, which is also used to implement virtual machines for programming language.

If it is the first type to encode, create the opcode ( instruction ) sequence required for encoding. From the second time onward, use typeptr to get the cached pre-built opcode sequence and encode it based on it. An example of the opcode sequence is shown below.

json.Marshal(struct{
    X int `json:"x"`
    Y string `json:"y"`
}{X: 1, Y: "hello"})

When encoding a structure like the one above, create a sequence of opcodes like this:

- opStructFieldHead ( `{` )
- opStructFieldInt ( `"x": 1,` )
- opStructFieldString ( `"y": "hello"` )
- opStructEnd ( `}` )
- opEnd

※ When processing each operation, write the letters on the right.

In addition, each opcode is managed by the following structure ( Pseudo code ).

type opType int
const (
    opStructFieldHead opType = iota
    opStructFieldInt
    opStructFieldStirng
    opStructEnd
    opEnd
)
type opcode struct {
    op opType
    key []byte
    next *opcode
}

The process of encoding using the opcode sequence is roughly implemented as follows.

func encode(code *opcode, b []byte, p unsafe.Pointer) ([]byte, error) {
    for {
        switch code.op {
        case opStructFieldHead:
            b = append(b, '{')
            code = code.next
        case opStructFieldInt:
            b = append(b, code.key...)
            b = appendInt((*int)(unsafe.Pointer(uintptr(p)+code.offset)))
            code = code.next
        case opStructFieldString:
            b = append(b, code.key...)
            b = appendString((*string)(unsafe.Pointer(uintptr(p)+code.offset)))
            code = code.next
        case opStructEnd:
            b = append(b, '}')
            code = code.next
        case opEnd:
            goto END
        }
    }
END:
    return b, nil
}

In this way, the huge switch-case is used to encode by manipulating the linked list opcodes to avoid unnecessary function calls.

Opcode sequence optimization

One of the advantages of encoding using the opcode sequence is the ease of optimization. The opcode sequence mentioned above is actually converted into the following optimized operations and used.

- opStructFieldHeadInt ( `{"x": 1,` )
- opStructEndString ( `"y": "hello"}` )
- opEnd

It has been reduced from 5 opcodes to 3 opcodes ! Reducing the number of opcodees means reducing the number of branches with switch-case. In other words, the closer the number of operations is to 1, the faster the processing can be performed.

In go-json, optimization to reduce the number of opcodes itself like the above and it speeds up by preparing opcodes with optimized paths.

Change recursive call from CALL to JMP

Recursive processing is required during encoding if the type is defined recursively as follows:

type T struct {
    X int
    U *U
}

type U struct {
    T *T
}

b, err := json.Marshal(&T{
    X: 1,
    U: &U{
        T: &T{
            X: 2,
        },
    },
})
fmt.Println(string(b)) // {"X":1,"U":{"T":{"X":2,"U":null}}}

In go-json, recursive processing is processed by the operation type of opStructFieldRecursive.

In this operation, after acquiring the opcode sequence used for recursive processing, the function is not called recursively as it is, but the necessary values ​​are saved by itself and implemented by moving to the next operation.

The technique of implementing recursive processing with the JMP operation while avoiding the CALL operation is a famous technique for implementing a high-speed virtual machine.

For more details, please refer to the article ( but Japanese only ).

Dispatch by typeptr from map to slice

When retrieving the data cached from the type information by typeptr, we usually use map. Map requires exclusive control, so use sync.Map for a naive implementation.

However, this is slow, so it's a good idea to use the atomic package for exclusive control as implemented by segmentio/encoding/json ( https://github.com/segmentio/encoding/blob/master/json/codec.go#L41-L55 ).

This implementation slows down the set instead of speeding up the get, but it works well because of the nature of the library, it encodes much more for the same type.

However, as a result of profiling, I noticed that runtime.mapaccess2 accounts for a significant percentage of the execution time. So I thought if I could change the lookup from map to slice.

There is an API named typelinks defined in the runtime package that the reflect package uses internally. This allows you to get all the type information defined in the binary at runtime.

The fact that all type information can be acquired means that by constructing slices in advance with the acquired total number of type information, it is possible to look up with the value of typeptr without worrying about out-of-range access.

However, if there is too much type information, it will use a lot of memory, so by default we will only use this optimization if the slice size fits within 2Mib .

If this approach is not available, it will fall back to the atomic based process described above.

If you want to know more, please refer to the implementation here

Decoder

Dispatch by typeptr from map to slice

Like the encoder, the decoder also uses typeptr to call the dedicated process.

Faster termination character inspection using NUL character

In order to decode, you have to traverse the input buffer character by position. At that time, if you check whether the buffer has reached the end, it will be very slow.

buf : []byte type variable. holds the string passed to the decoder cursor : int64 type variable. holds the current read position

buflen := len(buf)
for ; cursor < buflen; cursor++ { // compare cursor and buflen at all times, it is so slow.
    switch buf[cursor] {
    case ' ', '\n', '\r', '\t':
    }
}

Therefore, by adding the NUL (\000) character to the end of the read buffer as shown below, it is possible to check the termination character at the same time as other characters.

for {
    switch buf[cursor] {
    case ' ', '\n', '\r', '\t':
    case '\000':
        return nil
    }
    cursor++
}

Use Boundary Check Elimination

Due to the NUL character optimization, the Go compiler does a boundary check every time, even though buf[cursor] does not cause out-of-range access.

Therefore, go-json eliminates boundary check by fetching characters for hotspot by pointer operation. For example, the following code.

func char(ptr unsafe.Pointer, offset int64) byte {
	return *(*byte)(unsafe.Pointer(uintptr(ptr) + uintptr(offset)))
}

p := (*sliceHeader)(&unsafe.Pointer(buf)).data
for {
    switch char(p, cursor) {
    case ' ', '\n', '\r', '\t':
    case '\000':
        return nil
    }
    cursor++
}

Checking the existence of fields of struct using Bitmaps

I found by the profiling result, in the struct decode, lookup process for field was taking a long time.

For example, consider decoding a string like {"a":1,"b":2,"c":3} into the following structure:

type T struct {
    A int `json:"a"`
    B int `json:"b"`
    C int `json:"c"`
}

At this time, it was found that it takes a lot of time to acquire the decoding process corresponding to the field from the field name as shown below during the decoding process.

fieldName := decodeKey(buf, cursor) // "a" or "b" or "c"
decoder, exists := fieldToDecoderMap[fieldName] // so slow
if exists {
    decoder(buf, cursor)
} else {
    skipValue(buf, cursor)
}

To improve this process, json-iterator/go is optimized so that it can be branched by switch-case when the number of fields in the structure is 10 or less (switch-case is faster than map). However, there is a risk of hash collision because the value hashed by the FNV algorithm is used for conditional branching. Also, gojay processes this part at high speed by letting the library user yourself write switch-case.

go-json considers and implements a new approach that is different from these. I call this bitmap field optimization.

The range of values ​​per character can be represented by [256]byte. Also, if the number of fields in the structure is 8 or less, int8 type can represent the state of each field. In other words, it has the following structure.

  • Base ( 8bit ): 00000000
  • Key "a": 00000001 ( assign key "a" to the first bit )
  • Key "b": 00000010 ( assign key "b" to the second bit )
  • Key "c": 00000100 ( assign key "c" to the third bit )

Bitmap structure is the following

        | key index(0) |
------------------------
 0      | 00000000     |
 1      | 00000000     |
~~      |              |
97 (a)  | 00000001     |
98 (b)  | 00000010     |
99 (c)  | 00000100     |
~~      |              |
255     | 00000000     |

You can think of this as a Bitmap with a height of 256 and a width of the maximum string length in the field name. In other words, it can be represented by the following type .

[maxFieldKeyLength][256]int8`

When decoding a field character, check whether the corresponding character exists by referring to the pre-built bitmap like the following.

var curBit int8 = math.MaxInt8 // 11111111

c := char(buf, cursor)
bit := bitmap[keyIdx][c]
curBit &= bit
if curBit == 0 {
    // not found field
}

If curBit is not 0 until the end of the field string, then the string is You may have hit one of the fields. But the possibility is that if the decoded string is shorter than the field string, you will get a false hit.

  • input: {"a":1}
type T struct {
    X int `json:"abc"`
}

※ Since a is shorter than abc, it can decode to the end of the field character without curBit being 0.

Rest assured. In this case, it doesn't matter because you can tell if you hit by comparing the string length of a with the string length of abc.

Finally, calculate the position of the bit where 1 is set and get the corresponding value, and you're done.

Using this technique, field lookups are possible with only bitwise operations and access to slices.

go-json uses a similar technique for fields with 9 or more and 16 or less fields. At this time, Bitmap is constructed as [maxKeyLen][256]int16 type.

Currently, this optimization is not performed when the maximum length of the field name is long (specifically, 64 bytes or more) in addition to the limitation of the number of fields from the viewpoint of saving memory usage.

Others

I have done a lot of other optimizations. I will find time to write about them. If you have any questions about what's written here or other optimizations, please visit the #go-json channel on gophers.slack.com .

Reference

Regarding the story of go-json, there are the following articles in Japanese only.

License

MIT

Issues
  • Weird behavior when swapping from encoding/json to github.com/goccy/go-json

    Weird behavior when swapping from encoding/json to github.com/goccy/go-json

    % go version
    go version go1.16beta1 darwin/arm64
    

    In github.com/lestrrat-go/jwx (topic/v1.1.0-marshal-optimize branch, commit bc72347), if you just run some tests, it passes.

    % make smoke
    <snip>
    PASS
    ok  	github.com/lestrrat-go/jwx/examples	4.057s
    ok  	github.com/lestrrat-go/jwx	(cached)
    ?   	github.com/lestrrat-go/jwx/cmd/jwx	[no test files]
    <snip>
    

    Now, apply the following patch

    diff --git a/internal/json/json.go b/internal/json/json.go
    index 0eaf063..f90e8bf 100644
    --- a/internal/json/json.go
    +++ b/internal/json/json.go
    @@ -2,10 +2,10 @@ package json
     
     import (
            "bytes"
    -       "encoding/json"
            "io"
            "sync"
     
    +       "github.com/goccy/go-json"
            "github.com/lestrrat-go/jwx/internal/base64"
            "github.com/pkg/errors"
     )
    

    Then run

    make tidy   # runs go mod tidy where applicable
    

    and then run some tests.

    % make smoke
    <snip>
    --- FAIL: ExampleJWS_Message (0.00s)
    got:
    json: error calling MarshalJSON for type *jws.Message: failed to handle opcode. doesn't implement StructFieldOmitEmptyStringPtr
    <snip>
    

    Further more, run the following

    cd jwk && go test
    

    And then BOOOM

    unexpected fault address 0x23a0000015fd4
    fatal error: fault
    [signal SIGSEGV: segmentation violation code=0x2 addr=0x23a0000015fd4 pc=0x23a0000015fd4]
    
    goroutine 23 [running]:
    runtime.throw(0x105298293, 0x5)
    	/usr/local/go/src/runtime/panic.go:1112 +0x54 fp=0x14000167e30 sp=0x14000167e00 pc=0x104ef5204
    runtime.sigpanic()
    	/usr/local/go/src/runtime/signal_unix.go:737 +0x230 fp=0x14000167e70 sp=0x14000167e30 pc=0x104f0c7f0
    github.com/lestrrat-go/jwx/jwk_test.TestECDSA.func3(0x14000103380)
    	/Users/lestrrat/dev/src/github.com/lestrrat-go/jwx/jwk/ecdsa_test.go:189 +0x134 fp=0x14000167f70 sp=0x14000167e80 pc=0x10528b064
    testing.tRunner(0x14000103380, 0x1053fb868)
    	/usr/local/go/src/testing/testing.go:1194 +0xe0 fp=0x14000167fc0 sp=0x14000167f70 pc=0x104f9e4f0
    runtime.goexit()
    	/usr/local/go/src/runtime/asm_arm64.s:1130 +0x4 fp=0x14000167fc0 sp=0x14000167fc0 pc=0x104f2b634
    created by testing.(*T).Run
    	/usr/local/go/src/testing/testing.go:1239 +0x21c
    
    goroutine 1 [chan receive]:
    testing.(*T).Run(0x14000102a80, 0x1052994cb, 0x9, 0x1053fb888, 0x600998f7)
    	/usr/local/go/src/testing/testing.go:1240 +0x238
    testing.runTests.func1(0x14000102900)
    	/usr/local/go/src/testing/testing.go:1512 +0x74
    testing.tRunner(0x14000102900, 0x14000193da8)
    	/usr/local/go/src/testing/testing.go:1194 +0xe0
    testing.runTests(0x1400012c180, 0x1056310c0, 0x12, 0x12, 0xbffa84b3de79fc98, 0x8bb2dc144d, 0x105637340, 0x104f9609c)
    	/usr/local/go/src/testing/testing.go:1510 +0x284
    testing.(*M).Run(0x1400018a080, 0x0)
    	/usr/local/go/src/testing/testing.go:1418 +0x1b4
    main.main()
    	_testmain.go:79 +0x14c
    
    goroutine 19 [chan receive]:
    testing.(*T).Run(0x14000103380, 0x1052a525d, 0x1d, 0x1053fb868, 0x2e3b599a98d01)
    	/usr/local/go/src/testing/testing.go:1240 +0x238
    github.com/lestrrat-go/jwx/jwk_test.TestECDSA(0x14000102a80)
    	/Users/lestrrat/dev/src/github.com/lestrrat-go/jwx/jwk/ecdsa_test.go:170 +0xa0
    testing.tRunner(0x14000102a80, 0x1053fb888)
    	/usr/local/go/src/testing/testing.go:1194 +0xe0
    created by testing.(*T).Run
    	/usr/local/go/src/testing/testing.go:1239 +0x21c
    exit status 2
    FAIL	github.com/lestrrat-go/jwx/jwk	0.392s
    
    opened by lestrrat 25
  • [FOLLOW UP] - https://github.com/goccy/go-json/issues/104

    [FOLLOW UP] - https://github.com/goccy/go-json/issues/104

    https://github.com/goccy/go-json/issues/104

    I'm still getting same issue on same place

    encoder (escaped): opcode StructFieldOmitEmptyStruct has not been implemented
    
    opened by sujit-baniya 22
  • Fix decoding of slice with type implemented UnmarshalJSON

    Fix decoding of slice with type implemented UnmarshalJSON

    fix #195

    In decoding UnmarshalJSON, it is assumed that the passed pointer is used as it is as a receiver. If a type that implements UnmarshalJSON is specified as an element of slice, this should be taken into consideration so that the pointer of the reused slice element is not passed as it is.

    opened by goccy 15
  • Getting Error when trying to include json.Marshal in gofiber JsonEncoder

    Getting Error when trying to include json.Marshal in gofiber JsonEncoder

    I'm getting this error when trying to integrate this library with gofiber: https://github.com/gofiber/fiber

    failed to handle opcode. doesn't implement StructFieldOmitEmptyStruct

    opened by sujit-baniya 12
  • Add a lint action

    Add a lint action

    fixes #105

    opened by lestrrat 10
  • Panic on encoding struct in a loop

    Panic on encoding struct in a loop

    I am currently experiencing a panic when encoding a struct during some JSON-testdata generation. The testdata in this case is a repetition of the same struct being encoded 50k times. The first 3-5k encodings work, after that I always get a panic. Encoding with stdlib encoding/json works without a problem. If you need any additional information to debug this I am happy to assist.

    unexpected fault address 0xc000714000
    fatal error: fault
    [signal 0xc0000005 code=0x0 addr=0xc000714000 pc=0x705ea9]
    
    goroutine 27 [running]:
    runtime.throw({0xf9f839, 0x400})
    	C:/Program Files/Go/src/runtime/panic.go:1198 +0x76 fp=0xc00015c8a8 sp=0xc00015c878 pc=0x2e6f76
    runtime.sigpanic()
    	C:/Program Files/Go/src/runtime/signal_windows.go:260 +0x10c fp=0xc00015c8f0 sp=0xc00015c8a8 pc=0x2f9e6c
    github.com/goccy/go-json/internal/encoder/vm.ptrToUint64(...)
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/vm/util.go:71
    github.com/goccy/go-json/internal/encoder/vm.Run(0xc0008da9c0, {0xc0000f5c00, 0xc00015fdf0, 0x2bd234}, 0xc00015fda0)
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/vm/vm.go:4360 +0x1df29 fp=0xc00015fd60 sp=0xc00015c8f0 pc=0x705ea9
    github.com/goccy/go-json/internal/encoder/vm.DebugRun(0x8, {0xc0000f5c00, 0x0, 0x40}, 0x64184e)
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/vm/debug_vm.go:33 +0x7f fp=0xc00015fde8 sp=0xc00015fd60 pc=0x6e785f
    github.com/goccy/go-json.encodeRunCode(0xf04720, {0xc0000f5c00, 0xc0000f5c00, 0xc00093a29a}, 0x2030000)
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/encode.go:302 +0x45 fp=0xc00015fe20 sp=0xc00015fde8 pc=0x7196a5
    github.com/goccy/go-json.encode(0xc0008da9c0, {0xf04720, 0xc000713ff0})
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/encode.go:232 +0x21c fp=0xc00015fea0 sp=0xc00015fe20 pc=0x7191dc
    github.com/goccy/go-json.(*Encoder).encodeWithOption(0xc00015ffa8, 0xc0008da9c0, {0xf04720, 0xc000713ff0}, {0xc00015ff98, 0x1, 0xc00015ff58})
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/encode.go:74 +0xeb fp=0xc00015ff00 sp=0xc00015fea0 pc=0x718cab
    github.com/goccy/go-json.(*Encoder).EncodeWithOption(0xc0000f5c00, {0xf04720, 0xc000713ff0}, {0xc00015ff98, 0x1, 0x1})
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/encode.go:41 +0x92 fp=0xc00015ff68 sp=0xc00015ff00 pc=0x718b32
    gitlab.example.org/fsw-dl/dl/pkg/envelope.Envelope.MarshalJSON({{0xf98217, 0x2}, 0xcaa, {0xc04fcbe570f13814, 0x3475845, 0x193a680}, {0x0, 0x0}, {0xf04720, 0xc000713ff0}})
    	C:/Users/philband/go/src/gitlab.example.org/fsw-dl/dl/pkg/envelope/envelope.go:143 +0xb0 fp=0xc0001600a0 sp=0xc00015ff68 pc=0xd83830
    gitlab.example.org/fsw-dl/dl/pkg/envelope.(*Envelope).MarshalJSON(0xc00014f6d0)
    	<autogenerated>:1 +0x78 fp=0xc000160150 sp=0xc0001600a0 pc=0xd87758
    github.com/goccy/go-json/internal/encoder.AppendMarshalJSON(0xc0005bc000, 0xc00019e400, {0xc000baa000, 0x0, 0x400}, {0xf28140, 0xc00014f6d0})
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/encoder.go:387 +0x41c fp=0xc0001602c8 sp=0xc000160150 pc=0x64cf1c
    github.com/goccy/go-json/internal/encoder/vm.appendMarshalJSON(0x0, 0x0, {0xc000baa000, 0x0, 0x0}, {0xf28140, 0xc00014f6d0})
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/vm/util.go:136 +0x28 fp=0xc000160310 sp=0xc0001602c8 pc=0x6e7d88
    github.com/goccy/go-json/internal/encoder/vm.Run(0xc0005bc000, {0xc000baa000, 0x1b29d620108, 0x70}, 0xc000680000)
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/vm/vm.go:280 +0x17f3 fp=0xc000163780 sp=0xc000160310 pc=0x6e9773
    github.com/goccy/go-json/internal/encoder/vm.DebugRun(0x0, {0xc000baa000, 0x9, 0x70}, 0x1b2c2a5a7b6)
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/vm/debug_vm.go:33 +0x7f fp=0xc000163808 sp=0xc000163780 pc=0x6e785f
    github.com/goccy/go-json.encodeRunCode(0xf28140, {0xc000baa000, 0x2030000, 0x20}, 0x2030000)
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/encode.go:302 +0x45 fp=0xc000163840 sp=0xc000163808 pc=0x7196a5
    github.com/goccy/go-json.encode(0xc0005bc000, {0xf28140, 0xc00014f6d0})
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/encode.go:232 +0x21c fp=0xc0001638c0 sp=0xc000163840 pc=0x7191dc
    github.com/goccy/go-json.(*Encoder).encodeWithOption(0xc000163aa0, 0xc0005bc000, {0xf28140, 0xc00014f6d0}, {0xc000163a40, 0x1, 0xf28140})
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/encode.go:74 +0xeb fp=0xc000163920 sp=0xc0001638c0 pc=0x718cab
    github.com/goccy/go-json.(*Encoder).EncodeWithOption(0xf28140, {0xf28140, 0xc00014f6d0}, {0xc000163a40, 0x1, 0x1})
    	C:/Users/philband/go/pkg/mod/github.com/goccy/[email protected]/encode.go:41 +0x92 fp=0xc000163988 sp=0xc000163920 pc=0x718b32
    gitlab.example.org/fsw-dl/dl/pkg/envelope.(*Wal).Write(0xc00059e540, {0xc00013d800, 0x15, 0xc04fcbe570f13814})
    	C:/Users/philband/go/src/gitlab.example.org/fsw-dl/dl/pkg/envelope/wal.go:224 +0x43f fp=0xc000163af0 sp=0xc000163988 pc=0xd861df
    gitlab.example.org/fsw-dl/dl/pkg/envelope.(*EnvelopeLogger).RunLogger(0xc000834af0, {0x1184d58, 0xc00045c600}, 0xc0000a36e0)
    	C:/Users/philband/go/src/gitlab.example.org/fsw-dl/dl/pkg/envelope/envelope_logger.go:111 +0xc28 fp=0xc000163f48 sp=0xc000163af0 pc=0xd84788
    main.main.func1()
    	C:/Users/philband/go/src/gitlab.example.org/fsw-dl/dl/cmd/client/main.go:74 +0x29 fp=0xc000163f78 sp=0xc000163f48 pc=0xd889c9
    golang.org/x/sync/errgroup.(*Group).Go.func1()
    	C:/Users/philband/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:57 +0x67 fp=0xc000163fe0 sp=0xc000163f78 pc=0xabf9a7
    runtime.goexit()
    	C:/Program Files/Go/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc000163fe8 sp=0xc000163fe0 pc=0x314ce1
    created by golang.org/x/sync/errgroup.(*Group).Go
    	C:/Users/philband/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:54 +0x92
    

    The data being encoded (static only for testing)

    Envelope{
    	Type: LogMessageIvtName,
    	Data: LogMessageIvt{
    		Timestamp:   1456465,
    		Voltage:     600000,
    		Current:     50000,
    		Temperature: 544,
    	},
    }
    

    Relevant Code extracts

    type EnvelopeName string
    
    const (
    	LogMessageIvtName  EnvelopeName = "lm"
    )
    
    type Envelope struct {
    	Type      EnvelopeName `json:"t"`                   
    	Index     uint64       `json:"i,omitempty"`         
    	Timestamp time.Time    `json:"ts,omitempty"`        
    	Signature string       `json:"s,omitempty" hash:"-"`
    	Data      interface{}  `json:"d"`                   
    }
    
    type EnvelopeWithRawData struct {
    	Type      EnvelopeName    `json:"t"`                    
    	Index     uint64          `json:"i,omitempty"`          
    	Timestamp time.Time       `json:"ts,omitempty"`         
    	Signature string          `json:"s,omitempty" hash:"-"` 
    	Data      json.RawMessage `json:"d"`                    
    }
    
    type LogMessageIvt struct {
    	Timestamp   uint32 `json:"ts"`
    	Voltage     int32  `json:"u"`
    	Current     int32  `json:"i"`
    	Temperature int32  `json:"temp"`
    }
    
    func (t Envelope) MarshalJSON() ([]byte, error) {
    	jsonData, err := json.Marshal(t.Data)
    	if err != nil {
    		return nil, err
    	}
    	e := EnvelopeWithRawData{
    		Type:      t.Type,
    		Index:     t.Index,
    		Timestamp: t.Timestamp,
    		Signature: t.Signature,
    		Data:      jsonData,
    	}
    	if !e.Timestamp.IsZero() {
    		e.Timestamp = time.Now()
    	}
    	return json.Marshal(e)
    }
    
    opened by philband 8
  • compile: signal: killed

    compile: signal: killed

    Running with 7Gb outh of 16 Gb available on local machine using new Go 1.16 compiler:

    go build github.com/goccy/go-json: /usr/local/go/pkg/tool/linux_amd64/compile: signal: killed
    
     go get -u -x github.com/goccy/go-json 
    # get https://goproxy.internal.lan/github.com/goccy/@v/list
    # get https://goproxy.internal.lan/github.com/goccy/go-json/@v/list
    # get https://goproxy.internal.lan/github.com/@v/list
    # get https://goproxy.internal.lan/github.com/goccy/go-json/@v/list: 200 OK (0.057s)
    # get https://goproxy.internal.lan/github.com/goccy/@v/list: 404 Not Found (0.057s)
    # get https://goproxy.internal.lan/github.com/@v/list: 404 Not Found (0.057s)
    WORK=/tmp/go-build3285744396
    mkdir -p $WORK/b001/
    cat >$WORK/b001/importcfg << 'EOF' # internal
    # import config
    packagefile bytes=/usr/local/go/pkg/linux_amd64/bytes.a
    packagefile encoding=/usr/local/go/pkg/linux_amd64/encoding.a
    packagefile encoding/base64=/usr/local/go/pkg/linux_amd64/encoding/base64.a
    packagefile errors=/usr/local/go/pkg/linux_amd64/errors.a
    packagefile fmt=/usr/local/go/pkg/linux_amd64/fmt.a
    packagefile io=/usr/local/go/pkg/linux_amd64/io.a
    packagefile math=/usr/local/go/pkg/linux_amd64/math.a
    packagefile math/bits=/usr/local/go/pkg/linux_amd64/math/bits.a
    packagefile reflect=/usr/local/go/pkg/linux_amd64/reflect.a
    packagefile runtime=/usr/local/go/pkg/linux_amd64/runtime.a
    packagefile sort=/usr/local/go/pkg/linux_amd64/sort.a
    packagefile strconv=/usr/local/go/pkg/linux_amd64/strconv.a
    packagefile strings=/usr/local/go/pkg/linux_amd64/strings.a
    packagefile sync=/usr/local/go/pkg/linux_amd64/sync.a
    packagefile sync/atomic=/usr/local/go/pkg/linux_amd64/sync/atomic.a
    packagefile unicode=/usr/local/go/pkg/linux_amd64/unicode.a
    packagefile unicode/utf16=/usr/local/go/pkg/linux_amd64/unicode/utf16.a
    packagefile unicode/utf8=/usr/local/go/pkg/linux_amd64/unicode/utf8.a
    EOF
    
    /usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/b001/_pkg_.a -trimpath "$WORK/b001=>" -p github.com/goccy/go-json -lang=go1.12 -complete -buildid enu8Gz5ZXEiCvPU98SWs/enu8Gz5ZXEiCvPU98SWs -goversion go1.16 -D "" -importcfg $WORK/b001/importcfg -pack -c=4 /home/iru/go/pkg/mod/github.com/goccy/[email protected]/codec.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/compact.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_anonymous_field.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_array.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_bool.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_bytes.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_compile.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_compile_norace.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_context.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_float.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_int.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_interface.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_map.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_number.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_ptr.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_slice.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_stream.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_string.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_struct.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_uint.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_unmarshal_json.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_unmarshal_text.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_wrapped_string.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_compile.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_compile_norace.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_context.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_int.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_map113.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_opcode.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_optype.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_string.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_vm.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_vm_escaped.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_vm_escaped_indent.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_vm_indent.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/error.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/indent.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/json.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/option.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/rtype.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/struct_field.go
    go build github.com/goccy/go-json: /usr/local/go/pkg/tool/linux_amd64/compile: signal: killed
    
    via 🐹 v1.16 
    ❯ go get -u -x github.com/goccy/go-json
    # get https://goproxy.internal.lan/github.com/@v/list
    # get https://goproxy.internal.lan/github.com/goccy/@v/list
    # get https://goproxy.internal.lan/github.com/goccy/go-json/@v/list
    # get https://goproxy.internal.lan/github.com/goccy/@v/list: 404 Not Found (0.062s)
    # get https://goproxy.internal.lan/github.com/goccy/go-json/@v/list: 200 OK (0.062s)
    # get https://goproxy.internal.lan/github.com/@v/list: 404 Not Found (0.063s)
    WORK=/tmp/go-build3042749253
    mkdir -p $WORK/b001/
    cat >$WORK/b001/importcfg << 'EOF' # internal
    # import config
    packagefile bytes=/usr/local/go/pkg/linux_amd64/bytes.a
    packagefile encoding=/usr/local/go/pkg/linux_amd64/encoding.a
    packagefile encoding/base64=/usr/local/go/pkg/linux_amd64/encoding/base64.a
    packagefile errors=/usr/local/go/pkg/linux_amd64/errors.a
    packagefile fmt=/usr/local/go/pkg/linux_amd64/fmt.a
    packagefile io=/usr/local/go/pkg/linux_amd64/io.a
    packagefile math=/usr/local/go/pkg/linux_amd64/math.a
    packagefile math/bits=/usr/local/go/pkg/linux_amd64/math/bits.a
    packagefile reflect=/usr/local/go/pkg/linux_amd64/reflect.a
    packagefile runtime=/usr/local/go/pkg/linux_amd64/runtime.a
    packagefile sort=/usr/local/go/pkg/linux_amd64/sort.a
    packagefile strconv=/usr/local/go/pkg/linux_amd64/strconv.a
    packagefile strings=/usr/local/go/pkg/linux_amd64/strings.a
    packagefile sync=/usr/local/go/pkg/linux_amd64/sync.a
    packagefile sync/atomic=/usr/local/go/pkg/linux_amd64/sync/atomic.a
    packagefile unicode=/usr/local/go/pkg/linux_amd64/unicode.a
    packagefile unicode/utf16=/usr/local/go/pkg/linux_amd64/unicode/utf16.a
    packagefile unicode/utf8=/usr/local/go/pkg/linux_amd64/unicode/utf8.a
    EOF
    
    /usr/local/go/pkg/tool/linux_amd64/compile -o $WORK/b001/_pkg_.a -trimpath "$WORK/b001=>" -p github.com/goccy/go-json -lang=go1.12 -complete -buildid enu8Gz5ZXEiCvPU98SWs/enu8Gz5ZXEiCvPU98SWs -goversion go1.16 -D "" -importcfg $WORK/b001/importcfg -pack -c=4 /home/iru/go/pkg/mod/github.com/goccy/[email protected]/codec.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/compact.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_anonymous_field.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_array.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_bool.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_bytes.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_compile.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_compile_norace.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_context.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_float.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_int.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_interface.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_map.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_number.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_ptr.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_slice.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_stream.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_string.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_struct.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_uint.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_unmarshal_json.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_unmarshal_text.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/decode_wrapped_string.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_compile.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_compile_norace.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_context.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_int.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_map113.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_opcode.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_optype.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_string.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_vm.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_vm_escaped.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_vm_escaped_indent.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/encode_vm_indent.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/error.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/indent.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/json.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/option.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/rtype.go /home/iru/go/pkg/mod/github.com/goccy/[email protected]/struct_field.go
    go build github.com/goccy/go-json: /usr/local/go/pkg/tool/linux_amd64/compile: signal: killed
    

    Seems to be a similar issue -- https://github.com/blevesearch/segment/issues/5 -- "fast at runtime but slow compilation and this high memory requirement are side-effects"

    opened by franchb 8
  • [Bug] - Seems to have bug on unmarshal

    [Bug] - Seems to have bug on unmarshal

    Issue: Unable to unmarshal json if one of the field data type is different from actual data.

    Code:

    
    type Provider struct {
    	Name           string        `gorm:"name" json:"name" form:"name" query:"name"`
    	Slug           string        `gorm:"slug" json:"slug" form:"slug" query:"slug"`
    	Alias          string        `gorm:"alias" json:"alias" form:"alias" query:"alias"`
    	Description    string        `gorm:"description" json:"description" form:"description"`
    	ProviderType   string        `gorm:"provider_type" json:"provider_type" form:"provider_type" query:"provider_type"`
    	IsActive       bool          `gorm:"is_active" json:"is_active" form:"is_active"`
    	Currency       string        `gorm:"currency" json:"currency" form:"currency"`
    	Balance        money.Decimal `gorm:"column:balance;type:decimal(11,4);" json:"balance" form:"balance"`
    	AuthType       string        `gorm:"auth_type" json:"auth_type" form:"auth_type" query:"auth_type"`
    	Addr           string        `gorm:"addr" json:"addr" form:"addr" query:"addr"`
    	AuthAddr       string        `gorm:"auth_addr" json:"auth_addr" form:"auth_addr" query:"auth_addr"`
    	AccessToken    string        `gorm:"-" json:"access_token" form:"access_token" query:"access_token"`
    	ExpiresIn      time.Time     `gorm:"-" json:"expires_in" form:"expires_in" query:"expires_in"`
    	Username       string        `gorm:"username" json:"username" form:"username" query:"username"`
    	Password       string        `gorm:"password" json:"password" form:"password" query:"password"`
    	SystemType     string        `gorm:"system_type" json:"system_type" form:"system_type" query:"system_type"`
    	Throttle       int           `gorm:"throttle" json:"throttle" form:"throttle" query:"throttle"`
    	MaxConnection  int           `gorm:"max_connection" json:"max_connection" form:"max_connection" query:"max_connection"`
    	ReadTimeout    time.Duration `gorm:"read_timeout" json:"read_timeout" form:"read_timeout" query:"read_timeout"`
    	WriteTimeout   time.Duration `gorm:"write_timeout" json:"write_timeout" form:"write_timeout" query:"write_timeout"`
    	RebindDuration time.Duration `gorm:"rebind_duration" json:"rebind_duration" form:"rebind_duration" query:"rebind_duration"`
    }
    func main() {
    	var provider Provider
    	data := []byte(`{"type":{"title":"Smpp"},"name":"test 123","slug":"","alias":"test","description":"","addr":"test:2775","system_type":"","throttle":10,"read_timeout":"20","write_timeout":10,"rebind_duration":10,"auth_type":"basic","username":"test","password":"test","max_connection":1,"provider_type":"smpp"}`)
    	json.Unmarshal(data, &provider)
    	fmt.Println(provider)
    	fmt.Println(provider.ReadTimeout)
    }
    

    For e.g. ReadTimeout is time.Duration and passed string "20". Data is not unmarshalled to struct as intended

    opened by sujit-baniya 6
  • [Bug] - Nil pointer exception

    [Bug] - Nil pointer exception

    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x48721c3]
    
    goroutine 187 [running]:
    github.com/goccy/go-json/internal/encoder/vm.Run(0xc002f276c0, {0xc001d27c00, 0xc001070180, 0x0}, 0x0)
            /Users/sujit/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/vm/vm.go:197 +0x2f5a3
    github.com/goccy/go-json.encodeRunCode(0x4de31c0, {0xc001d27c00, 0x6806d90, 0xc0009bf580}, 0xc00106e000)
            /Users/sujit/go/pkg/mod/github.com/goccy/[email protected]/encode.go:307 +0x68
    github.com/goccy/go-json.encode(0xc002f276c0, {0x4de31c0, 0xc0011cb7d0})
            /Users/sujit/go/pkg/mod/github.com/goccy/[email protected]/encode.go:232 +0x21c
    github.com/goccy/go-json.marshal({0x4de31c0, 0xc0011cb7d0}, {0x0, 0x0, 0x40169aa})
            /Users/sujit/go/pkg/mod/github.com/goccy/[email protected]/encode.go:147 +0xba
    github.com/goccy/go-json.MarshalWithOption(...)
            /Users/sujit/go/pkg/mod/github.com/goccy/[email protected]/json.go:186
    github.com/goccy/go-json.Marshal({0x4de31c0, 0xc0011cb7d0})
            /Users/sujit/go/pkg/mod/github.com/goccy/[email protected]/json.go:171 +0x2a
    
    opened by sujit-baniya 6
  • [Bug] - Error on latest release

    [Bug] - Error on latest release

    I'm getting errors when trying to upgrade to latest version.

    unexpected fault address 0x0
    fatal error: fault
    [signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0xcc587d]
    
    goroutine 222 [running]:
    runtime.throw(0x15833ab, 0x5)
            /usr/local/go/src/runtime/panic.go:1117 +0x72 fp=0xc0020577f0 sp=0xc0020577c0 pc=0x440052
    runtime.sigpanic()
            /usr/local/go/src/runtime/signal_unix.go:741 +0x268 fp=0xc002057828 sp=0xc0020577f0 pc=0x457768
    github.com/goccy/go-json/internal/encoder.AppendString(0xc0003680c0, 0xc000f29400, 0x177, 0x400, 0x663a2264696c6156, 0x74222c7d65736c61, 0xc0003fbde8, 0x0, 0xc000f29400)
            /home/sujit/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/string.go:422 +0xfd fp=0xc002057900 sp=0xc002057828 pc=0xcc587d
    github.com/goccy/go-json/internal/encoder/vm.Run(0xc0003680c0, 0xc000f29400, 0x177, 0x400, 0xc000e92e80, 0x0, 0x0, 0x0, 0x0, 0x0)
            /home/sujit/go/pkg/mod/github.com/goccy/[email protected]/internal/encoder/vm/vm.go:114 +0x20ea fp=0xc0020589c0 sp=0xc002057900 pc=0xcfe8ea
    github.com/goccy/go-json.encodeRunCode(0xc0003680c0, 0xc000f29400, 0x0, 0x400, 0xc000e92e80, 0x10, 0x10, 0xc00204caa0, 0x490eb4, 0x21e5540)
            /home/sujit/go/pkg/mod/github.com/goccy/[email protected]/encode.go:266 +0x1f8 fp=0xc002058a20 sp=0xc0020589c0 pc=0xd58a18
    github.com/goccy/go-json.encode(0xc0003680c0, 0x1364860, 0xc000f3a200, 0xc0009d59e0, 0x10, 0x41667b, 0xc00160f6d0, 0x10)
            /home/sujit/go/pkg/mod/github.com/goccy/[email protected]/encode.go:191 +0x215 fp=0xc002058ab0 sp=0xc002058a20 pc=0xd58635
    github.com/goccy/go-json.marshal(0x1364860, 0xc000f3a200, 0x0, 0x0, 0x0, 0x203000, 0x0, 0x14e3560, 0x10, 0xc00160f6d0)
            /home/sujit/go/pkg/mod/github.com/goccy/[email protected]/encode.go:106 +0xd3 fp=0xc002058b38 sp=0xc002058ab0 pc=0xd58293
    github.com/goccy/go-json.MarshalWithOption(...)
            /home/sujit/go/pkg/mod/github.com/goccy/[email protected]/json.go:168
    github.com/goccy/go-json.Marshal(0x1364860, 0xc000f3a200, 0x80, 0x78, 0x15054e0, 0x10, 0x10)
            /home/sujit/go/pkg/mod/github.com/goccy/[email protected]/json.go:158 +0x46 fp=0xc002058b98 sp=0xc002058b38 pc=0xd58ac6
    github.com/sujit-baniya/verify-rest/pkg/asynq/lib/base.EncodeMessage(...)
            /home/sujit/Sites/projects/sms/verify-rest/pkg/asynq/lib/base/base.go:189
    github.com/sujit-baniya/verify-rest/pkg/asynq/lib/rdb.(*RDB).Enqueue(0xc000162cd0, 0xc000f3a200, 0x60b739da, 0xbc47632d30fc8ad2)
            /home/sujit/Sites/projects/sms/verify-rest/pkg/asynq/lib/rdb/rdb.go:55 +0x3e fp=0xc002058c20 sp=0xc002058b98 pc=0x108bc9e
    github.com/sujit-baniya/verify-rest/pkg/asynq.(*Client).enqueue(0xc000e8a5d0, 0xc000f3a200, 0x0, 0xc025ec56b0fc8ad2, 0x185bec6cc4)
            /home/sujit/Sites/projects/sms/verify-rest/pkg/asynq/client.go:369 +0x86 fp=0xc002058c58 sp=0xc002058c20 pc=0x109ff26
    github.com/sujit-baniya/verify-rest/pkg/asynq.(*Client).Enqueue(0xc000e8a5d0, 0xc00204cef0, 0xc00204cf08, 0x2, 0x2, 0xc001936000, 0xc00204cf10, 0x501925)
            /home/sujit/Sites/projects/sms/verify-rest/pkg/asynq/client.go:344 +0x5b0 fp=0xc002058ea8 sp=0xc002058c58 pc=0x109f7d0
    github.com/sujit-baniya/verify-rest/internal/services.SendSms(0x0, 0xc025ec56b0f998ad, 0x185be97aa3, 0x23a1f40, 0xc0015c4348, 0x14, 0xc000648a08, 0x14, 0xc0015c42e8, 0x14, ...)
            /home/sujit/Sites/projects/sms/verify-rest/internal/services/sms_service.go:128 +0x18b fp=0xc002059150 sp=0xc002058ea8 pc=0x11d170b
    github.com/sujit-baniya/verify-rest/internal/services.SendCampaignPost(0xc00041c780, 0x1, 0xa, 0xc00012b228, 0x2, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
            /home/sujit/Sites/projects/sms/verify-rest/internal/services/sms_service.go:214 +0xb14 fp=0xc002059e90 sp=0xc002059150 pc=0x11d2914
    runtime.goexit()
            /usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc002059e98 sp=0xc002059e90 pc=0x477c41
    created by github.com/sujit-baniya/verify-rest/rest/controllers.SendCampaignPost
            /home/sujit/Sites/projects/sms/verify-rest/rest/controllers/sms_controller.go:150 +0x250
    
    goroutine 1 [chan receive, 1 minutes]:
    github.com/sujit-baniya/verify-rest/config.(*ServerConfig).ServeWithGraceFullShutdown(0xc0000aa990, 0x0, 0x0, 0x0, 0x0, 0x0)
            /home/sujit/Sites/projects/sms/verify-rest/config/server.go:109 +0x208
    github.com/sujit-baniya/verify-rest/cmd.Execute()
            /home/sujit/Sites/projects/sms/verify-rest/cmd/root.go:47 +0x465
    main.main()
            /home/sujit/Sites/projects/sms/verify-rest/main.go:6 +0x25
    
    goroutine 6 [select]:
    github.com/gofiber/storage/memory.(*Storage).gc(0xc000224960)
            /home/sujit/go/pkg/mod/github.com/gofiber/storage/[email protected]/memory.go:104 +0x18d
    created by github.com/gofiber/storage/memory.New
            /home/sujit/go/pkg/mod/github.com/gofiber/storage/[email protected]/memory.go:34 +0xd9
    
    goroutine 7 [select]:
    github.com/gofiber/storage/memory.(*Storage).gc(0xc0003f49c0)
            /home/sujit/go/pkg/mod/github.com/gofiber/storage/[email protected]/memory.go:104 +0x18d
    created by github.com/gofiber/storage/memory.New
            /home/sujit/go/pkg/mod/github.com/gofiber/storage/[email protected]/memory.go:34 +0xd9
    
    goroutine 18 [select]:
    github.com/gofiber/storage/memory.(*Storage).gc(0xc00068e030)
            /home/sujit/go/pkg/mod/github.com/gofiber/storage/[email protected]/memory.go:104 +0x18d
    created by github.com/gofiber/storage/memory.New
            /home/sujit/go/pkg/mod/github.com/gofiber/storage/[email protected]/memory.go:34 +0xd9
    
    goroutine 19 [select, 1 minutes]:
    github.com/go-redis/redis/v8/internal/pool.(*ConnPool).reaper(0xc0006a4000, 0xdf8475800)
            /home/sujit/go/pkg/mod/github.com/go-redis/redis/[email protected]/internal/pool/pool.go:455 +0xd7
    created by github.com/go-redis/redis/v8/internal/pool.NewConnPool
            /home/sujit/go/pkg/mod/github.com/go-redis/redis/[email protected]/internal/pool/pool.go:110 +0x1fc
    
    goroutine 204 [chan receive, 1 minutes]:
    github.com/valyala/fasthttp/stackless.funcWorker(0xc00007e300, 0x177cf28)
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:53 +0x8d
    created by github.com/valyala/fasthttp/stackless.NewFunc.func1
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:30 +0x85
    
    goroutine 203 [chan receive, 1 minutes]:
    github.com/valyala/fasthttp/stackless.funcWorker(0xc00007e300, 0x177cf28)
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:53 +0x8d
    created by github.com/valyala/fasthttp/stackless.NewFunc.func1
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:30 +0x85
    
    goroutine 202 [chan receive, 1 minutes]:
    github.com/valyala/fasthttp/stackless.funcWorker(0xc00007e300, 0x177cf28)
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:53 +0x8d
    created by github.com/valyala/fasthttp/stackless.NewFunc.func1
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:30 +0x85
    
    goroutine 201 [chan receive, 1 minutes]:
    github.com/valyala/fasthttp/stackless.funcWorker(0xc00007e300, 0x177cf28)
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:53 +0x8d
    created by github.com/valyala/fasthttp/stackless.NewFunc.func1
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:30 +0x85
    
    goroutine 200 [chan receive, 1 minutes]:
    github.com/valyala/fasthttp/stackless.funcWorker(0xc00007e300, 0x177cf28)
            /home/sujit/go/pkg/mod/github.com/valyala/[email protected]/stackless/func.go:53 +0x8d
    created by github.com/valyala/fasthttp/stackless.NewFunc.func1... (3 KB left)
    
    opened by sujit-baniya 6
  • Not compatible with encoding/json when using

    Not compatible with encoding/json when using "shadowing" technique for structs

    In my code I rely on an Unmarshalling technique called shadowing, for one specific case: I need to look into outer json field first to be able to parse into correct struct the inner json map, so I do the following:

    // unmarshal Message payload depending on it's app type.
    func (m *Message) UnmarshalJSON(data []byte) error {
    	type MM Message // to prevent infinite recursion on UnmarshalJSON
    	mm := struct {
    		Payload json.RawMessage `json:"payload"`
    		MM                      // it's payload will always be nil cause we shadowed it
    	}{}
    	if err := json.Unmarshal(data, &mm); err != nil {
    		return err
    	}
    ...
        switch m.App {
    	case "payment":
    		payload := &PaymentPayload{}
    		err = json.Unmarshal(mm.Payload, payload)
    		m.Payload = payload
    	case "asset":
    		payload := &Asset{}
    		err = json.Unmarshal(mm.Payload, payload)
    		m.Payload = payload
    	default:
    		var payload interface{}
    		err = json.Unmarshal(mm.Payload, &payload)
    		m.Payload = payload
    	}
    

    As you can see, to parse Message, I delay parsing Message.Payload (using json.RawMessage) by wrapping it into temporary mm struct with embedded Message struct and extracted outer Payload that "shadows" the wrapped MM.Payload field. Using "encoding/json" works perfectly, but go-json yields empty Payload for both outer Payload and wrapped mm struct :(

    opened by kakysha 0
  • Fix stream decoding with error

    Fix stream decoding with error

    fix #296 If error occurred when decoding, decoder progress the cursor position to the end of value.

    opened by goccy 1
  • Different behaviour from encoding/json

    Different behaviour from encoding/json

    When processing a batch of comma-separated json, where one/more are incorrect(error expected):

    • encoding/json - gives error for the incorrect ones, processes the rest
    • go-json - goes into an infinite loop as soon as it is unable to decode an input

    sample program:

    package main
    
    import (
    	stdjson "encoding/json"
    	"fmt"
    	"strings"
    
    	goccy_json "github.com/goccy/go-json"
    )
    
    type Test struct {
    	Val int `json:"val"`
    }
    
    func main() {
    	p := Test{}
    
    	decoder := stdjson.NewDecoder(strings.NewReader("[12,{\"val\" : 2}]"))
    	decoder.Token()
    	for decoder.More() {
    		err := decoder.Decode(&p)
    		fmt.Println("StdJSON:", err)
    		fmt.Println("Value:", p)
    	}
    
    	decoder2 := goccy_json.NewDecoder(strings.NewReader("[12,{\"val\" : 2}]"))
    	decoder2.Token()
    	for decoder2.More() {
    		err := decoder2.Decode(&p)
    		fmt.Println("Goccy JSON:", err)
    		fmt.Println("Value:", p)
    	}
    
    }
    
    • output for encoding/json :
    StdJSON: json: cannot unmarshal number into Go value of type main.Test
    Value: {0}
    StdJSON: <nil>
    Value: {2}
    
    • output for go-json : infinite loop
    bug 
    opened by harjas27 1
  • Support  Custom Name Style

    Support Custom Name Style

    Support custom name style, such as CamelCase ,PascalCase,SnakeCase

    duplicate enhancement 
    opened by shumin1027 0
  • Marshal with case preference

    Marshal with case preference

    Would it be possible to create an option for the encoder to be able to specify the case style the struct fields are marshalled with?

    For example, if we have

    type mystruct struct {
       FieldOne string
       FieldTwo string
    } 
    

    the output will be

    {
      "FieldOne" : "",
      "FieldTwo" : ""
    }
    

    but if we could specify an option to convert to say camelcase (even something as simple as lowercase the first letter) to get

    {
      "fieldOne" : "",
      "fieldTwo" : ""
    }
    

    it would be much nicer than manually specifying json tags! Not sure how it would look in terms of performance. Obviously a general solution is a callback/transformation function of some kind, or could just support camelCase as that would be fast.

    enhancement feature request 
    opened by jhwz 0
  • Question regarding NoEscape reliability

    Question regarding NoEscape reliability

    Initially this feature was the default behavior of go-json. But after careful testing, I found that I passed a large value to json.Marshal() and if the argument could not be assigned to the stack, it could not be properly escaped to the heap (a bug in the Go compiler).

    Therefore, this feature will be provided as an optional until this issue is resolved.

    Do you know the current status of this compiler bug as of Go 1.17, or what data sizes are likely to be affected by the bug? In general, how reliable do you think MarshalNoEscape and UnmarshalNoEscape should be for JSON string or Go value sizes that are, say, under 8 KB? Or is it unpredictable and super variable depending on the system/application?

    Thanks.

    question 
    opened by opalzenith 0
  • RegisterTypeEncoder Support

    RegisterTypeEncoder Support

    We are currently using jsoniter because it allows us to register custom encoders for things like time.Time without having to implement custom types in all our structs across all our apps.

    Is something like this possible with go-json? And if not, could this be considered as a feature?

    Thanks!

    enhancement feature request 
    opened by dolfelt 1
  • Update benchmarks in README to newest values and to include

    Update benchmarks in README to newest values and to include "fastjson"

    Some commits ago "fastjson" got merged as a new benchmark entry but it never to added to the README of the project.

    Since there are continuous improvements and fixes it would also be interesting to see the newest official results.

    enhancement 
    opened by zimmski 0
  • Propose: use an fast int-int map to cache type information

    Propose: use an fast int-int map to cache type information

    Hi, this json package is awesome, the performance is very impressive. I saw the idea of "Dispatch by typeptr from map to slice" is limited by the type slice size and I got an idea to help this.

    Some time ago, I discovered a very fast int key to int value map implemention here: https://github.com/brentp/intintmap/. From my recent benchmark, it shows that a further optimized copy-on-write version of the int-int map can be nearly fast as slice index (without unsafe slice bounds checking elimination), I think it may be a better choice than the current implementation, and it doesn't waste memory.

    cpu: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
    BenchmarkConcurrentStdMapGet_NoLock-12                  72221664                15.79 ns/op
    BenchmarkConcurrentStdMapGet_RWMutex-12                  3255450               369.01 ns/op
    BenchmarkConcurrentSyncMapGet-12                        27300724                44.59 ns/op
    BenchmarkConcurrentCOWMapGet-12                        344179628                 3.487 ns/op
    BenchmarkConcurrentSliceIndex-12                       908571164                 1.213 ns/op
    

    The benchmark code is here: https://github.com/jxskiss/gopkg/blob/master/intintmap/cow_test.go

    Do you think it's a good idea to use the int-int map for the codec cache? I can send a PR if it's welcomed.

    (PS. saying the int-int map, I am not meaning to introduce external dependencies, we can just implement the functionality that we need for the type information cache.)

    enhancement performance 
    opened by jxskiss 7
Releases(v0.8.0)
  • v0.8.0(Dec 2, 2021)

    What's Changed

    • Fix embedded field conflict behavior by @goccy in https://github.com/goccy/go-json/pull/300
    • Refactor compiler for encoder by @goccy in https://github.com/goccy/go-json/pull/301
    • Refactor vm code for encoder by @goccy in https://github.com/goccy/go-json/pull/302

    Full Changelog: https://github.com/goccy/go-json/compare/v0.7.10...v0.8.0

    Source code(tar.gz)
    Source code(zip)
  • v0.7.10(Oct 16, 2021)

    What's Changed

    • Fix conversion from pointer to uint64 by @goccy in https://github.com/goccy/go-json/pull/294

    Full Changelog: https://github.com/goccy/go-json/compare/v0.7.9...v0.7.10

    Source code(tar.gz)
    Source code(zip)
  • v0.7.9(Sep 28, 2021)

  • v0.7.8(Sep 1, 2021)

    • Fix mapassign_faststr for indirect struct type ( #283 )
    • Fix encoding of not empty interface type ( #284 )
    • Fix encoding of empty struct interface type ( #286 )
    Source code(tar.gz)
    Source code(zip)
  • v0.7.7(Aug 25, 2021)

  • v0.7.6(Aug 13, 2021)

  • v0.7.5(Aug 12, 2021)

    • Fix encoding of embedded struct with tags ( #265 )
    • Fix encoding of embedded struct that isn't first field ( #272 )
    • Fix decoding of binary type with escaped char ( #273 )
    Source code(tar.gz)
    Source code(zip)
  • v0.7.4(Jul 6, 2021)

  • v0.7.3(Jun 29, 2021)

  • v0.7.2(Jun 26, 2021)

    Fix decoder

    • Add decoder for func type to fix decoding of nil function value ( #257 )
    • Fix stream decoding of []byte type ( #258 )

    Performance

    • Improve decoding performance of map[string]interface{} type ( use mapassign_faststr ) ( #256 )
    • Improve encoding performance of empty interface type ( remove recursive calling of vm.Run ) ( #259 )

    Benchmark

    • Add bytedance/sonic as benchmark target ( #254 )
    Source code(tar.gz)
    Source code(zip)
  • v0.7.1(Jun 17, 2021)

  • v0.7.0(Jun 12, 2021)

    Support context for MarshalJSON and UnmarshalJSON ( #248 )

    • json.MarshalContext(context.Context, interface{}, ...json.EncodeOption) ([]byte, error)
    • json.NewEncoder(io.Writer).EncodeContext(context.Context, interface{}, ...json.EncodeOption) error
    • json.UnmarshalContext(context.Context, []byte, interface{}, ...json.DecodeOption) error
    • json.NewDecoder(io.Reader).DecodeContext(context.Context, interface{}) error
    type MarshalerContext interface {
      MarshalJSON(context.Context) ([]byte, error)
    }
    
    type UnmarshalerContext interface {
      UnmarshalJSON(context.Context, []byte) error
    }
    

    Add DecodeFieldPriorityFirstWin option ( #242 )

    In the default behavior, go-json, like encoding/json, will reflect the result of the last evaluation when a field with the same name exists. I've added new options to allow you to change this behavior. json.DecodeFieldPriorityFirstWin option reflects the result of the first evaluation if a field with the same name exists. This behavior has a performance advantage as it allows the subsequent strings to be skipped if all fields have been evaluated.

    Fix encoder

    • Fix indent number contains recursive type ( #249 )
    • Fix encoding of using empty interface as map key ( #244 )

    Fix decoder

    • Fix decoding fields containing escaped characters ( #237 )

    Refactor

    • Move some tests to subdirectory ( #243 )
    • Refactor package layout for decoder ( #238 )
    Source code(tar.gz)
    Source code(zip)
  • v0.6.1(Jun 2, 2021)

  • v0.6.0(Jun 1, 2021)

    Support Colorize option for encoding (#233)

    b, err := json.MarshalWithOption(v, json.Colorize(json.DefaultColorScheme))
    if err != nil {
      ...
    }
    fmt.Println(string(b)) // print colored json
    

    screenshot

    Refactor

    • Fix opcode layout - Adjust memory layout of the opcode to 128 bytes in a 64-bit environment ( #230 )
    • Refactor encode option ( #231 )
    • Refactor escape string ( #232 )
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(May 20, 2021)

    Optimization

    • Add type addrShift to enable bigger encoder/decoder cache ( #213 )

    Fix decoder

    • Keep original reference of slice element ( #229 )

    Refactor

    • Refactor Debug mode for encoding ( #226 )
    • Generate VM sources for encoding ( #227 )
    • Refactor validator for null/true/false for decoding ( #221 )
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(May 8, 2021)

    Supports using omitempty and string tags at the same time ( #216 )

    Fix decoder

    • Fix stream decoder for unicode char ( #215 )
    • Fix decoding of slice element ( #219 )
    • Fix calculating of buffer length for stream decoder ( #220 )

    Refactor

    • replace skipWhiteSpace goto by loop ( #212 )
    Source code(tar.gz)
    Source code(zip)
  • v0.4.14(May 3, 2021)

    Benchmark

    • Add valyala/fastjson to benchmark ( #193 )
    • Add benchmark task for CI ( #211 )

    Fix decoder

    • Fix decoding of slice with unmarshal json type ( #198 )
    • Fix decoding of null value for interface type that does not implement Unmarshaler ( #205 )
    • Fix decoding of null value to []byte by json.Unmarshal ( #206 )
    • Fix decoding of backslash char at the end of string ( #207 )
    • Fix stream decoder for null/true/false value ( #208 )
    • Fix stream decoder for slow reader ( #211 )

    Performance

    • If cap of slice is enough, reuse slice data for compatibility with encoding/json ( #200 )
    Source code(tar.gz)
    Source code(zip)
  • v0.4.13(Apr 20, 2021)

    Fix json.Compact and json.Indent

    • Support validation the input buffer for json.Compact and json.Indent ( #189 )
    • Optimize json.Compact and json.Indent ( improve memory footprint ) ( #190 )
    Source code(tar.gz)
    Source code(zip)
  • v0.4.12(Apr 15, 2021)

    Fix encoder

    • Fix unnecessary indent for empty slice type ( #181 )
    • Fix encoding of omitempty feature for the slice or interface type ( #183 )
    • Fix encoding custom types zero values with omitempty when marshaller exists ( #187 )

    Fix decoder

    • Fix decoder for invalid top level value ( #184 )
    • Fix decoder for invalid number value ( #185 )
    Source code(tar.gz)
    Source code(zip)
  • v0.4.11(Apr 2, 2021)

  • v0.4.10(Apr 2, 2021)

  • v0.4.9(Mar 29, 2021)

    Add debug mode

    If you use json.MarshalWithOption(v, json.Debug()) and panic occurred in go-json, produces debug information to console.

    Support a new feature for compatible with encoding/json

    • invalid UTF-8 is coerced to valid UTF-8 ( without performance down )

    Fix encoder

    • Fixed handling of MarshalJSON of function type

    Fix decoding of slice of pointer type

    If there is a pointer value, go-json will use it. (This behavior is necessary to achieve the ability to prioritize pre-filled values). However, since slices are reused internally, there was a bug that referred to the previous pointer value. Therefore, it is not necessary to refer to the pointer value in advance for the slice element, so we explicitly initialize slice element by nil.

    Source code(tar.gz)
    Source code(zip)
  • v0.4.8(Mar 20, 2021)

    Reduce memory usage at compile time

    • go-json have used about 2GB of memory at compile time, but now it can compile with about less than 550MB.

    Fix any encoder's bug

    • Add many test cases for encoder
    • Fix composite type ( slice/array/map )
    • Fix pointer types
    • Fix encoding of MarshalJSON or MarshalText or json.Number type

    Refactor encoder

    • Change package layout for reducing memory usage at compile
    • Remove anonymous and only operation
    • Remove root property from encodeCompileContext and opcode

    Fix CI

    • Add Go 1.16
    • Remove Go 1.13
    • Fix make cover task

    Number/Delim/Token/RawMessage use the types defined in encoding/json by type alias

    Source code(tar.gz)
    Source code(zip)
  • v0.4.7(Feb 22, 2021)

    Fix decoder

    • Fix decoding of deep recursive structure
    • Fix decoding of embedded unexported pointer field
    • Fix invalid test case
    • Fix decoding of invalid value
    • Fix decoding of prefilled value
    • Fix not being able to return UnmarshalTypeError when it should be returned
    • Fix decoding of null value
    • Fix decoding of type of null string
    • Use pre allocated pointer if exists it at decoding

    Reduce memory usage at compile

    • Integrate int/int8/int16/int32/int64 and uint/uint8/uint16/uint32/uint64 operation to reduce memory usage at compile

    Remove unnecessary optype

    Source code(tar.gz)
    Source code(zip)
Owner
Masaaki Goshima
Masaaki Goshima
goi - The “Quite OK Image” format encoder / decoder for Go.

goi - The “Quite OK Image” format encoder / decoder for Go. QOI - The “Quite OK Image” - is losslessly image format that offering speedup both compres

neguse 6 Nov 27, 2021
A high-performance 100% compatible drop-in replacement of "encoding/json"

A high-performance 100% compatible drop-in replacement of "encoding/json" You can also use thrift like JSON using thrift-iterator Benchmark Source cod

Jsoniter 10.2k Dec 2, 2021
Fast and flexible JSON encoder for Go

Jettison Jettison is a fast and flexible JSON encoder for the Go programming language, inspired by bet365/jingo, with a richer features set, aiming at

William Poussier 112 Nov 30, 2021
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

High Performance, Kubernetes Native Object Storage 2 Aug 9, 2021
Fork of Go's standard library json encoder

A fork of the Go standard library's json encoder Why? https://github.com/golang/go/issues/6213 was proposed in 2013 but was never accepted. Difference

unchain.io 0 Nov 25, 2021
An efficient JSON decoder

pkg/json An alternative JSON decoder for Go. Features pkg/json aims to be a drop in replacement for encoding/json. It features: json.Scanner which, wh

null 282 Nov 27, 2021
Golang JSON decoder supporting case-sensitive, number-preserving, and strict decoding use cases

Golang JSON decoder supporting case-sensitive, number-preserving, and strict decoding use cases

Kubernetes SIGs 8 Nov 22, 2021
json encoding and decoding

jx Package jx implements encoding and decoding of json [RFC 7159]. Lightweight fork of jsoniter. go get github.com/go-faster/jx Usage and examples Roa

go faster 9 Nov 25, 2021
JSON:API compatible query string parser

QParser The package helps to parse part of the URL path and its parameters string to a handy structure. The structure format is compatible with the JS

Velmie 1 Oct 26, 2021
Fast JSON parser and validator for Go. No custom structs, no code generation, no reflection

fastjson - fast JSON parser and validator for Go Features Fast. As usual, up to 15x faster than the standard encoding/json. See benchmarks. Parses arb

Aliaksandr Valialkin 1.4k Dec 3, 2021
Fast JSON serializer for golang.

easyjson Package easyjson provides a fast and easy way to marshal/unmarshal Go structs to/from JSON without the use of reflection. In performance test

Free and open source software developed at Mail.Ru 3.5k Nov 29, 2021
Fast Color JSON Marshaller + Pretty Printer for Golang

ColorJSON: The Fast Color JSON Marshaller for Go What is this? This package is based heavily on hokaccha/go-prettyjson but has some noticible differen

Tyler Brock 91 Nov 22, 2021
A fast json parser for go

rjson rjson is a json parser that relies on Ragel-generated state machines for most parsing. rjson's api is minimal and focussed on efficient parsing.

WillAbides 47 Nov 15, 2021
A blazingly fast JSON serializing & deserializing library

Sonic A blazingly fast JSON serializing & deserializing library, accelerated by JIT(just-in-time compiling) and SIMD(single-instruction-multi-data). B

Bytedance Inc. 1.6k Dec 1, 2021
Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang

kazaam Description Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang. This functionality provides

Qntfy 205 Sep 17, 2021
Fast json for go. Lightweight fork of jsoniter.

jx Fast json for go. Lightweight fork of jsoniter. Features Reduced scope (no reflection or encoding/json adapter) Fuzzing, improved test coverage Dra

ogen 7 Nov 23, 2021
Get JSON values quickly - JSON parser for Go

get json values quickly GJSON is a Go package that provides a fast and simple way to get values from a json document. It has features such as one line

Josh Baker 9.4k Dec 5, 2021
JSON diff library for Go based on RFC6902 (JSON Patch)

jsondiff jsondiff is a Go package for computing the diff between two JSON documents as a series of RFC6902 (JSON Patch) operations, which is particula

William Poussier 138 Nov 27, 2021
Abstract JSON for golang with JSONPath support

Abstract JSON Abstract JSON is a small golang package provides a parser for JSON with support of JSONPath, in case when you are not sure in its struct

Stepan Pyzhov 86 Dec 2, 2021