An efficient JSON decoder

Overview

pkg/json GoDoc

An alternative JSON decoder for Go.

Features

pkg/json aims to be a drop in replacement for encoding/json. It features:

  • json.Scanner which, when provided an external buffer, does not allocate.
  • io.Reader friendly; you don't need to buffer your input in memory.
  • json.Decoder.Token() replacement that is 2-3x faster than encoding/json.
  • json.Decoder.NextToken() is almost allocation free.

Is it faster than fastjson/ultrajson/megajson/fujson?

Honestly, I don't know. I have some benchmarks that show that pkg/json is faster than encoding/json for tokenisation, but this package isn't finished yet.

The Decoder.Token API is between 2-3x faster than encoding/json.Decoder.Token:

BenchmarkDecoderToken/pkgjson/canada.json.gz-16                       51          21387427 ns/op         105.25 MB/s     4402975 B/op     222279 allocs/op
BenchmarkDecoderToken/encodingjson/canada.json.gz-16                  18          63962471 ns/op          35.19 MB/s    17740379 B/op     889106 allocs/op
BenchmarkDecoderToken/pkgjson/citm_catalog.json.gz-16                235           5110849 ns/op         337.95 MB/s      966008 B/op      81995 allocs/op
BenchmarkDecoderToken/encodingjson/citm_catalog.json.gz-16            61          19635188 ns/op          87.96 MB/s     5665660 B/op     324799 allocs/op
BenchmarkDecoderToken/pkgjson/twitter.json.gz-16                     495           2424694 ns/op         260.45 MB/s      768370 B/op      38992 allocs/op
BenchmarkDecoderToken/encodingjson/twitter.json.gz-16                 98          12028296 ns/op          52.50 MB/s     3660269 B/op     187815 allocs/op
BenchmarkDecoderToken/pkgjson/code.json.gz-16                         75          16111648 ns/op         120.44 MB/s     4304252 B/op     320235 allocs/op
BenchmarkDecoderToken/encodingjson/code.json.gz-16                    15          75464079 ns/op          25.71 MB/s    23355964 B/op    1319125 allocs/op
BenchmarkDecoderToken/pkgjson/example.json.gz-16                   23210             51699 ns/op         251.92 MB/s       16032 B/op        914 allocs/op
BenchmarkDecoderToken/encodingjson/example.json.gz-16               4767            257169 ns/op          50.64 MB/s       82416 B/op       4325 allocs/op
BenchmarkDecoderToken/pkgjson/sample.json.gz-16                     1411            849803 ns/op         809.00 MB/s      213776 B/op       5081 allocs/op
BenchmarkDecoderToken/encodingjson/sample.json.gz-16                 345           3550453 ns/op         193.63 MB/s      759686 B/op      26643 allocs/op

The alternative Decoder.NextToken API is between 8-10x faster than encoding/json.Decoder.Token while producing virtually no allocations:

BenchmarkDecoderNextToken/pkgjson/canada.json.gz-16                  222           5157299 ns/op         436.48 MB/s         152 B/op          3 allocs/op
BenchmarkDecoderNextToken/encodingjson/canada.json.gz-16              18          67566098 ns/op          33.32 MB/s    17740510 B/op     889106 allocs/op
BenchmarkDecoderNextToken/pkgjson/citm_catalog.json.gz-16            510           2260514 ns/op         764.08 MB/s         152 B/op          3 allocs/op
BenchmarkDecoderNextToken/encodingjson/citm_catalog.json.gz-16        61          20026083 ns/op          86.25 MB/s     5665620 B/op     324799 allocs/op
BenchmarkDecoderNextToken/pkgjson/twitter.json.gz-16                1032           1093466 ns/op         577.53 MB/s         168 B/op          4 allocs/op
BenchmarkDecoderNextToken/encodingjson/twitter.json.gz-16             93          12548028 ns/op          50.33 MB/s     3660291 B/op     187815 allocs/op
BenchmarkDecoderNextToken/pkgjson/code.json.gz-16                    211           5322161 ns/op         364.60 MB/s         264 B/op          6 allocs/op
BenchmarkDecoderNextToken/encodingjson/code.json.gz-16                14          78195551 ns/op          24.82 MB/s    23355961 B/op    1319125 allocs/op
BenchmarkDecoderNextToken/pkgjson/example.json.gz-16               49245             22873 ns/op         569.41 MB/s         168 B/op          4 allocs/op
BenchmarkDecoderNextToken/encodingjson/example.json.gz-16           4500            269313 ns/op          48.36 MB/s       82416 B/op       4325 allocs/op
BenchmarkDecoderNextToken/pkgjson/sample.json.gz-16                 2032            553729 ns/op        1241.57 MB/s        1160 B/op          9 allocs/op
BenchmarkDecoderNextToken/encodingjson/sample.json.gz-16             332           3574484 ns/op         192.33 MB/s      759685 B/op      26643 allocs/op

Decoding into an interface{} this package is 40-50% faster than encoding/json:

BenchmarkDecoderDecodeInterfaceAny/pkgjson/canada.json.gz-16                  43          27181309 ns/op          82.82 MB/s     8747225 B/op     281408 allocs/op
BenchmarkDecoderDecodeInterfaceAny/encodingjson/canada.json.gz-16             30          36930290 ns/op          60.95 MB/s    20647699 B/op     392553 allocs/op
BenchmarkDecoderDecodeInterfaceAny/pkgjson/citm_catalog.json.gz-16           156           7635508 ns/op         226.21 MB/s     5197871 B/op      89673 allocs/op
BenchmarkDecoderDecodeInterfaceAny/encodingjson/citm_catalog.json.gz-16       73          15766855 ns/op         109.55 MB/s     9410808 B/op      95496 allocs/op
BenchmarkDecoderDecodeInterfaceAny/pkgjson/twitter.json.gz-16                327           3664618 ns/op         172.33 MB/s     2130822 B/op      30182 allocs/op
BenchmarkDecoderDecodeInterfaceAny/encodingjson/twitter.json.gz-16           175           6800730 ns/op          92.86 MB/s     4359944 B/op      31775 allocs/op
BenchmarkDecoderDecodeInterfaceAny/pkgjson/code.json.gz-16                    64          17302914 ns/op         112.15 MB/s     7331655 B/op     232059 allocs/op
BenchmarkDecoderDecodeInterfaceAny/encodingjson/code.json.gz-16               49          23759564 ns/op          81.67 MB/s    12332748 B/op     271292 allocs/op
BenchmarkDecoderDecodeInterfaceAny/pkgjson/example.json.gz-16              15920             76561 ns/op         170.11 MB/s       50988 B/op        739 allocs/op
BenchmarkDecoderDecodeInterfaceAny/encodingjson/example.json.gz-16          8670            139230 ns/op          93.54 MB/s       82840 B/op        782 allocs/op
BenchmarkDecoderDecodeInterfaceAny/pkgjson/sample.json.gz-16                1084           1119477 ns/op         614.12 MB/s      399983 B/op       5542 allocs/op
BenchmarkDecoderDecodeInterfaceAny/encodingjson/sample.json.gz-16            217           5418480 ns/op         126.88 MB/s     2697781 B/op       8075 allocs/op

Should I use this?

Right now? No. In the future, maybe.

I've found a bug!

Great! Please follow this two step process:

  1. Raise an issue
  2. Send a PR (unless I fix it first)

Standing on the shoulders of giants

This project is heavily influenced by Steven Schveighoffer's iopipe and Phil Pearl.

Issues
  • Faster whitespace detection

    Faster whitespace detection

    Lookup via a [256]bool table is cache inefficient and for most inputs which have at least one whitespace character not as fast as a brute force comparision.

    The degredation in the canada example is measurable, but this is because canada.json has little whitespace and is dominated by number parsing. I have some plans to improve the performance of parseNumber so perhaps this change is worthwhile.

    name                             old time/op    new time/op    delta
    Scanner/canada.json.gz-16          3.56ms ± 1%    3.68ms ± 0%   +3.20%  (p=0.008 n=5+5)
    Scanner/citm_catalog.json.gz-16    1.57ms ± 1%    1.48ms ± 1%   -5.70%  (p=0.008 n=5+5)
    Scanner/twitter.json.gz-16          849µs ± 1%     823µs ± 0%   -3.00%  (p=0.008 n=5+5)
    Scanner/code.json.gz-16            3.69ms ± 1%    3.76ms ± 1%   +1.70%  (p=0.008 n=5+5)
    Scanner/example.json.gz-16         16.8µs ± 1%    16.3µs ± 0%   -3.11%  (p=0.008 n=5+5)
    Scanner/sample.json.gz-16           476µs ± 0%     418µs ± 1%  -12.32%  (p=0.008 n=5+5)
    
    name                             old speed      new speed      delta
    Scanner/canada.json.gz-16         632MB/s ± 1%   612MB/s ± 0%   -3.10%  (p=0.008 n=5+5)
    Scanner/citm_catalog.json.gz-16  1.10GB/s ± 1%  1.16GB/s ± 1%   +6.05%  (p=0.008 n=5+5)
    Scanner/twitter.json.gz-16        744MB/s ± 1%   767MB/s ± 0%   +3.10%  (p=0.008 n=5+5)
    Scanner/code.json.gz-16           525MB/s ± 1%   517MB/s ± 1%   -1.67%  (p=0.008 n=5+5)
    Scanner/example.json.gz-16        775MB/s ± 1%   799MB/s ± 0%   +3.21%  (p=0.008 n=5+5)
    Scanner/sample.json.gz-16        1.44GB/s ± 0%  1.65GB/s ± 1%  +14.06%  (p=0.008 n=5+5)
    
    name                             old alloc/op   new alloc/op   delta
    Scanner/canada.json.gz-16           0.00B          0.00B          ~     (all equal)
    Scanner/citm_catalog.json.gz-16     0.00B          0.00B          ~     (all equal)
    Scanner/twitter.json.gz-16          0.00B          0.00B          ~     (all equal)
    Scanner/code.json.gz-16             0.00B          0.00B          ~     (all equal)
    Scanner/example.json.gz-16          0.00B          0.00B          ~     (all equal)
    Scanner/sample.json.gz-16           0.00B          0.00B          ~     (all equal)
    
    name                             old allocs/op  new allocs/op  delta
    Scanner/canada.json.gz-16            0.00           0.00          ~     (all equal)
    Scanner/citm_catalog.json.gz-16      0.00           0.00          ~     (all equal)
    Scanner/twitter.json.gz-16           0.00           0.00          ~     (all equal)
    Scanner/code.json.gz-16              0.00           0.00          ~     (all equal)
    Scanner/example.json.gz-16           0.00           0.00          ~     (all equal)
    Scanner/sample.json.gz-16            0.00           0.00          ~     (all equal)
    
    opened by davecheney 1
  • Improvements to number parsing

    Improvements to number parsing

    benchstat {old,new}.txt
    name                                     old time/op    new time/op    delta
    DecoderNextToken/pkgjson/canada-4          7.25ms ± 0%    7.09ms ± 0%  -2.17%  (p=0.008 n=5+5)
    DecoderNextToken/pkgjson/citm_catalog-4    3.11ms ± 1%    3.01ms ± 1%  -3.23%  (p=0.008 n=5+5)
    DecoderNextToken/pkgjson/twitter-4         1.48ms ± 1%    1.48ms ± 1%    ~     (p=0.690 n=5+5)
    DecoderNextToken/pkgjson/code-4            7.30ms ± 1%    7.26ms ± 1%    ~     (p=0.222 n=5+5)
    DecoderNextToken/pkgjson/example-4         30.9µs ± 0%    31.3µs ± 1%  +1.22%  (p=0.008 n=5+5)
    DecoderNextToken/pkgjson/sample-4           666µs ± 1%     660µs ± 0%  -0.98%  (p=0.008 n=5+5)
    
    name                                     old speed      new speed      delta
    DecoderNextToken/pkgjson/canada-4         311MB/s ± 0%   317MB/s ± 0%  +2.21%  (p=0.008 n=5+5)
    DecoderNextToken/pkgjson/citm_catalog-4   556MB/s ± 1%   575MB/s ± 1%  +3.33%  (p=0.008 n=5+5)
    DecoderNextToken/pkgjson/twitter-4        428MB/s ± 1%   427MB/s ± 1%    ~     (p=0.643 n=5+5)
    DecoderNextToken/pkgjson/code-4           266MB/s ± 1%   267MB/s ± 1%    ~     (p=0.222 n=5+5)
    DecoderNextToken/pkgjson/example-4        421MB/s ± 0%   416MB/s ± 1%  -1.20%  (p=0.008 n=5+5)
    DecoderNextToken/pkgjson/sample-4        1.03GB/s ± 1%  1.04GB/s ± 0%  +0.99%  (p=0.008 n=5+5)
    
    opened by davecheney 0
  • Speed up scanning by 7-10%

    Speed up scanning by 7-10%

    Small improvements to overall scanning performance

    name                                 old time/op    new time/op    delta
    Scanner/canada-4                       6.16ms ± 0%    5.51ms ± 0%  -10.64%  (p=0.008 n=5+5)
    Scanner/citm_catalog-4                 2.48ms ± 1%    2.30ms ± 1%   -7.29%  (p=0.008 n=5+5)
    Scanner/twitter-4                      1.25ms ± 1%    1.15ms ± 1%   -7.83%  (p=0.008 n=5+5)
    Scanner/code-4                         5.57ms ± 1%    5.19ms ± 1%   -6.75%  (p=0.008 n=5+5)
    Scanner/example-4                      25.0µs ± 1%    22.8µs ± 0%   -9.08%  (p=0.008 n=5+5)
    Scanner/sample-4                        693µs ± 1%     619µs ± 2%  -10.62%  (p=0.008 n=5+5)
    DecoderToken/pkgjson/canada-4          29.4ms ± 0%    29.3ms ± 0%   -0.57%  (p=0.016 n=5+5)
    DecoderToken/pkgjson/citm_catalog-4    7.37ms ± 4%    7.16ms ± 2%     ~     (p=0.056 n=5+5)
    DecoderToken/pkgjson/twitter-4         3.37ms ± 1%    3.30ms ± 1%   -1.99%  (p=0.008 n=5+5)
    DecoderToken/pkgjson/code-4            23.2ms ± 2%    22.7ms ± 1%   -2.29%  (p=0.008 n=5+5)
    DecoderToken/pkgjson/example-4         73.5µs ± 0%    72.1µs ± 1%   -1.87%  (p=0.008 n=5+5)
    DecoderToken/pkgjson/sample-4          1.14ms ± 1%    1.06ms ± 1%   -7.16%  (p=0.008 n=5+5)
    
    name                                 old speed      new speed      delta
    Scanner/canada-4                      365MB/s ± 0%   409MB/s ± 0%  +11.91%  (p=0.008 n=5+5)
    Scanner/citm_catalog-4                697MB/s ± 1%   752MB/s ± 1%   +7.87%  (p=0.008 n=5+5)
    Scanner/twitter-4                     505MB/s ± 1%   548MB/s ± 1%   +8.49%  (p=0.008 n=5+5)
    Scanner/code-4                        349MB/s ± 1%   374MB/s ± 1%   +7.23%  (p=0.008 n=5+5)
    Scanner/example-4                     520MB/s ± 1%   572MB/s ± 0%   +9.98%  (p=0.008 n=5+5)
    Scanner/sample-4                      993MB/s ± 1%  1111MB/s ± 2%  +11.89%  (p=0.008 n=5+5)
    DecoderToken/pkgjson/canada-4        76.5MB/s ± 0%  76.9MB/s ± 0%   +0.56%  (p=0.016 n=5+5)
    DecoderToken/pkgjson/citm_catalog-4   234MB/s ± 4%   241MB/s ± 2%     ~     (p=0.056 n=5+5)
    DecoderToken/pkgjson/twitter-4        188MB/s ± 1%   191MB/s ± 1%   +2.03%  (p=0.008 n=5+5)
    DecoderToken/pkgjson/code-4          83.5MB/s ± 2%  85.4MB/s ± 1%   +2.34%  (p=0.008 n=5+5)
    DecoderToken/pkgjson/example-4        177MB/s ± 0%   181MB/s ± 1%   +1.91%  (p=0.008 n=5+5)
    DecoderToken/pkgjson/sample-4         602MB/s ± 1%   649MB/s ± 1%   +7.71%  (p=0.008 n=5+5)
    
    opened by davecheney 0
  • Demote parseString to a function

    Demote parseString to a function

    After the byteReader change landed the only piece of state parseString needed from Scanner was pos, which can always be inferred from the return value, length.

    name                             old time/op    new time/op    delta
    Scanner/canada.json.gz-16          3.48ms ± 1%    3.48ms ± 0%    ~     (p=1.000 n=5+5)
    Scanner/citm_catalog.json.gz-16    1.56ms ± 1%    1.54ms ± 1%    ~     (p=0.095 n=5+5)
    Scanner/twitter.json.gz-16          831µs ± 1%     822µs ± 1%  -1.14%  (p=0.008 n=5+5)
    Scanner/code.json.gz-16            3.64ms ± 0%    3.60ms ± 1%  -1.13%  (p=0.032 n=5+5)
    Scanner/example.json.gz-16         16.5µs ± 1%    16.4µs ± 1%    ~     (p=1.000 n=5+5)
    Scanner/sample.json.gz-16           469µs ± 1%     467µs ± 3%    ~     (p=0.151 n=5+5)
    
    name                             old speed      new speed      delta
    Scanner/canada.json.gz-16         648MB/s ± 1%   646MB/s ± 0%    ~     (p=1.000 n=5+5)
    Scanner/citm_catalog.json.gz-16  1.11GB/s ± 1%  1.12GB/s ± 1%    ~     (p=0.095 n=5+5)
    Scanner/twitter.json.gz-16        760MB/s ± 1%   769MB/s ± 1%  +1.15%  (p=0.008 n=5+5)
    Scanner/code.json.gz-16           533MB/s ± 0%   540MB/s ± 1%  +1.15%  (p=0.032 n=5+5)
    Scanner/example.json.gz-16        790MB/s ± 1%   792MB/s ± 1%    ~     (p=1.000 n=5+5)
    Scanner/sample.json.gz-16        1.47GB/s ± 1%  1.47GB/s ± 2%    ~     (p=0.151 n=5+5)
    
    name                             old alloc/op   new alloc/op   delta
    Scanner/canada.json.gz-16           0.00B          0.00B         ~     (all equal)
    Scanner/citm_catalog.json.gz-16     0.00B          0.00B         ~     (all equal)
    Scanner/twitter.json.gz-16          0.00B          0.00B         ~     (all equal)
    Scanner/code.json.gz-16             0.00B          0.00B         ~     (all equal)
    Scanner/example.json.gz-16          0.00B          0.00B         ~     (all equal)
    Scanner/sample.json.gz-16           0.00B          0.00B         ~     (all equal)
    
    name                             old allocs/op  new allocs/op  delta
    Scanner/canada.json.gz-16            0.00           0.00         ~     (all equal)
    Scanner/citm_catalog.json.gz-16      0.00           0.00         ~     (all equal)
    Scanner/twitter.json.gz-16           0.00           0.00         ~     (all equal)
    Scanner/code.json.gz-16              0.00           0.00         ~     (all equal)
    Scanner/example.json.gz-16           0.00           0.00         ~     (all equal)
    Scanner/sample.json.gz-16            0.00           0.00         ~     (all equal)
    
    opened by davecheney 0
  • Make validateToken a function

    Make validateToken a function

    Now the byteReader change is landed, validateToken doesn't need access to the scanner any more. Rewrite it to take a *byteReader as a faux receiver. It turns out this is ~1% faster overall, and I prefer the purity of moving this from a function to a method. If the trend holds true for parseString and parseNumber, this will make it easier to refactor the latter.

    name                             old time/op    new time/op    delta
    Scanner/canada.json.gz-16          3.56ms ± 1%    3.50ms ± 0%  -1.65%  (p=0.008 n=5+5)
    Scanner/citm_catalog.json.gz-16    1.57ms ± 1%    1.56ms ± 1%  -1.02%  (p=0.008 n=5+5)
    Scanner/twitter.json.gz-16          849µs ± 1%     833µs ± 1%  -1.85%  (p=0.008 n=5+5)
    Scanner/code.json.gz-16            3.69ms ± 1%    3.62ms ± 1%  -1.96%  (p=0.008 n=5+5)
    Scanner/example.json.gz-16         16.8µs ± 1%    16.6µs ± 1%  -1.25%  (p=0.016 n=5+5)
    Scanner/sample.json.gz-16           476µs ± 0%     468µs ± 0%  -1.69%  (p=0.008 n=5+5)
    
    name                             old speed      new speed      delta
    Scanner/canada.json.gz-16         632MB/s ± 1%   642MB/s ± 0%  +1.68%  (p=0.008 n=5+5)
    Scanner/citm_catalog.json.gz-16  1.10GB/s ± 1%  1.11GB/s ± 1%  +1.02%  (p=0.008 n=5+5)
    Scanner/twitter.json.gz-16        744MB/s ± 1%   758MB/s ± 1%  +1.89%  (p=0.008 n=5+5)
    Scanner/code.json.gz-16           525MB/s ± 1%   536MB/s ± 1%  +2.00%  (p=0.008 n=5+5)
    Scanner/example.json.gz-16        775MB/s ± 1%   784MB/s ± 1%  +1.26%  (p=0.016 n=5+5)
    Scanner/sample.json.gz-16        1.44GB/s ± 0%  1.47GB/s ± 0%  +1.72%  (p=0.008 n=5+5)
    
    name                             old alloc/op   new alloc/op   delta
    Scanner/canada.json.gz-16           0.00B          0.00B         ~     (all equal)
    Scanner/citm_catalog.json.gz-16     0.00B          0.00B         ~     (all equal)
    Scanner/twitter.json.gz-16          0.00B          0.00B         ~     (all equal)
    Scanner/code.json.gz-16             0.00B          0.00B         ~     (all equal)
    Scanner/example.json.gz-16          0.00B          0.00B         ~     (all equal)
    Scanner/sample.json.gz-16           0.00B          0.00B         ~     (all equal)
    
    name                             old allocs/op  new allocs/op  delta
    Scanner/canada.json.gz-16            0.00           0.00         ~     (all equal)
    Scanner/citm_catalog.json.gz-16      0.00           0.00         ~     (all equal)
    Scanner/twitter.json.gz-16           0.00           0.00         ~     (all equal)
    Scanner/code.json.gz-16              0.00           0.00         ~     (all equal)
    Scanner/example.json.gz-16           0.00           0.00         ~     (all equal)
    Scanner/sample.json.gz-16            0.00           0.00         ~     (all equal)
    
    opened by davecheney 0
  • Rename buffer to byteReader

    Rename buffer to byteReader

    This is a large change intended to reduce the surface area of the byteReader (nee buffer) type. The original design came from iopipe where the buffer type was an abstract implementation and the reader or scanner built on top of it to extend the buffer then fill it from a reader. In this implementation this separation of concerns didn't make sense as there would only be one buffer impl, and it made more sense that the byteReader knew how to refill itself, thereby removing several helper methods like avail and remaining, bringing the design close to the original iopipe API.

    Overall performance is unaffected as extend only happens when the buffer is exhausted, profiling shows this to be sub 1%.

    benchstat {old,new}.txt
    name                             old time/op    new time/op    delta
    Scanner/canada.json.gz-16          3.60ms ± 2%    3.56ms ± 0%    ~     (p=0.690 n=5+5)
    Scanner/citm_catalog.json.gz-16    1.63ms ± 1%    1.58ms ± 0%  -3.09%  (p=0.016 n=5+4)
    Scanner/twitter.json.gz-16          837µs ± 1%     846µs ± 0%  +1.06%  (p=0.008 n=5+5)
    Scanner/code.json.gz-16            3.69ms ± 1%    3.72ms ± 2%    ~     (p=0.421 n=5+5)
    Scanner/example.json.gz-16         16.7µs ± 1%    16.7µs ± 1%    ~     (p=1.000 n=5+5)
    Scanner/sample.json.gz-16           504µs ± 0%     477µs ± 0%  -5.39%  (p=0.008 n=5+5)
    
    name                             old speed      new speed      delta
    Scanner/canada.json.gz-16         625MB/s ± 2%   632MB/s ± 0%    ~     (p=0.690 n=5+5)
    Scanner/citm_catalog.json.gz-16  1.06GB/s ± 1%  1.10GB/s ± 0%  +3.19%  (p=0.016 n=5+4)
    Scanner/twitter.json.gz-16        755MB/s ± 1%   747MB/s ± 0%  -1.05%  (p=0.008 n=5+5)
    Scanner/code.json.gz-16           526MB/s ± 1%   522MB/s ± 2%    ~     (p=0.421 n=5+5)
    Scanner/example.json.gz-16        779MB/s ± 1%   779MB/s ± 1%    ~     (p=1.000 n=5+5)
    Scanner/sample.json.gz-16        1.36GB/s ± 0%  1.44GB/s ± 0%  +5.70%  (p=0.008 n=5+5)
    
    name                             old alloc/op   new alloc/op   delta
    Scanner/canada.json.gz-16           0.00B          0.00B         ~     (all equal)
    Scanner/citm_catalog.json.gz-16     0.00B          0.00B         ~     (all equal)
    Scanner/twitter.json.gz-16          0.00B          0.00B         ~     (all equal)
    Scanner/code.json.gz-16             0.00B          0.00B         ~     (all equal)
    Scanner/example.json.gz-16          0.00B          0.00B         ~     (all equal)
    Scanner/sample.json.gz-16           0.00B          0.00B         ~     (all equal)
    
    name                             old allocs/op  new allocs/op  delta
    Scanner/canada.json.gz-16            0.00           0.00         ~     (all equal)
    Scanner/citm_catalog.json.gz-16      0.00           0.00         ~     (all equal)
    Scanner/twitter.json.gz-16           0.00           0.00         ~     (all equal)
    Scanner/code.json.gz-16              0.00           0.00         ~     (all equal)
    Scanner/example.json.gz-16           0.00           0.00         ~     (all equal)
    Scanner/sample.json.gz-16            0.00           0.00         ~     (all equal)
    
    opened by davecheney 0
  • support for decoding struct types

    support for decoding struct types

    It seems the current version of pkg/json does not yet support decoding into struct tyoes. As pkg/josn is still in development, the issue is probably known. This issue is meant to track the progress of implementing support for decoding into struct types.

    $ go test
    --- FAIL: TestDecodeIntoStruct (0.00s)
        foo_test.go:29: unable to decode into struct; decodeValue: unhandled type: struct
    FAIL
    exit status 1
    FAIL	github.com/pkg/json	0.276s
    

    The following test case passes when using encoding/json but fails with unable to decode into struct; decodeValue: unhandled type: struct when using github.com/pkg/json.

    package json
    
    import (
    	//"encoding/json"
    	"reflect"
    	"strings"
    	"testing"
    )
    
    func TestDecodeIntoStruct(t *testing.T) {
    	type Foo struct {
    		X int
    		Y int
    	}
    	golden := []struct {
    		want Foo
    		in   string
    	}{
    		{
    			want: Foo{X: 1, Y: 2},
    			in:   `{"X": 1, "Y": 2}`,
    		},
    	}
    	for _, g := range golden {
    		r := strings.NewReader(g.in)
    		dec := NewDecoder(r)
    		got := Foo{}
    		if err := dec.Decode(&got); err != nil {
    			t.Errorf("unable to decode into struct; %v", err)
    			continue
    		}
    		if g.want != got {
    			t.Errorf("decoded value mismatch; expected %#v, got %#v", g.want, got)
    		}
    	}
    }
    
    func TestDecodeIntoMap(t *testing.T) {
    	type Foo map[string]int
    	golden := []struct {
    		want Foo
    		in   string
    	}{
    		{
    			want: Foo{"X": 1, "Y": 2},
    			in:   `{"X": 1, "Y": 2}`,
    		},
    	}
    	for _, g := range golden {
    		r := strings.NewReader(g.in)
    		dec := NewDecoder(r)
    		got := Foo{}
    		if err := dec.Decode(&got); err != nil {
    			t.Errorf("unable to decode into struct; %v", err)
    			continue
    		}
    		if !reflect.DeepEqual(g.want, got) {
    			t.Errorf("decoded value mismatch; expected %#v, got %#v", g.want, got)
    		}
    	}
    }
    

    Cheers, Robin

    opened by mewmew 0
  • encoding/json compat: missing .More() method on Decoder

    encoding/json compat: missing .More() method on Decoder

    When re-encoding a stream back to text, the encoding/json library uses the .More() method to tell if it needs to place a comma.

    You can peek ahead to the next token to see if you need to place a comma, so it's still possible to work. It just means that pkg/json is not a drop in, API compatible replacement.

    opened by superchalupa 0
  • BenchmarkParseNumber in scanner_test.go

    BenchmarkParseNumber in scanner_test.go

    func BenchmarkParseNumber(b *testing.B) {

    Hi Dave I think a small change should be done for this func() To see an issue quickly just run this line go test -bench=.Parse.

    `

    panic: runtime error: index out of range [0] with length 0

    goroutine 290 [running]: github.com/pkg/json.BenchmarkParseNumber.func1(0xc000194240) C:/Users/VE/go/dev/src/davecheney/pkg/json-master/scanner_test.go:157 +0x51 testing.(*B).runN(0xc000194240, 0x1) C:/Go/src/testing/benchmark.go:191 +0xf2 testing.(*B).run1.func1(0xc000194240) C:/Go/src/testing/benchmark.go:231 +0x5e created by testing.(*B).run1 C:/Go/src/testing/benchmark.go:224 +0x85 exit status 2 `

    // --------------------------------------------------------------------------------------------------------------

    `

    		for i := 0; i < b.N; i++ {
    			r.Seek(0, 0)
    			scanner := &Scanner{
    				br: byteReader{
    					data: buf[:0],
    					r:    r,
    				},
    			}
    			scanner.Next()                                       // add scanner.Next() here to escape panic
    			n := scanner.parseNumber(scanner.br.window(0)[0])    // panic was here
    

    `

    opened by ivanburak 0
Owner
Artisanal, hand crafted, barrel aged, Go packages
null
Fast JSON encoder/decoder compatible with encoding/json for Go

Fast JSON encoder/decoder compatible with encoding/json for Go

Masaaki Goshima 1.7k Aug 7, 2022
Golang JSON decoder supporting case-sensitive, number-preserving, and strict decoding use cases

Golang JSON decoder supporting case-sensitive, number-preserving, and strict decoding use cases

Kubernetes SIGs 12 Apr 3, 2022
Go encoder and decoder for the NetBPM/PNM image formats. Compatible with Go's image packages.

gpnm This package implements an encoder and decoder for PNM image formats. It can be used with Go's image library. It covers all formats as defined by

null 0 Nov 26, 2021
goi - The “Quite OK Image” format encoder / decoder for Go.

goi - The “Quite OK Image” format encoder / decoder for Go. QOI - The “Quite OK Image” - is losslessly image format that offering speedup both compres

neguse 14 Mar 3, 2022
Decoder/Encoder for GhostControls Gate Remotes

ghostcontrols Decoder/Encoder for GhostControls Gate Remotes GhostControls makes a variety of automatic gate operators, transmitters and keypads & rec

null 0 Jan 1, 2022
Get JSON values quickly - JSON parser for Go

get json values quickly GJSON is a Go package that provides a fast and simple way to get values from a json document. It has features such as one line

Josh Baker 10.8k Aug 12, 2022
JSON diff library for Go based on RFC6902 (JSON Patch)

jsondiff jsondiff is a Go package for computing the diff between two JSON documents as a series of RFC6902 (JSON Patch) operations, which is particula

William Poussier 181 Aug 10, 2022
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

High Performance, Kubernetes Native Object Storage 2 Jun 26, 2022
Json-go - CLI to convert JSON to go and vice versa

Json To Go Struct CLI Install Go version 1.17 go install github.com/samit22/js

Samit Ghimire 7 Jul 29, 2022
JSON Spanner - A Go package that provides a fast and simple way to filter or transform a json document

JSON SPANNER JSON Spanner is a Go package that provides a fast and simple way to

null 4 Jul 18, 2022
Abstract JSON for golang with JSONPath support

Abstract JSON Abstract JSON is a small golang package provides a parser for JSON with support of JSONPath, in case when you are not sure in its struct

Stepan Pyzhov 122 Jul 27, 2022
Fast JSON parser and validator for Go. No custom structs, no code generation, no reflection

fastjson - fast JSON parser and validator for Go Features Fast. As usual, up to 15x faster than the standard encoding/json. See benchmarks. Parses arb

Aliaksandr Valialkin 1.6k Aug 14, 2022
Small utility to create JSON objects

gjo Small utility to create JSON objects. This was inspired by jpmens/jo. Support OS Mac Linux Windows Requirements Go 1.1.14~ Git Installtion Build $

skanehira 110 Jul 7, 2022
A Go package for handling common HTTP JSON responses.

go-respond A Go package for handling common HTTP JSON responses. Installation go get github.com/nicklaw5/go-respond Usage The goal of go-respond is to

Nick Law 48 Jul 17, 2022
JSON query in Golang

gojq JSON query in Golang. Install go get -u github.com/elgs/gojq This library serves three purposes: makes parsing JSON configuration file much easie

Qian Chen 182 Apr 27, 2022
Automatically generate Go (golang) struct definitions from example JSON

gojson gojson generates go struct definitions from json or yaml documents. Example $ curl -s https://api.github.com/repos/chimeracoder/gojson | gojson

Aditya Mukerjee 2.5k Aug 8, 2022
A JSON diff utility

JayDiff A JSON diff utility. Install Downloading the compiled binary Download the latest version of the binary: releases extract the archive and place

Guillaume de Sagazan 88 Aug 5, 2022
Fast and flexible JSON encoder for Go

Jettison Jettison is a fast and flexible JSON encoder for the Go programming language, inspired by bet365/jingo, with a richer features set, aiming at

William Poussier 129 Jul 10, 2022
Create go type representation from json

json2go Package json2go provides utilities for creating go type representation from json inputs. Json2go can be used in various ways: CLI tool Web pag

Maciek Zając 104 Aug 6, 2022