One of the fastest alternative JSON parser for Go that does not require schema

Overview

Go Report Card License

Alternative JSON parser for Go (10x times faster standard library)

It does not require you to know the structure of the payload (eg. create structs), and allows accessing fields by providing the path to them. It is up to 10 times faster than standard encoding/json package (depending on payload size and usage), allocates no memory. See benchmarks below.

Rationale

Originally I made this for a project that relies on a lot of 3rd party APIs that can be unpredictable and complex. I love simplicity and prefer to avoid external dependecies. encoding/json requires you to know exactly your data structures, or if you prefer to use map[string]interface{} instead, it will be very slow and hard to manage. I investigated what's on the market and found that most libraries are just wrappers around encoding/json, there is few options with own parsers (ffjson, easyjson), but they still requires you to create data structures.

Goal of this project is to push JSON parser to the performance limits and not sacrifice with compliance and developer user experience.

Example

For the given JSON our goal is to extract the user's full name, number of github followers and avatar.

import "github.com/buger/jsonparser"

...

data := []byte(`{
  "person": {
    "name": {
      "first": "Leonid",
      "last": "Bugaev",
      "fullName": "Leonid Bugaev"
    },
    "github": {
      "handle": "buger",
      "followers": 109
    },
    "avatars": [
      { "url": "https://avatars1.githubusercontent.com/u/14009?v=3&s=460", "type": "thumbnail" }
    ]
  },
  "company": {
    "name": "Acme"
  }
}`)

// You can specify key path by providing arguments to Get function
jsonparser.Get(data, "person", "name", "fullName")

// There is `GetInt` and `GetBoolean` helpers if you exactly know key data type
jsonparser.GetInt(data, "person", "github", "followers")

// When you try to get object, it will return you []byte slice pointer to data containing it
// In `company` it will be `{"name": "Acme"}`
jsonparser.Get(data, "company")

// If the key doesn't exist it will throw an error
var size int64
if value, err := jsonparser.GetInt(data, "company", "size"); err == nil {
  size = value
}

// You can use `ArrayEach` helper to iterate items [item1, item2 .... itemN]
jsonparser.ArrayEach(data, func(value []byte, dataType jsonparser.ValueType, offset int, err error) {
	fmt.Println(jsonparser.Get(value, "url"))
}, "person", "avatars")

// Or use can access fields by index!
jsonparser.GetString(data, "person", "avatars", "[0]", "url")

// You can use `ObjectEach` helper to iterate objects { "key1":object1, "key2":object2, .... "keyN":objectN }
jsonparser.ObjectEach(data, func(key []byte, value []byte, dataType jsonparser.ValueType, offset int) error {
        fmt.Printf("Key: '%s'\n Value: '%s'\n Type: %s\n", string(key), string(value), dataType)
	return nil
}, "person", "name")

// The most efficient way to extract multiple keys is `EachKey`

paths := [][]string{
  []string{"person", "name", "fullName"},
  []string{"person", "avatars", "[0]", "url"},
  []string{"company", "url"},
}
jsonparser.EachKey(data, func(idx int, value []byte, vt jsonparser.ValueType, err error){
  switch idx {
  case 0: // []string{"person", "name", "fullName"}
    ...
  case 1: // []string{"person", "avatars", "[0]", "url"}
    ...
  case 2: // []string{"company", "url"},
    ...
  }
}, paths...)

// For more information see docs below

Need to speedup your app?

I'm available for consulting and can help you push your app performance to the limits. Ping me at: [email protected].

Reference

Library API is really simple. You just need the Get method to perform any operation. The rest is just helpers around it.

You also can view API at godoc.org

Get

func Get(data []byte, keys ...string) (value []byte, dataType jsonparser.ValueType, offset int, err error)

Receives data structure, and key path to extract value from.

Returns:

  • value - Pointer to original data structure containing key value, or just empty slice if nothing found or error
  • dataType - Can be: NotExist, String, Number, Object, Array, Boolean or Null
  • offset - Offset from provided data structure where key value ends. Used mostly internally, for example for ArrayEach helper.
  • err - If the key is not found or any other parsing issue, it should return error. If key not found it also sets dataType to NotExist

Accepts multiple keys to specify path to JSON value (in case of quering nested structures). If no keys are provided it will try to extract the closest JSON value (simple ones or object/array), useful for reading streams or arrays, see ArrayEach implementation.

Note that keys can be an array indexes: jsonparser.GetInt("person", "avatars", "[0]", "url"), pretty cool, yeah?

GetString

func GetString(data []byte, keys ...string) (val string, err error)

Returns strings properly handing escaped and unicode characters. Note that this will cause additional memory allocations.

GetUnsafeString

If you need string in your app, and ready to sacrifice with support of escaped symbols in favor of speed. It returns string mapped to existing byte slice memory, without any allocations:

s, _, := jsonparser.GetUnsafeString(data, "person", "name", "title")
switch s {
  case 'CEO':
    ...
  case 'Engineer'
    ...
  ...
}

Note that unsafe here means that your string will exist until GC will free underlying byte slice, for most of cases it means that you can use this string only in current context, and should not pass it anywhere externally: through channels or any other way.

GetBoolean, GetInt and GetFloat

func GetBoolean(data []byte, keys ...string) (val bool, err error)

func GetFloat(data []byte, keys ...string) (val float64, err error)

func GetInt(data []byte, keys ...string) (val int64, err error)

If you know the key type, you can use the helpers above. If key data type do not match, it will return error.

ArrayEach

func ArrayEach(data []byte, cb func(value []byte, dataType jsonparser.ValueType, offset int, err error), keys ...string)

Needed for iterating arrays, accepts a callback function with the same return arguments as Get.

ObjectEach

func ObjectEach(data []byte, callback func(key []byte, value []byte, dataType ValueType, offset int) error, keys ...string) (err error)

Needed for iterating object, accepts a callback function. Example:

var handler func([]byte, []byte, jsonparser.ValueType, int) error
handler = func(key []byte, value []byte, dataType jsonparser.ValueType, offset int) error {
	//do stuff here
}
jsonparser.ObjectEach(myJson, handler)

EachKey

func EachKey(data []byte, cb func(idx int, value []byte, dataType jsonparser.ValueType, err error), paths ...[]string)

When you need to read multiple keys, and you do not afraid of low-level API EachKey is your friend. It read payload only single time, and calls callback function once path is found. For example when you call multiple times Get, it has to process payload multiple times, each time you call it. Depending on payload EachKey can be multiple times faster than Get. Path can use nested keys as well!

paths := [][]string{
	[]string{"uuid"},
	[]string{"tz"},
	[]string{"ua"},
	[]string{"st"},
}
var data SmallPayload

jsonparser.EachKey(smallFixture, func(idx int, value []byte, vt jsonparser.ValueType, err error){
	switch idx {
	case 0:
		data.Uuid, _ = value
	case 1:
		v, _ := jsonparser.ParseInt(value)
		data.Tz = int(v)
	case 2:
		data.Ua, _ = value
	case 3:
		v, _ := jsonparser.ParseInt(value)
		data.St = int(v)
	}
}, paths...)

Set

func Set(data []byte, setValue []byte, keys ...string) (value []byte, err error)

Receives existing data structure, key path to set, and value to set at that key. This functionality is experimental.

Returns:

  • value - Pointer to original data structure with updated or added key value.
  • err - If any parsing issue, it should return error.

Accepts multiple keys to specify path to JSON value (in case of updating or creating nested structures).

Note that keys can be an array indexes: jsonparser.Set(data, []byte("http://github.com"), "person", "avatars", "[0]", "url")

Delete

func Delete(data []byte, keys ...string) value []byte

Receives existing data structure, and key path to delete. This functionality is experimental.

Returns:

  • value - Pointer to original data structure with key path deleted if it can be found. If there is no key path, then the whole data structure is deleted.

Accepts multiple keys to specify path to JSON value (in case of updating or creating nested structures).

Note that keys can be an array indexes: jsonparser.Delete(data, "person", "avatars", "[0]", "url")

What makes it so fast?

  • It does not rely on encoding/json, reflection or interface{}, the only real package dependency is bytes.
  • Operates with JSON payload on byte level, providing you pointers to the original data structure: no memory allocation.
  • No automatic type conversions, by default everything is a []byte, but it provides you value type, so you can convert by yourself (there is few helpers included).
  • Does not parse full record, only keys you specified

Benchmarks

There are 3 benchmark types, trying to simulate real-life usage for small, medium and large JSON payloads. For each metric, the lower value is better. Time/op is in nanoseconds. Values better than standard encoding/json marked as bold text. Benchmarks run on standard Linode 1024 box.

Compared libraries:

TLDR

If you want to skip next sections we have 2 winner: jsonparser and easyjson. jsonparser is up to 10 times faster than standard encoding/json package (depending on payload size and usage), and almost infinitely (literally) better in memory consumption because it operates with data on byte level, and provide direct slice pointers. easyjson wins in CPU in medium tests and frankly i'm impressed with this package: it is remarkable results considering that it is almost drop-in replacement for encoding/json (require some code generation).

It's hard to fully compare jsonparser and easyjson (or ffson), they a true parsers and fully process record, unlike jsonparser which parse only keys you specified.

If you searching for replacement of encoding/json while keeping structs, easyjson is an amazing choice. If you want to process dynamic JSON, have memory constrains, or more control over your data you should try jsonparser.

jsonparser performance heavily depends on usage, and it works best when you do not need to process full record, only some keys. The more calls you need to make, the slower it will be, in contrast easyjson (or ffjson, encoding/json) parser record only 1 time, and then you can make as many calls as you want.

With great power comes great responsibility! :)

Small payload

Each test processes 190 bytes of http log as a JSON record. It should read multiple fields. https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_small_payload_test.go

Library time/op bytes/op allocs/op
encoding/json struct 7879 880 18
encoding/json interface{} 8946 1521 38
Jeffail/gabs 10053 1649 46
bitly/go-simplejson 10128 2241 36
antonholmquist/jason 27152 7237 101
github.com/ugorji/go/codec 8806 2176 31
mreiferson/go-ujson 7008 1409 37
a8m/djson 3862 1249 30
pquerna/ffjson 3769 624 15
mailru/easyjson 2002 192 9
buger/jsonparser 1367 0 0
buger/jsonparser (EachKey API) 809 0 0

Winners are ffjson, easyjson and jsonparser, where jsonparser is up to 9.8x faster than encoding/json and 4.6x faster than ffjson, and slightly faster than easyjson. If you look at memory allocation, jsonparser has no rivals, as it makes no data copy and operates with raw []byte structures and pointers to it.

Medium payload

Each test processes a 2.4kb JSON record (based on Clearbit API). It should read multiple nested fields and 1 array.

https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_medium_payload_test.go

Library time/op bytes/op allocs/op
encoding/json struct 57749 1336 29
encoding/json interface{} 79297 10627 215
Jeffail/gabs 83807 11202 235
bitly/go-simplejson 88187 17187 220
antonholmquist/jason 94099 19013 247
github.com/ugorji/go/codec 114719 6712 152
mreiferson/go-ujson 56972 11547 270
a8m/djson 28525 10196 198
pquerna/ffjson 20298 856 20
mailru/easyjson 10512 336 12
buger/jsonparser 15955 0 0
buger/jsonparser (EachKey API) 8916 0 0

The difference between ffjson and jsonparser in CPU usage is smaller, while the memory consumption difference is growing. On the other hand easyjson shows remarkable performance for medium payload.

gabs, go-simplejson and jason are based on encoding/json and map[string]interface{} and actually only helpers for unstructured JSON, their performance correlate with encoding/json interface{}, and they will skip next round. go-ujson while have its own parser, shows same performance as encoding/json, also skips next round. Same situation with ugorji/go/codec, but it showed unexpectedly bad performance for complex payloads.

Large payload

Each test processes a 24kb JSON record (based on Discourse API) It should read 2 arrays, and for each item in array get a few fields. Basically it means processing a full JSON file.

https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_large_payload_test.go

Library time/op bytes/op allocs/op
encoding/json struct 748336 8272 307
encoding/json interface{} 1224271 215425 3395
a8m/djson 510082 213682 2845
pquerna/ffjson 312271 7792 298
mailru/easyjson 154186 6992 288
buger/jsonparser 85308 0 0

jsonparser now is a winner, but do not forget that it is way more lightweight parser than ffson or easyjson, and they have to parser all the data, while jsonparser parse only what you need. All ffjson, easysjon and jsonparser have their own parsing code, and does not depend on encoding/json or interface{}, thats one of the reasons why they are so fast. easyjson also use a bit of unsafe package to reduce memory consuption (in theory it can lead to some unexpected GC issue, but i did not tested enough)

Also last benchmark did not included EachKey test, because in this particular case we need to read lot of Array values, and using ArrayEach is more efficient.

Questions and support

All bug-reports and suggestions should go though Github Issues.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Development

All my development happens using Docker, and repo include some Make tasks to simplify development.

  • make build - builds docker image, usually can be called only once
  • make test - run tests
  • make fmt - run go fmt
  • make bench - run benchmarks (if you need to run only single benchmark modify BENCHMARK variable in make file)
  • make profile - runs benchmark and generate 3 files- cpu.out, mem.mprof and benchmark.test binary, which can be used for go tool pprof
  • make bash - enter container (i use it for running go tool pprof above)
Issues
  • Add ObjectEach() for iterating over object key-value pairs

    Add ObjectEach() for iterating over object key-value pairs

    This PR implements the ObjectEach function to iterate over the key-value pairs of a JSON object. Its signature is as follows:

    ObjectEach(
      data []byte,
      callback func(key []byte, value []byte, valueType ValueType, offset int) error,
    )
    

    ObjectEach will invoke callback for each key-value pair in the JSON object contained in data as follows:

    • key will be the unescaped key string of the key-value pair
    • value will be the raw bytes composing the value of the key-value pair (as if returned by Get(); quotes are stripped from strings, null is represented as an empty byte slice, and all other values are passed as their literal bytes)
    • valueType is the JSON data type of value
    • offset is the byte offset just past the end of the value of the key-value pair

    If any invocation of callback returns a non-nil error, the iteration terminates and that error is returned from ObjectEach itself.

    The name ObjectEach was chosen to mirror the analogous ArrayEach, which iterates over the elements of a JSON array with a callback.

    Relevant benchmarks:

    $ goapp test -bench "BenchmarkJsonParser.*Struct.*" -benchtime 10s -benchmem
    BenchmarkJsonParserEachKeyStructMedium-8     1000000         15307 ns/op         544 B/op         11 allocs/op
    BenchmarkJsonParserObjectEachStructMedium-8   500000         23988 ns/op         608 B/op         12 allocs/op
    BenchmarkJsonParserEachKeyStructSmall-8     10000000          2248 ns/op         176 B/op          7 allocs/op
    BenchmarkJsonParserObjectEachStructSmall-8  10000000          2129 ns/op         240 B/op          8 allocs/op
    

    The decreased performance seen on the Medium test is because ObjectEach double-parses some values. For example, when looking up key person.name.fullName, ObjectEach visits the name key within person, which causes the value {"fullName": ..., ... } to be scanned to find its extent. Then, inside the callback (which receives the byte slice for the value of name), we call GetString to extract the value of fullName within this inner object, which scans that object a second time.

    The bottom line is that ObjectEach seems very efficient when scanning a leaf object (i.e., an object whose values are primitives), or when the inner objects will not be accessed (it should be efficient at simply skipping object values), but there is some overhead when values that are objects will be accessed further; in this case, EachKey or simple Gets may be a better choice.

    opened by daboyuka 22
  • escapes in keys are not supported

    escapes in keys are not supported

    https://github.com/buger/jsonparser/blob/f0ef2b717711d101fbf0856ca3cf58e8199a09cf/parser.go#L126

    Escapes can appear in both keys and values in json. The code linked to just compares the raw byte values. See eg https://tools.ietf.org/html/rfc7159#section-8.3

    opened by bazzargh 21
  • Add GetInt helper (and make GetNumber faster)?

    Add GetInt helper (and make GetNumber faster)?

    First, thank you for this package!

    What do you think about adding a new helper function: GetInt(data []byte, keys ...string) (val int64, offset int, err error)? I think this is a frequent use-case, so this function would be quite useful.

    Also to increase the performances of GetNumber and GetInt, you could copy the strconv package from the stdlib into your package and update the signatures of ParseFloat and ParseInt to use []byte instead of string. It would avoid allocating memory and make your package even faster.

    I think this is the kind of optimization that is acceptable in a performance-oriented package like yours.

    What do you think?

    opened by alexcesaro 17
  • Faster+safer use of unsafe package + non-unsafe fallback implementations

    Faster+safer use of unsafe package + non-unsafe fallback implementations

    Pardon the unhelpful PR title; wasn't sure what to call this...

    Removed the unsafeBytesToString function, instead encapsulating unsafe functionality in a separate set of files (fastbytes*), thus preventing misuse by client code. Included are fast implementations of BytesEqualStr, which takes one string and one []byte argument, and BytesParseFloat and BytesParseInt, which each take a []byte argument. None of these functions let unsafe strings escape, thus improving overall safety.

    The main benefit of this refactoring is the ability to have non-unsafe fallback versions of these functions for platforms without the unsafe package (such as Google App Engine). Such implementations are included in fastbytessafe.go.

    A somewhat faster implementation of BytesEqualStr is included than was previously used in the codebase. Sadly, string equality checks are a small portion of overall Get*() cost now, so there is little net performance impact. A significantly faster implementation of BytesParseInt is also included, but has little net impact for the same reason.

    No noticeable net change in the benchmark, however the code is now cleaner, safer, and more portable.

    Note that this patch exports Bytes{EqualStr,ParseInt,ParseFloat}, as they may be of use to client code. This is far safer than the old method of exporting GetUnsafeString, as these new functions are safe.

    Benchmark before change:

    BenchmarkJsonParserLarge-8    100000         62860 ns/op           0 B/op          0 allocs/op
    BenchmarkJsonParserMedium-8   500000         10639 ns/op           0 B/op          0 allocs/op
    BenchmarkJsonParserSmall-8  10000000           985 ns/op           0 B/op          0 allocs/op
    

    Benchmark after change:

    BenchmarkJsonParserLarge-8    100000         62428 ns/op           0 B/op          0 allocs/op
    BenchmarkJsonParserMedium-8  1000000         10779 ns/op           0 B/op          0 allocs/op
    BenchmarkJsonParserSmall-8  10000000           996 ns/op           0 B/op          0 allocs/op
    
    opened by daboyuka 16
  • Added ParsePrimitiveValue convenience function for users

    Added ParsePrimitiveValue convenience function for users

    Users will often want to convert a []byte returned by Get() or ArrayEach() that represents a primitive value (Null, Boolean, Number, String) into the corresponding Go type (nil, bool, float64, string). This PR provides a convenience method, ParsePrimitiveValue([]byte) (interface{}, error), which performs this service for such users.

    Since this is just a convenience method, Get(), etc. are unchanged (except some gofmt whitespace diff in searchKeys...)

    opened by daboyuka 12
  • Get Structure?

    Get Structure?

    Would it be possible to use the parser to get the JSON's Structure?

    I know there are existing packages out there, but all that I tried can't keep the original JSON's Structure's order. I.e., the output structure are all sorted by keys, which is both a blessed and cursed at the same time.

    Since this parser keeps the original order, it seems to be the only candidate that is able to do it.

    Thanks

    opened by suntong 11
  • Get() key search can bleed through levels of JSON hierarchy

    Get() key search can bleed through levels of JSON hierarchy

    I want to first thank you, @buger, for your work on this library. Looking up a few JSON key paths in large JSON blobs is a significant bottleneck in a project I'm working on, and your library could give us a big speedup without changing our data format.

    Unfortunately, I've discovered an issue in Get(): when searching for a key, Get() may locate that key outside the current JSON object. Here is an example test case that breaks (written using check.v1):

    package jsonparser_test
    
    import (
        "github.com/buger-jsonparser"
        . "gopkg.in/check.v1"
        "testing"
    )
    
    func (s *JsonParserTests) TestJsonParserSearchBleed(c *C) {
        killer := []byte(`{
          "parentkey": {
            "childkey": {
              "grandchildkey": 123
            },
            "otherchildkey": 123
          }
        }`)
    
        var jtype int
    
        _, jtype, _, _ = jsonparser.Get(killer, "childkey")
        c.Assert(jtype, Equals, jsonparser.NotExist) // fails, returns data parentkey.childkey
    
        _, jtype, _, _ = jsonparser.Get(killer, "parentkey", "childkey", "otherchildkey")
        c.Assert(jtype, Equals, jsonparser.NotExist) // fails, returns data parentkey.otherchildkey
    }
    
    // Boilerplate
    func Test(t *testing.T) { TestingT(t) }
    type JsonParserTests struct{}
    var _ = Suite(&JsonParserTests{})
    

    The issue is that Get() uses bytes.Index() to find the next key it's looking for, but only validates it by checking that it is surrounded by double quotes and followed by a colon. In particular, it does not check whether it has crossed an unmatched sequence of braces, which would indicate transitioning into another JSON object level.

    I don't have a great suggestion as to how to fix this, sadly. Best of luck.

    opened by daboyuka 11
  • panic: runtime error: slice bounds out of range

    panic: runtime error: slice bounds out of range

    payload: func main() { testJson := [ s, _ := jsonparser.GetString([]byte(testJson), testJson) fmt.Println(s) }

    panic: runtime error: slice bounds out of range [1:0]

    goroutine 1 [running]: github.com/buger/jsonparser.searchKeys(0x2c050000, 0x1, 0x1, 0xc0000d7e78, 0x1, 0x1, 0xc00003a000) D:/Go/golibsrc/src/github.com/buger/jsonparser/parser.go:311 +0xfdb github.com/buger/jsonparser.internalGet(0x2c050000, 0x1, 0x1, 0xc0000d7e78, 0x1, 0x1, 0xc0000d7d38, 0x65e120, 0x56afb0, 0xc0000d7dc0, ...) D:/Go/golibsrc/src/github.com/buger/jsonparser/parser.go:891 +0x3a6 github.com/buger/jsonparser.Get(0x2c050000, 0x1, 0x1, 0xc0000d7e78, 0x1, 0x1, 0xc0000d7e87, 0x0, 0xc0000d7e14, 0xc0000d7e87, ...) D:/Go/golibsrc/src/github.com/buger/jsonparser/parser.go:885 +0x90 github.com/buger/jsonparser.GetString(0x2c050000, 0x1, 0x1, 0xc0000d7e78, 0x1, 0x1, 0x9, 0x9, 0x0, 0x0) D:/Go/golibsrc/src/github.com/buger/jsonparser/parser.go:1122 +0x9e

    opened by toptotu 10
  • Set values

    Set values

    It would be really nice if this package provided a Set(fieldName string, value []byte) or Set(fieldName string, value json.RawMessage) function. Right now we are processing huge amount of big json-line files. Each line has to be unmarshalled as map[string][json.RawMessage](https://golang.org/pkg/encoding/json/#RawMessage), then update just one filed and finally marshal it again. It would be really nice if we could update a value without unmarshalling! What do you think?

    opened by pavelnikolov 10
  • Add EachKey

    Add EachKey

    Description: Fetch multiple keys at once. Instead of reading whole payload for each key, it does read only 1 time.

    Using new API you can get nearly 2x improvement when fetching multiple keys, and the more keys you fetch, the bigger difference.

    Example:

    paths := [][]string{
        []string{"uuid"},
        []string{"tz"},
        []string{"ua"},
        []string{"st"},
    }
    
    jsonparser.EachKey(smallFixture, func(idx int, value []byte, vt jsonparser.ValueType, err error){
        switch idx {
        case 0: // uuid
            // jsonparser.ParseString(value)
        case 1: // tz
            jsonparser.ParseInt(value)
        case 2: // ua
            // jsonparser.ParseString(value)
        case 3: //sa
            jsonparser.ParseInt(value)
        }
    }, paths...)
    

    Benchmark before change:

    BenchmarkJsonParserLarge      100000         84621 ns/op           0 B/op          0 allocs/op
    BenchmarkJsonParserMedium     500000         15803 ns/op           0 B/op          0 allocs/op
    BenchmarkJsonParserSmall     5000000          1378 ns/op           0 B/op          0 allocs/op
    

    Added 2 new benchmarks:

    BenchmarkJsonParserEachKeyManualMedium   1000000          8312 ns/op           0 B/op          0 allocs/op
    BenchmarkJsonParserEachKeyManualSmall   10000000           798 ns/op           0 B/op          0 allocs/op
    
    opened by buger 9
  • Handle escaped keys and values in JSON

    Handle escaped keys and values in JSON

    Added proper JSON escaped string handling in both keys and values (as per RFC 7159). These changes necessarily resulted in some performance degradation, but it's relatively minor (< 10%, it seems).

    (Note: this is a refinement of #39)

    Benchmark before:

    BenchmarkJsonParserLarge-8           64506 ns/op           0 allocs/op
    BenchmarkJsonParserMedium-8          10865 ns/op           0 allocs/op
    BenchmarkJsonParserSmall-8            1026 ns/op           0 allocs/op
    

    Benchmark after:

    BenchmarkJsonParserLarge-8           66558 ns/op           0 allocs/op
    BenchmarkJsonParserMedium-8          11958 ns/op           0 allocs/op
    BenchmarkJsonParserSmall-8            1061 ns/op           0 allocs/op
    
    opened by daboyuka 8
  • update jsonparser version to v1.1.1 in benchmark package

    update jsonparser version to v1.1.1 in benchmark package

    Description: What this PR does update jsonparser version to v1.1.1 in benchmark package

    Benchmark before change:

    Benchmark after change:

    For running benchmarks use:

    go test -test.benchmem -bench JsonParser ./benchmark/ -benchtime 5s -v
    # OR
    make bench (runs inside docker)
    
    opened by romberli 0
  • package github.com/buger/jsonparser is not a main package

    package github.com/buger/jsonparser is not a main package

    I'm finding this problem when i run "go install github.com/buger/[email protected]" in my terminal, could someone help?

    image I'm using, GoLang 1.18 and the vscode terminal up there /\ and the regular windows terminal / right bellow image

    when i try to import this msgs appears image

    thanks for the time and help, nice day for everyone

    opened by Raulbarossi 2
  • The set function does not work

    The set function does not work

    data := []byte({ "person": { "name": { "first": "Leonid", "last": "Bugaev", "fullName": "Leonid Bugaev" }, "github": { "handle": "buger", "followers": 109 }, "avatars": [ { "url": "https://avatars1.githubusercontent.com/u/14009?v=3&s=460", "type": "thumbnail" } ] }, "company": { "name": "Acme" } }`)

    x, err := jsonparser.Set(data, []byte("http://github.com"), "person", "avatars", "[0]", "url")
    if err == nil {
    	log.Println(err.Error())
    }
    

    `

    The code above gives error: runtime error: invalid memory address or nil pointer dereference

    opened by fahimbinkhair 0
  • how to add item in slice?

    how to add item in slice?

    // Is this a right way?
    func main() {
    	var data = []byte(`[1,2,3]`)
    	cnt := 2
    	data,_  = jsonparser.Set(data,[]byte("3,4"),fmt.Sprintf("[%v]", cnt))
    	fmt.Println(string(data))
    	// Output: [1,2,3,4]
    	tmp := []int{}
    	_ = json.Unmarshal(data,&tmp)
    	fmt.Println(len(tmp))
    	// Output: 4
    }
    
    opened by cwww3 0
  • Rename default branch from `master`

    Rename default branch from `master`

    Could we consider renaming the default branch from master to something more inclusive? It appears that most providers have chosen to change their default from master to main (e.g. GitHub, GitLab). Once deleted, I believe references to master will redirect to main, reducing the pain caused by this transition. Nevertheless, I appreciate this is a non-trivial change.

    opened by BenjaminEHowe 1
Releases(v1.1.1)
  • v1.1.1(Jan 8, 2021)

  • v1.1.0(Dec 24, 2020)

    Christmas present 🎁

    • Improve Set performance by 20% https://github.com/buger/jsonparser/pull/196
    • Memory allocation improvements during Array iteration https://github.com/buger/jsonparser/pull/208
    • Add support for ppc64-le architecture https://github.com/buger/jsonparser/pull/213
    • Fixed repeated paths from incorrectly incrementing data offset https://github.com/buger/jsonparser/pull/215
    • Fixed array iteration when you have more than 64 items https://github.com/buger/jsonparser/pull/216
    • Fix possible memory confusion in unsafe slice cast https://github.com/buger/jsonparser/pull/204
    • Fixed usage of Array iteration with Set https://github.com/buger/jsonparser/issues/199
    • Integration with go-fuzz https://github.com/buger/jsonparser/pull/217

    Thanks to everyone who participated in the project! @AllenX2018 @saginadir @xsandr @nagesh4193 @rrgilchrist @floren @AdamKorcz @jlauinger

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(May 11, 2020)

Owner
Leonid Bugaev
👉 👉 👉 👉 👉 👉 👉 👉 👉 👉 👉
Leonid Bugaev
Fastest JSON interperter for golang

Welcome To JIN "Your wish is my command" Fast and Easy Way to Deal With JSON Jin is a comprehensive JSON manipulation tool bundle. All functions teste

eco 57 May 14, 2022
Get JSON values quickly - JSON parser for Go

get json values quickly GJSON is a Go package that provides a fast and simple way to get values from a json document. It has features such as one line

Josh Baker 10.3k May 19, 2022
converts text-formats from one to another, it is very useful if you want to re-format a json file to yaml, toml to yaml, csv to yaml, ... etc

re-txt reformates a text file from a structure to another, i.e: convert from json to yaml, toml to json, ... etc Supported Source Formats json yaml hc

Mohammed Al Ashaal 66 Jan 29, 2022
Fast JSON parser and validator for Go. No custom structs, no code generation, no reflection

fastjson - fast JSON parser and validator for Go Features Fast. As usual, up to 15x faster than the standard encoding/json. See benchmarks. Parses arb

Aliaksandr Valialkin 1.5k May 11, 2022
A JSON stream parser for Go

pjson A JSON stream parser for Go Example The example below prints all string values from a JSON document. package

Josh Baker 59 Dec 2, 2021
A fast json parser for go

rjson rjson is a json parser that relies on Ragel-generated state machines for most parsing. rjson's api is minimal and focussed on efficient parsing.

WillAbides 49 Feb 23, 2022
Slow and unreliable JSON parser generator (in progress)

VivaceJSON Fast and reliable JSON parser generator Todo List parse fields parse types generate struct generate (keypath+key) to struct Value Mapping F

null 7 Mar 3, 2022
JSON:API compatible query string parser

QParser The package helps to parse part of the URL path and its parameters string to a handy structure. The structure format is compatible with the JS

Velmie 1 Dec 21, 2021
Easy JSON parser for Go. No custom structs, no code generation, no reflection

Easy JSON parser for Go. No custom structs, no code generation, no reflection

null 2 Jan 4, 2022
A JSON parser/generator for Go

FASTJSON fastjson是java版本的fastjson库的api做一个翻译,方便习惯java的人操作json数据 主要适用场景:层级和字段都不能确定的json 这个库并不实现高性能json解析,依赖标准库json 这个库并没有100%实现java版的api 安装 go get -u gi

wuyunhua 2 Dec 29, 2021
Example to validate performance using append or not in golang

benchtest-arr-go This code is a example to validate performance using append or not in golang result benchtests go test -benchmem -bench . goos: darwi

Renan Bastos 1 Jan 10, 2022
JSON diff library for Go based on RFC6902 (JSON Patch)

jsondiff jsondiff is a Go package for computing the diff between two JSON documents as a series of RFC6902 (JSON Patch) operations, which is particula

William Poussier 166 May 7, 2022
Fast JSON encoder/decoder compatible with encoding/json for Go

Fast JSON encoder/decoder compatible with encoding/json for Go

Masaaki Goshima 1.5k May 14, 2022
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

High Performance, Kubernetes Native Object Storage 3 May 10, 2022
Json-go - CLI to convert JSON to go and vice versa

Json To Go Struct CLI Install Go version 1.17 go install github.com/samit22/js

Samit Ghimire 5 Mar 3, 2022
JSON Spanner - A Go package that provides a fast and simple way to filter or transform a json document

JSON SPANNER JSON Spanner is a Go package that provides a fast and simple way to

null 2 May 20, 2022
Simple Email Parser

mp - mail parser mp is a simple cli email parser. It currently takes stdin and outputs JSON. Example: cat fixtures/test.eml | mp { "Text": "Hello w

Mark 45 Apr 8, 2022
Go-json5 - A parser that supports a subset of the JSON5 specification

barney.ci/go-json5 This library implements a parser that supports a subset of th

Barney CI 3 Jan 7, 2022
Abstract JSON for golang with JSONPath support

Abstract JSON Abstract JSON is a small golang package provides a parser for JSON with support of JSONPath, in case when you are not sure in its struct

Stepan Pyzhov 114 May 14, 2022