A simple Go package to Query over JSON/YAML/XML/CSV Data

Overview

gojsonq-logo

Build Status Project status Go Report Card Coverage Status GoDoc License

A simple Go package to Query over JSON Data. It provides simple, elegant and fast ODM like API to access, query JSON document

Installation

Install the package using

$ go get github.com/thedevsaddam/gojsonq/v2

Usage

To use the package import it in your *.go code

import "github.com/thedevsaddam/gojsonq/v2"

Let's see a quick example:

See in playground

package main

import gojsonq "github.com/thedevsaddam/gojsonq/v2"

func main() {
	const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`
	name := gojsonq.New().FromString(json).Find("name.first")
	println(name.(string)) // Tom
}

Another example:

See in playground

package main

import (
	"fmt"

	gojsonq "github.com/thedevsaddam/gojsonq/v2"
)

func main() {
	const json = `{"city":"dhaka","type":"weekly","temperatures":[30,39.9,35.4,33.5,31.6,33.2,30.7]}`
	avg := gojsonq.New().FromString(json).From("temperatures").Avg()
	fmt.Printf("Average temperature: %.2f", avg) // 33.471428571428575
}

You can query your document using the various query methods such as Find, First, Nth, Pluck, Where, OrWhere, WhereIn, WhereStartsWith, WhereEndsWith, WhereContains, Sort, GroupBy, SortBy and so on. Also you can aggregate data after query using Avg, Count, Max, Min, Sum etc.

Find more query API in Wiki page

Bugs and Issues

If you encounter any bugs or issues, feel free to open an issue at github.

Also, you can shoot me an email to mailto:[email protected] for hugs or bugs.

Credit

Special thanks to Nahid Bin Azhar for the inspiration and guidance for the package. Thanks to Ahmed Shamim Hasan Shaon for his support from the very beginning.

Contributors

Contribution

If you are interested to make the package better please send pull requests or create an issue so that others can fix. Read the contribution guide here

License

The gojsonq is an open-source software licensed under the MIT License.

Issues
  • add handling of objects to processQuery

    add handling of objects to processQuery

    Change to enable queries on standalone objects. This makes queries like this possible:

    j := gojsonq.New().JSONString(`{"name":{"first":"Tom","last":"Hanks"},"age":61}`)
    ct := j.Where("age", ">=", 60).Where("name.last", "eq", "Hanks").Count() // 1
    
    opened by mrgleeco 9
  • NOT an issue, more a touch of help. Using parameters with gojsonq package

    NOT an issue, more a touch of help. Using parameters with gojsonq package

    I am trying to build a server that can fetch certain json entries based on a parameter. So for example if a user enters localhost:8080/store/1 It returns the json entry with the ID 1.

    The code i currently have works if i hardcode the id field into the query, but trying to put in parameters i have so far had no luck. Trying this:

    func getGame(w http.ResponseWriter, r *http.Request) {
    	params := mux.Vars(r)
    	jq := gojsonq.New().File("./data/games.json")
    	res := jq.From("games").Where("id", "=", params["id"]).Get()
    	fmt.Println(res)
    }
    

    Suggestions please, before i throw something out the window.

    question 
    opened by JamieW87 9
  • Multiple nested array support

    Multiple nested array support

    Is it possible to traverse multiple nested arrays in this library without knowing the array indexes? I have a scenario that works well with where clauses on the first array, but I haven't had any success with a second array below that.

    This is a successful query using Find (but assumes I know the index to each array):

    Find("data.items.[1].nestedA.[3].nestedB.value")

    I've tried many variations of: From("data.items").Where("itemValue", "=", "asdf").From("nestedA").Where("nestedBOtherValue", "eq", "fdsa").Find("nestedB.value") without any luck.

    opened by johndww 8
  • Add more error info on As() function

    Add more error info on As() function

    issue

    When using :

    tempTags = []Tag{} // Structure
    jsonResult.As(&tempTags)
    

    This error is displayed an error from reflect library:

    panic: reflect: call of reflect.Value.Call on zero Value [recovered]
            panic: reflect: call of reflect.Value.Call on zero Value
    

    Expected

    Should display a more specific error that give us more clues in order to understand what is going on

    Panic: the method '*[]repository.Tag' must be part of 'map[*[]bool:BoolSlice *[]float32:Float32Slice *[]float64:Float64Slice *[]int:IntSlice *[]int16:Int16Slice *[]int32:Int32Slice *[]int8:Int8Slice *[]string:StringSlice *[]time.Duration:DurationSlice *[]uint:UintSlice *[]uint16:Uint16Slice *[]uint32:Uint32Slice *[]uint8:Uint8Slice *bool:Bool *float32:Float32 *float64:Float64 *int:Int *int16:Int16 *int32:Int32 *int8:Int8 *string:String *time.Duration:Duration *uint:Uint *uint16:Uint16 *uint32:Uint32 *uint8:Uint8]' [recovered]
            panic: the method '*[]repository.Tag' must be part of 'map[*[]bool:BoolSlice *[]float32:Float32Slice *[]float64:Float64Slice *[]int:IntSlice *[]int16:Int16Slice *[]int32:Int32Slice *[]int8:Int8Slice *[]string:StringSlice *[]time.Duration:DurationSlice *[]uint:UintSlice *[]uint16:Uint16Slice *[]uint32:Uint32Slice *[]uint8:Uint8Slice *bool:Bool *float32:Float32 *float64:Float64 *int:Int *int16:Int16 *int32:Int32 *int8:Int8 *string:String *time.Duration:Duration *uint:Uint *uint16:Uint16 *uint32:Uint32 *uint8:Uint8]'
    
    opened by cifren 6
  • Query json for array size.

    Query json for array size.

    Hello,

    This is an "info request".

    I have the following json and I would like to check if the input contains a father (fid) with two children.

    {
      "data": [
        {
          "fid": 10,
          "children": [
            {
              "cid": 123
            },
            {
              "cid": 124
            }
           
          ]
        }
      ]
    }
    

    The code so far is

    package main
    
    import (
    	"fmt"
    
    	"github.com/thedevsaddam/gojsonq"
    )
    
    const json = `{
    	"data": [
    	  {
    		"fid": 10,
    		"children": [
    		  {
    			"cid": 123
    		  },
    		  {
    			"cid": 124
    		  }
    		 
    		]
    	  }
    	]
      }`
    
    func main() {
    
    	q := gojsonq.New().JSONString(json).From("data").
    		Where("fid", "=", 10)
    	fmt.Println(q.Get())
    }
    
    

    How can I modify the source code to decide if fid=10 AND children.size() = 2?

    In principle how can I use the "array size" to query the json file/input?

    Thanks

    enhancement question discussion priority-mid 
    opened by lefscode 6
  • Map[string]interface{} root

    Map[string]interface{} root

    Doesn't seem to work when you don't have nicely sliceable structures.

    IE: https://www.iinet.net.au/status/event/events.json

    I tried Where("DRW", "=", "Darwin") but kept getting nothing back

    enhancement 
    opened by freman 5
  • FromInterface custom decoder

    FromInterface custom decoder

    Hi there, Is there any way to load JSON document from map[string]interface{}? Simplest use case is querying MongoDB, which returns result documents in that format(without using schema struct). I am willing to make a PR if little help is supplied. Thanks

    Feature request 
    opened by Dragomir-Ivanov 3
  • Go-critic warnings

    Go-critic warnings

    go-critic linter found some issues , here corresponding log:

    check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 0 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 1 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 2 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 3 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 4 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 5 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 6 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\jsonq.go:271:3: range ValCopy: each iteration copies 48 bytes (consider pointers or indexing) check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\jsonq.go:45:18: typeU nparen: could simplify to [][]query

    In #18 I propose fixing for them

    Why replacerange with for loop? You must create an issue regarding this refactor before sending PR.

    Thanks, good point. The way that I made in my PR not very accurate and should be replaced into this one as written in description to rangeValCopy.

    opened by Kvaz1r 3
  • Added documentation and examples

    Added documentation and examples

    I saw a post on r/golang probably yours. And I wondered why you did not add the examples in the linked article to your project, so I did add some and of your README-file. These will be shown on https://godoc.org/github.com/thedevsaddam/gojsonq the godoc page.

    You have to edit the import-path of the examples_test.go file to your repo as I plan to remove my fork as soon as the changes are merged.

    opened by D1CED 3
  • Why interfaces?

    Why interfaces?

    Thanks for sharing this project. One question. Why do you do this:

    // strStrictContains checks if x contains y
    // This is case sensitive search
    func strStrictContains(x, y interface{}) (bool, error) {
    	xv, okX := x.(string)
    	if !okX {
    		return false, fmt.Errorf("%v must be string", x)
    	}
    	yv, okY := y.(string)
    	if !okY {
    		return false, fmt.Errorf("%v must be string", y)
    	}
    	return strings.Contains(xv, yv), nil
    }
    

    ...and not this:

    // strStrictContains checks if x contains y
    // This is case sensitive search
    func strStrictContains(x, y string) (bool) {
    	return strings.Contains(xv, yv)
    }
    

    ...or in this case, simply strings.Contains(xv, yv) without a wrapper.

    In this example, you are only allowing string inputs anyway, so why not use strongly typed strings as input params instead of interface{}?

    opened by nkev 2
  • Support for unselect

    Support for unselect

    I have an array of JSON objects which all contain a few common keys and then a whole bunch of keys that are different. I want to select a subset of this array that drops a single common key.

    The only way to do that right now is to use a select or only with all the possible keys the object can contain minus the one I want to drop. It would be nice to support something like "unselect" or "drop" where I can drop a specific key from the result while keeping the rest.

    opened by gboor 2
  • Is it possible to parse multiple json together?

    Is it possible to parse multiple json together?

    Team,

    I have tried parsing this json

    {
      "module": {
        "ABCD": {
          "source": "",
          "version": "",
          "location": "",
          "resource_group_name": ""
        }
      },
      "module": {
        "PQRS": {
          "source": "",
          "version": "",
          "sa_name": "",
          "resource_group": ""
        }
      }
    }
    

    But I'm always getting the details of the last node i.e. PQRS. Is there anyway to get the details of both the nodes considering parent node as modules. The output required is

    {
     "ABCD": {
          "source": "",
          "version": "",
          "location": "",
          "resource_group_name": ""
        },
    "PQRS": {
          "source": "",
          "version": "",
          "sa_name": "",
          "resource_group": ""
        }
    }
    
    opened by smartaquarius10 0
  • Possible query

    Possible query

    is it possible to compare a query with 2 keys? I tried 2 3 tests, but it didn't work for me.

    {
        "authorizedWriters": {
            "teams": [],
            "users": []
        },
        "created": 1627654462505,
        "creator": "E7jJBMMAEAA",
        "dashboards": ["E7jJCz8AEAA"],
        "lastUpdated": 1627654462744,
        "lastUpdatedBy": "E7jJBMMAEAA",
        "teams": []
    }
    

    What i try to do:

    // give me all objects which have a different updater like creator
    // if creator == lastUpdatedBy
    
    json:= gojsonq.New().JSONString(json).From("results").Select("dashboardId","name").Where(.....?????)
    

    greetings

    opened by AchimGrolimund 0
  • result is []interface

    result is []interface

    this jsonStr ="{"uid":17766,"openid":"ni8vhveqn6","appOpenId":"nq46kiae15","unionId":"i2jki7ttm2","nickName":"","avatarUrl":"","gender":"0","cpc":"","wxappShareImg":"","inviteCode":"","tel":"","loginTime":1603268944,"status":0,"loginIP":"","platform":"","channel":"","versionCode":"","loginType":1,"CreatedAt":"2020-10-21T16:29:05+08:00","UpdatedAt":"2020-10-21T16:29:05+08:00"}"`

    i want get uid and openid ,why my get is 【】interface

    userJsonq := gojsonq.New().FromString(strJson) userRes := userJsonq.Select("uid","openid").Get()

    opened by huanBird 1
  • Can SUM func Support DOT Operation

    Can SUM func Support DOT Operation

    For example, I have a json string like [{"s":1, "a": {"b":1}},{"s":1, "a": {"b":1}},{"s":1, "a": {"b":5}}] I want jq.Sum("a.b"), Is this possible?

    opened by lnn1988 0
  • (De-)Serializing support

    (De-)Serializing support

    Hi,

    It is just a feature request ;) Please consider support for serializing and deserializing queries (i.e. to/from JSON format), so they could be easily stored or send from client to server, what would allow execution that query on server and just return queried values.

    Thanks!

    opened by dudududi 0
Releases(v2.5.2)
  • v2.5.2(Mar 27, 2020)

  • v2.5.1(Mar 11, 2020)

  • v2.5.0(Jan 13, 2020)

  • v2.4.0(Nov 20, 2019)

    Result.As() supports all the available method in Result type except for the methods with an argument like Time

    result, _ := gojsonq.New().FromString(jsonStr).From("prices").GetR() // handle error
    var prices []float32
    result.As(&prices) // handle error
    fmt.Printf("%#v\n", prices)
    
    Source code(tar.gz)
    Source code(zip)
  • v2.3.0(Sep 23, 2019)

    The Result instance will return different methods to get static type.

    const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`
    result, err := gojsonq.New().JSONString(json).FindR("name.first")
    if err != nil {
    	log.Fatal(err)
    }
    name, _ := result.String()
    fmt.Println(name)
    
    Source code(tar.gz)
    Source code(zip)
  • v2.2.1(May 7, 2019)

    Fix select attributes for First,Last & Nth methods

    res := gojsonq.New().File("./data.json").From("items").Select("name", "price").First()
    fmt.Printf("%#v\n", res)
    

    Output

    map[string]interface {}{"name":"MacBook Pro 13 inch retina", "price":1350}
    
    Source code(tar.gz)
    Source code(zip)
  • v2.2.0(Apr 5, 2019)

    Offset

    gojsonq.New().File("./data.json").From("items").Offset(10).Limit(10).Get()
    

    Writer

    gojsonq.New().File("./data.json").From("items").SortBy("price").Writer(os.Stdout)
    

    FromString

    gojsonq.New().FromString(jsonStr).From("items").Get()
    
    Source code(tar.gz)
    Source code(zip)
  • v2.0.1(Feb 20, 2019)

  • v2.0.0(Feb 20, 2019)

  • v1.9.1(Dec 2, 2018)

  • v1.9.0(Feb 20, 2019)

Owner
Saddam H
Sr. Software Engineer @ Pathao Inc | Open Source Enthusiast | Love to write elegant code | Golang | Distributed Systems
Saddam H
JSONata in Go Package jsonata is a query and transformation language for JSON

JSONata in Go Package jsonata is a query and transformation language for JSON. It's a Go port of the JavaScript library JSONata.

Blues Inc 25 May 27, 2022
JSON Spanner - A Go package that provides a fast and simple way to filter or transform a json document

JSON SPANNER JSON Spanner is a Go package that provides a fast and simple way to

null 3 Jun 30, 2022
/ˈdʏf/ - diff tool for YAML files, and sometimes JSON

dyff is inspired by the way the old BOSH v1 deployment output reported changes from one version to another by only showing the parts of a YAML file that change.

null 638 Jun 23, 2022
A convenient syntax to generate JSON (or YAML) for commandline

clon A convenient syntax to generate JSON (or YAML) for commandline "mumbo-jumbo". Syntax Overview Syntax resembles that of JSON with a few caveats: a

Luca Sepe 9 May 14, 2021
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

High Performance, Kubernetes Native Object Storage 3 May 10, 2022
JSON query in Golang

gojq JSON query in Golang. Install go get -u github.com/elgs/gojq This library serves three purposes: makes parsing JSON configuration file much easie

Qian Chen 182 Apr 27, 2022
Console JSON formatter with query feature

Console JSON formatter with query feature. Install: $ go get github.com/miolini/jsonf Usage: Usage of jsonf: -c=true: colorize output -d=false: de

Artem Andreenko 63 Jan 23, 2022
A library to query the godoc.org JSON API.

gopkg This repository provides minimal Go package that makes queries against the godoc.org JSON API. Since that site has mostly been subsumed by pkg.g

M. J. Fromberger 2 Nov 26, 2021
JSON:API compatible query string parser

QParser The package helps to parse part of the URL path and its parameters string to a handy structure. The structure format is compatible with the JS

Velmie 1 Dec 21, 2021
An example of how to parse json data using go....a typical step for preparing data prior to uploading to a db.

JSON parser using GO An example of parsing json data in go, when you already know the schema of the data Example input: { "num_listings":"36",

null 0 Jan 12, 2022
Get JSON values quickly - JSON parser for Go

get json values quickly GJSON is a Go package that provides a fast and simple way to get values from a json document. It has features such as one line

Josh Baker 10.5k Jun 30, 2022
JSON diff library for Go based on RFC6902 (JSON Patch)

jsondiff jsondiff is a Go package for computing the diff between two JSON documents as a series of RFC6902 (JSON Patch) operations, which is particula

William Poussier 172 Jun 23, 2022
Fast JSON encoder/decoder compatible with encoding/json for Go

Fast JSON encoder/decoder compatible with encoding/json for Go

Masaaki Goshima 1.6k Jun 26, 2022
Json-go - CLI to convert JSON to go and vice versa

Json To Go Struct CLI Install Go version 1.17 go install github.com/samit22/js

Samit Ghimire 5 Mar 3, 2022
library for working amorphous data (as when you decode json into an interface{})

Introduction Decoding json into an interface{} produces an hierarchical arrangement of four data types: float64, string are 'primative types' and form

Chuck Luciano 9 Jul 5, 2021
Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang

kazaam Description Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang. This functionality provides

Qntfy 205 Sep 17, 2021
A tool to aggregate and mine data from JSON reports of Go tests.

teststat A tool to aggregate and mine data from JSON reports of Go tests. Why? Mature Go projects often have a lot of tests, and not all of them are i

Viacheslav Poturaev 3 Jan 3, 2022
Jsonmask use for mask sensitive data from json format

Jsonmask use for mask sensitive data from json format Installation go get -u github.com/rkritchat/jsonmask Code example package main import ( "fmt"

rkritchat 2 Mar 26, 2022
Senml-go - a Golang module for the JSON-based SenML sensor data format

ThingWave SenML module for Golang This is a Golang module for the JSON-based Sen

ThingWave 0 Jan 2, 2022