A simple Go package to Query over JSON/YAML/XML/CSV Data

Overview

gojsonq-logo

Build Status Project status Go Report Card Coverage Status GoDoc License

A simple Go package to Query over JSON Data. It provides simple, elegant and fast ODM like API to access, query JSON document

Installation

Install the package using

$ go get github.com/thedevsaddam/gojsonq/v2

Usage

To use the package import it in your *.go code

import "github.com/thedevsaddam/gojsonq/v2"

Let's see a quick example:

See in playground

package main

import gojsonq "github.com/thedevsaddam/gojsonq/v2"

func main() {
	const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`
	name := gojsonq.New().FromString(json).Find("name.first")
	println(name.(string)) // Tom
}

Another example:

See in playground

package main

import (
	"fmt"

	gojsonq "github.com/thedevsaddam/gojsonq/v2"
)

func main() {
	const json = `{"city":"dhaka","type":"weekly","temperatures":[30,39.9,35.4,33.5,31.6,33.2,30.7]}`
	avg := gojsonq.New().FromString(json).From("temperatures").Avg()
	fmt.Printf("Average temperature: %.2f", avg) // 33.471428571428575
}

You can query your document using the various query methods such as Find, First, Nth, Pluck, Where, OrWhere, WhereIn, WhereStartsWith, WhereEndsWith, WhereContains, Sort, GroupBy, SortBy and so on. Also you can aggregate data after query using Avg, Count, Max, Min, Sum etc.

Find more query API in Wiki page

Bugs and Issues

If you encounter any bugs or issues, feel free to open an issue at github.

Also, you can shoot me an email to mailto:[email protected] for hugs or bugs.

Credit

Special thanks to Nahid Bin Azhar for the inspiration and guidance for the package. Thanks to Ahmed Shamim Hasan Shaon for his support from the very beginning.

Contributors

Contribution

If you are interested to make the package better please send pull requests or create an issue so that others can fix. Read the contribution guide here

Special Thanks

License

The gojsonq is an open-source software licensed under the MIT License.

Issues
  • add handling of objects to processQuery

    add handling of objects to processQuery

    Change to enable queries on standalone objects. This makes queries like this possible:

    j := gojsonq.New().JSONString(`{"name":{"first":"Tom","last":"Hanks"},"age":61}`)
    ct := j.Where("age", ">=", 60).Where("name.last", "eq", "Hanks").Count() // 1
    
    opened by mrgleeco 9
  • NOT an issue, more a touch of help. Using parameters with gojsonq package

    NOT an issue, more a touch of help. Using parameters with gojsonq package

    I am trying to build a server that can fetch certain json entries based on a parameter. So for example if a user enters localhost:8080/store/1 It returns the json entry with the ID 1.

    The code i currently have works if i hardcode the id field into the query, but trying to put in parameters i have so far had no luck. Trying this:

    func getGame(w http.ResponseWriter, r *http.Request) {
    	params := mux.Vars(r)
    	jq := gojsonq.New().File("./data/games.json")
    	res := jq.From("games").Where("id", "=", params["id"]).Get()
    	fmt.Println(res)
    }
    

    Suggestions please, before i throw something out the window.

    question 
    opened by JamieW87 9
  • Multiple nested array support

    Multiple nested array support

    Is it possible to traverse multiple nested arrays in this library without knowing the array indexes? I have a scenario that works well with where clauses on the first array, but I haven't had any success with a second array below that.

    This is a successful query using Find (but assumes I know the index to each array):

    Find("data.items.[1].nestedA.[3].nestedB.value")

    I've tried many variations of: From("data.items").Where("itemValue", "=", "asdf").From("nestedA").Where("nestedBOtherValue", "eq", "fdsa").Find("nestedB.value") without any luck.

    opened by johndww 8
  • Add more error info on As() function

    Add more error info on As() function

    issue

    When using :

    tempTags = []Tag{} // Structure
    jsonResult.As(&tempTags)
    

    This error is displayed an error from reflect library:

    panic: reflect: call of reflect.Value.Call on zero Value [recovered]
            panic: reflect: call of reflect.Value.Call on zero Value
    

    Expected

    Should display a more specific error that give us more clues in order to understand what is going on

    Panic: the method '*[]repository.Tag' must be part of 'map[*[]bool:BoolSlice *[]float32:Float32Slice *[]float64:Float64Slice *[]int:IntSlice *[]int16:Int16Slice *[]int32:Int32Slice *[]int8:Int8Slice *[]string:StringSlice *[]time.Duration:DurationSlice *[]uint:UintSlice *[]uint16:Uint16Slice *[]uint32:Uint32Slice *[]uint8:Uint8Slice *bool:Bool *float32:Float32 *float64:Float64 *int:Int *int16:Int16 *int32:Int32 *int8:Int8 *string:String *time.Duration:Duration *uint:Uint *uint16:Uint16 *uint32:Uint32 *uint8:Uint8]' [recovered]
            panic: the method '*[]repository.Tag' must be part of 'map[*[]bool:BoolSlice *[]float32:Float32Slice *[]float64:Float64Slice *[]int:IntSlice *[]int16:Int16Slice *[]int32:Int32Slice *[]int8:Int8Slice *[]string:StringSlice *[]time.Duration:DurationSlice *[]uint:UintSlice *[]uint16:Uint16Slice *[]uint32:Uint32Slice *[]uint8:Uint8Slice *bool:Bool *float32:Float32 *float64:Float64 *int:Int *int16:Int16 *int32:Int32 *int8:Int8 *string:String *time.Duration:Duration *uint:Uint *uint16:Uint16 *uint32:Uint32 *uint8:Uint8]'
    
    opened by cifren 6
  • Query json for array size.

    Query json for array size.

    Hello,

    This is an "info request".

    I have the following json and I would like to check if the input contains a father (fid) with two children.

    {
      "data": [
        {
          "fid": 10,
          "children": [
            {
              "cid": 123
            },
            {
              "cid": 124
            }
           
          ]
        }
      ]
    }
    

    The code so far is

    package main
    
    import (
    	"fmt"
    
    	"github.com/thedevsaddam/gojsonq"
    )
    
    const json = `{
    	"data": [
    	  {
    		"fid": 10,
    		"children": [
    		  {
    			"cid": 123
    		  },
    		  {
    			"cid": 124
    		  }
    		 
    		]
    	  }
    	]
      }`
    
    func main() {
    
    	q := gojsonq.New().JSONString(json).From("data").
    		Where("fid", "=", 10)
    	fmt.Println(q.Get())
    }
    
    

    How can I modify the source code to decide if fid=10 AND children.size() = 2?

    In principle how can I use the "array size" to query the json file/input?

    Thanks

    enhancement question discussion priority-mid 
    opened by lefscode 6
  • Map[string]interface{} root

    Map[string]interface{} root

    Doesn't seem to work when you don't have nicely sliceable structures.

    IE: https://www.iinet.net.au/status/event/events.json

    I tried Where("DRW", "=", "Darwin") but kept getting nothing back

    enhancement 
    opened by freman 5
  • FromInterface custom decoder

    FromInterface custom decoder

    Hi there, Is there any way to load JSON document from map[string]interface{}? Simplest use case is querying MongoDB, which returns result documents in that format(without using schema struct). I am willing to make a PR if little help is supplied. Thanks

    Feature request 
    opened by Dragomir-Ivanov 3
  • Go-critic warnings

    Go-critic warnings

    go-critic linter found some issues , here corresponding log:

    check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 0 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 1 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 2 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 3 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 4 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 5 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\helper.go:45:2: typeS witchVar: case 6 can benefit from type switch with assignment check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\jsonq.go:271:3: range ValCopy: each iteration copies 48 bytes (consider pointers or indexing) check-package: $GOPATH\src\github.com\thedevsaddam\gojsonq\jsonq.go:45:18: typeU nparen: could simplify to [][]query

    In #18 I propose fixing for them

    Why replacerange with for loop? You must create an issue regarding this refactor before sending PR.

    Thanks, good point. The way that I made in my PR not very accurate and should be replaced into this one as written in description to rangeValCopy.

    opened by Kvaz1r 3
  • Added documentation and examples

    Added documentation and examples

    I saw a post on r/golang probably yours. And I wondered why you did not add the examples in the linked article to your project, so I did add some and of your README-file. These will be shown on https://godoc.org/github.com/thedevsaddam/gojsonq the godoc page.

    You have to edit the import-path of the examples_test.go file to your repo as I plan to remove my fork as soon as the changes are merged.

    opened by D1CED 3
  • Why interfaces?

    Why interfaces?

    Thanks for sharing this project. One question. Why do you do this:

    // strStrictContains checks if x contains y
    // This is case sensitive search
    func strStrictContains(x, y interface{}) (bool, error) {
    	xv, okX := x.(string)
    	if !okX {
    		return false, fmt.Errorf("%v must be string", x)
    	}
    	yv, okY := y.(string)
    	if !okY {
    		return false, fmt.Errorf("%v must be string", y)
    	}
    	return strings.Contains(xv, yv), nil
    }
    

    ...and not this:

    // strStrictContains checks if x contains y
    // This is case sensitive search
    func strStrictContains(x, y string) (bool) {
    	return strings.Contains(xv, yv)
    }
    

    ...or in this case, simply strings.Contains(xv, yv) without a wrapper.

    In this example, you are only allowing string inputs anyway, so why not use strongly typed strings as input params instead of interface{}?

    opened by nkev 2
  • Support for unselect

    Support for unselect

    I have an array of JSON objects which all contain a few common keys and then a whole bunch of keys that are different. I want to select a subset of this array that drops a single common key.

    The only way to do that right now is to use a select or only with all the possible keys the object can contain minus the one I want to drop. It would be nice to support something like "unselect" or "drop" where I can drop a specific key from the result while keeping the rest.

    opened by gboor 2
  • Unable to sort multiple fields with OrderBy

    Unable to sort multiple fields with OrderBy

    Supposed that we have list of objects which have internalScore and externalScore. OrderBy can only sort one field only, either those two. If there are 2 records having the same internalScore, when it's already sorted by internalScore, then externalScore cannot be sorted. So, at the moment OrderBy cannot do this.

    I wish it allowed us to call OrderBy multiple times which adds up instead of overriding it.

    It's similar to SQL functionality:

    SELECT * FROM tbl_Employee ORDER BY lastName ASC, Salary DESC
    
    opened by sancho21 0
  • Is it possible to parse multiple json together?

    Is it possible to parse multiple json together?

    Team,

    I have tried parsing this json

    {
      "module": {
        "ABCD": {
          "source": "",
          "version": "",
          "location": "",
          "resource_group_name": ""
        }
      },
      "module": {
        "PQRS": {
          "source": "",
          "version": "",
          "sa_name": "",
          "resource_group": ""
        }
      }
    }
    

    But I'm always getting the details of the last node i.e. PQRS. Is there anyway to get the details of both the nodes considering parent node as modules. The output required is

    {
     "ABCD": {
          "source": "",
          "version": "",
          "location": "",
          "resource_group_name": ""
        },
    "PQRS": {
          "source": "",
          "version": "",
          "sa_name": "",
          "resource_group": ""
        }
    }
    
    opened by smartaquarius10 0
  • Possible query

    Possible query

    is it possible to compare a query with 2 keys? I tried 2 3 tests, but it didn't work for me.

    {
        "authorizedWriters": {
            "teams": [],
            "users": []
        },
        "created": 1627654462505,
        "creator": "E7jJBMMAEAA",
        "dashboards": ["E7jJCz8AEAA"],
        "lastUpdated": 1627654462744,
        "lastUpdatedBy": "E7jJBMMAEAA",
        "teams": []
    }
    

    What i try to do:

    // give me all objects which have a different updater like creator
    // if creator == lastUpdatedBy
    
    json:= gojsonq.New().JSONString(json).From("results").Select("dashboardId","name").Where(.....?????)
    

    greetings

    opened by AchimGrolimund 0
  • result is []interface

    result is []interface

    this jsonStr ="{"uid":17766,"openid":"ni8vhveqn6","appOpenId":"nq46kiae15","unionId":"i2jki7ttm2","nickName":"","avatarUrl":"","gender":"0","cpc":"","wxappShareImg":"","inviteCode":"","tel":"","loginTime":1603268944,"status":0,"loginIP":"","platform":"","channel":"","versionCode":"","loginType":1,"CreatedAt":"2020-10-21T16:29:05+08:00","UpdatedAt":"2020-10-21T16:29:05+08:00"}"`

    i want get uid and openid ,why my get is 【】interface

    userJsonq := gojsonq.New().FromString(strJson) userRes := userJsonq.Select("uid","openid").Get()

    opened by huanBird 1
  • Can SUM func Support DOT Operation

    Can SUM func Support DOT Operation

    For example, I have a json string like [{"s":1, "a": {"b":1}},{"s":1, "a": {"b":1}},{"s":1, "a": {"b":5}}] I want jq.Sum("a.b"), Is this possible?

    opened by lnn1988 0
Releases(v2.5.2)
  • v2.5.2(Mar 27, 2020)

  • v2.5.1(Mar 11, 2020)

  • v2.5.0(Jan 13, 2020)

  • v2.4.0(Nov 20, 2019)

    Result.As() supports all the available method in Result type except for the methods with an argument like Time

    result, _ := gojsonq.New().FromString(jsonStr).From("prices").GetR() // handle error
    var prices []float32
    result.As(&prices) // handle error
    fmt.Printf("%#v\n", prices)
    
    Source code(tar.gz)
    Source code(zip)
  • v2.3.0(Sep 23, 2019)

    The Result instance will return different methods to get static type.

    const json = `{"name":{"first":"Tom","last":"Hanks"},"age":61}`
    result, err := gojsonq.New().JSONString(json).FindR("name.first")
    if err != nil {
    	log.Fatal(err)
    }
    name, _ := result.String()
    fmt.Println(name)
    
    Source code(tar.gz)
    Source code(zip)
  • v2.2.1(May 7, 2019)

    Fix select attributes for First,Last & Nth methods

    res := gojsonq.New().File("./data.json").From("items").Select("name", "price").First()
    fmt.Printf("%#v\n", res)
    

    Output

    map[string]interface {}{"name":"MacBook Pro 13 inch retina", "price":1350}
    
    Source code(tar.gz)
    Source code(zip)
  • v2.2.0(Apr 5, 2019)

    Offset

    gojsonq.New().File("./data.json").From("items").Offset(10).Limit(10).Get()
    

    Writer

    gojsonq.New().File("./data.json").From("items").SortBy("price").Writer(os.Stdout)
    

    FromString

    gojsonq.New().FromString(jsonStr).From("items").Get()
    
    Source code(tar.gz)
    Source code(zip)
  • v2.0.1(Feb 20, 2019)

  • v2.0.0(Feb 20, 2019)

  • v1.9.1(Dec 2, 2018)

  • v1.9.0(Feb 20, 2019)

Owner
Saddam H
Sr. Software Engineer @ Pathao Inc | Open Source Enthusiast | Love to write elegant code | Golang | Distributed Systems
Saddam H
JSON query expression library in Golang.

jsonql JSON query expression library in Golang. This library enables query against JSON. Currently supported operators are: (precedences from low to h

Qian Chen 253 Aug 11, 2022
Simple filter query language parser so that you can build SQL, Elasticsearch, etc. queries safely from user input.

fexpr fexpr is a filter query language parser that generates extremely easy to work with AST structure so that you can create safely SQL, Elasticsearc

Gani Georgiev 22 Aug 3, 2022
Resource Query Language for REST

RQL is a resource query language for REST. It provides a simple and light-weight API for adding dynamic querying capabilities to web-applications that

Ariel Mashraki 261 Jul 12, 2022
Query Parser for REST

Query Parser for REST Query Parser is a library for easy building dynamic SQL queries to Database. It provides a simple API for web-applications which

Tim S. 42 Aug 6, 2022
GSQL is a structured query language code builder for golang.

GSQL is a structured query language code builder for golang.

Leon Ding 107 Jun 5, 2022
json slicer

JSON Slice What is it? JSON Slice is a Go package which allows to execute fast jsonpath queries without unmarshalling the whole data. Sometimes you ne

Michael Gurov 68 Jul 27, 2022
This app is an attempt towards using go lang with graphql data fetch in react front end.

go_movies _A React js + GraphQL supported with backend in GoLang. This app is an attempt towards using go lang with graphql data fetch in react front

Abhijit Mukherjee 0 Dec 7, 2021
A simple Go, GraphQL, and PostgreSQL starter template

Simple Go/GraphQL/PostgreSQL template Purpose Have a good starting point for any project that needs a graphql, go, and postgres backend. It's a very l

Chris 0 Jan 8, 2022
A simple 2D demo in go using embiten game engine

a little demo in go using the ebiten game engine vaguely follows my memory of ho

null 0 Dec 29, 2021
A simple (yet effective) GraphQL to HTTP / REST router

ReGraphQL A simple (yet effective) GraphQL to REST / HTTP router. ReGraphQL helps you expose your GraphQL queries / mutations as REST / HTTP endpoints

Ezequiel Aceto 4 May 26, 2022
A simple Go package to Query over JSON/YAML/XML/CSV Data

A simple Go package to Query over JSON Data. It provides simple, elegant and fast ODM like API to access, query JSON document Installation Install the

Saddam H 1.9k Aug 4, 2022
Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.

dasel Dasel (short for data-selector) allows you to query and modify data structures using selector strings. Comparable to jq / yq, but supports JSON,

Tom Wright 3.5k Aug 9, 2022
Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.

dasel Dasel (short for data-selector) allows you to query and modify data structures using selector strings. Comparable to jq / yq, but supports JSON,

Tom Wright 3.5k Aug 3, 2022
converts text-formats from one to another, it is very useful if you want to re-format a json file to yaml, toml to yaml, csv to yaml, ... etc

re-txt reformates a text file from a structure to another, i.e: convert from json to yaml, toml to json, ... etc Supported Source Formats json yaml hc

Mohammed Al Ashaal 67 Jun 8, 2022
Dasel - Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool.

Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool. Supports conversion between formats and can be used as a Go package.

Tom Wright 3.6k Aug 15, 2022
Prometheus Common Data Exporter can parse JSON, XML, yaml or other format data from various sources (such as HTTP response message, local file, TCP response message and UDP response message) into Prometheus metric data.

Prometheus Common Data Exporter Prometheus Common Data Exporter 用于将多种来源(如http响应报文、本地文件、TCP响应报文、UDP响应报文)的Json、xml、yaml或其它格式的数据,解析为Prometheus metric数据。

null 7 May 18, 2022
Universal JSON, BSON, YAML, CSV, XML converter with templates

Universal JSON, BSON, YAML, CSV, XML translator to ANY format using templates Key features Various input formats (json, bson, yaml, csv, xml) Flexible

Mario 59 Jul 28, 2022
Simple, lightweight and faster response (JSON, JSONP, XML, YAML, HTML, File) rendering package for Go

Package renderer Simple, lightweight and faster response (JSON, JSONP, XML, YAML, HTML, File) rendering package for Go Installation Install the packag

Saddam H 241 Jul 3, 2022
yaml-patch is a version of Evan Phoenix's json-patch, which is an implementation of JSON Patch, directly transposed to YAML

yaml-patch yaml-patch is a version of Evan Phoenix's json-patch, which is an implementation of JavaScript Object Notation (JSON) Patch, directly trans

Steve Coffman 0 Jan 15, 2022
XPath package for Golang, supports HTML, XML, JSON document query.

XPath XPath is Go package provides selecting nodes from XML, HTML or other documents using XPath expression. Implementation htmlquery - an XPath query

null 504 Aug 12, 2022
Go encoding/xml package that improves support for XML namespaces

encoding/xml with namespaces This is a fork of the Go encoding/xml package that improves support for XML namespaces, kept in sync with golang/go#48641

nb.io 3 Dec 1, 2021
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.

omniparser Omniparser is a native Golang ETL parser that ingests input data of various formats (CSV, txt, fixed length/width, XML, EDI/X12/EDIFACT, JS

JF Technology 484 Aug 12, 2022
Go package for easily rendering JSON, XML, binary data, and HTML templates responses.

Render Render is a package that provides functionality for easily rendering JSON, XML, text, binary data, and HTML templates. This package is based on

Cory Jacobsen 1.7k Aug 4, 2022
Data visualization with chart, Create CSV file, Read Write CSV file

Data visualization with chart, Create CSV file, Read Write CSV file, Read from json file and many more in single project ......

MD FAISAL PORAG 1 Jan 13, 2022
Gosaxml is a streaming XML decoder and encoder, similar in interface to the encoding/xml

gosaxml is a streaming XML decoder and encoder, similar in interface to the encoding/xml, but with a focus on performance, low memory footprint and on

HBT Hamburger Berater Team GmbH 4 Jul 26, 2022
xmlquery is Golang XPath package for XML query.

xmlquery Overview xmlquery is an XPath query package for XML documents, allowing you to extract data or evaluate from XML documents with an XPath expr

null 301 Aug 1, 2022
`go-redash-query` is a simple library to get structed data from `redash query` sources

go-redash-query go-redash-query is a simple library to get structed data from redash query sources Example Source table id name email 1 Dannyhann rhrn

danny han 3 May 22, 2022
Simple DB using yaml. A project for managing the content of yaml files.

DB Yaml Simple DB using yaml. A project for managing the content of yaml files. Table of Contents DB Yaml Features Usage Write to DB Query DB Get Firs

Christos Kotsis 49 Jul 18, 2022
This library provides an ASTERIX Frame(binary data) decoding/parsing(json,xml) capabilities for Go.

GoAsterix This library provides an ASTERIX Frame(binary data) decoding/parsing(json,xml) capabilities for Go. ASTERIX ASTERIX (All Purpose Structured

MMK 3 Jun 1, 2022