Automatically generate Go (golang) struct definitions from example JSON

Overview

Build Status gojson

gojson generates go struct definitions from json or yaml documents.

Example

$ curl -s https://api.github.com/repos/chimeracoder/gojson | gojson -name=Repository

package main

type Repository struct {
	ArchiveURL       string      `json:"archive_url"`
	AssigneesURL     string      `json:"assignees_url"`
	BlobsURL         string      `json:"blobs_url"`
	BranchesURL      string      `json:"branches_url"`
	CloneURL         string      `json:"clone_url"`
	CollaboratorsURL string      `json:"collaborators_url"`
	CommentsURL      string      `json:"comments_url"`
	CommitsURL       string      `json:"commits_url"`
	CompareURL       string      `json:"compare_url"`
	ContentsURL      string      `json:"contents_url"`
	ContributorsURL  string      `json:"contributors_url"`
	CreatedAt        string      `json:"created_at"`
	DefaultBranch    string      `json:"default_branch"`
	Description      string      `json:"description"`
	DownloadsURL     string      `json:"downloads_url"`
	EventsURL        string      `json:"events_url"`
	Fork             bool        `json:"fork"`
	Forks            float64     `json:"forks"`
	ForksCount       float64     `json:"forks_count"`
	ForksURL         string      `json:"forks_url"`
	FullName         string      `json:"full_name"`
	GitCommitsURL    string      `json:"git_commits_url"`
	GitRefsURL       string      `json:"git_refs_url"`
	GitTagsURL       string      `json:"git_tags_url"`
	GitURL           string      `json:"git_url"`
	HasDownloads     bool        `json:"has_downloads"`
	HasIssues        bool        `json:"has_issues"`
	HasWiki          bool        `json:"has_wiki"`
	Homepage         interface{} `json:"homepage"`
	HooksURL         string      `json:"hooks_url"`
	HtmlURL          string      `json:"html_url"`
	ID               float64     `json:"id"`
	IssueCommentURL  string      `json:"issue_comment_url"`
	IssueEventsURL   string      `json:"issue_events_url"`
	IssuesURL        string      `json:"issues_url"`
	KeysURL          string      `json:"keys_url"`
	LabelsURL        string      `json:"labels_url"`
	Language         string      `json:"language"`
	LanguagesURL     string      `json:"languages_url"`
	MasterBranch     string      `json:"master_branch"`
	MergesURL        string      `json:"merges_url"`
	MilestonesURL    string      `json:"milestones_url"`
	MirrorURL        interface{} `json:"mirror_url"`
	Name             string      `json:"name"`
	NetworkCount     float64     `json:"network_count"`
	NotificationsURL string      `json:"notifications_url"`
	OpenIssues       float64     `json:"open_issues"`
	OpenIssuesCount  float64     `json:"open_issues_count"`
	Owner            struct {
		AvatarURL         string  `json:"avatar_url"`
		EventsURL         string  `json:"events_url"`
		FollowersURL      string  `json:"followers_url"`
		FollowingURL      string  `json:"following_url"`
		GistsURL          string  `json:"gists_url"`
		GravatarID        string  `json:"gravatar_id"`
		HtmlURL           string  `json:"html_url"`
		ID                float64 `json:"id"`
		Login             string  `json:"login"`
		OrganizationsURL  string  `json:"organizations_url"`
		ReceivedEventsURL string  `json:"received_events_url"`
		ReposURL          string  `json:"repos_url"`
		SiteAdmin         bool    `json:"site_admin"`
		StarredURL        string  `json:"starred_url"`
		SubscriptionsURL  string  `json:"subscriptions_url"`
		Type              string  `json:"type"`
		URL               string  `json:"url"`
	} `json:"owner"`
	Private         bool    `json:"private"`
	PullsURL        string  `json:"pulls_url"`
	PushedAt        string  `json:"pushed_at"`
	Size            float64 `json:"size"`
	SshURL          string  `json:"ssh_url"`
	StargazersURL   string  `json:"stargazers_url"`
	StatusesURL     string  `json:"statuses_url"`
	SubscribersURL  string  `json:"subscribers_url"`
	SubscriptionURL string  `json:"subscription_url"`
	SvnURL          string  `json:"svn_url"`
	TagsURL         string  `json:"tags_url"`
	TeamsURL        string  `json:"teams_url"`
	TreesURL        string  `json:"trees_url"`
	UpdatedAt       string  `json:"updated_at"`
	URL             string  `json:"url"`
	Watchers        float64 `json:"watchers"`
	WatchersCount   float64 `json:"watchers_count"`
}

CLI Installation

$ go get github.com/ChimeraCoder/gojson/gojson

Assuming $GOPATH/bin is in your PATH, you can now invoke gojson directly.

API Installation

$ go get github.com/ChimeraCoder/gojson/gojson

Development

$ git clone https://github.com/ChimeraCoder/gojson.git
$ cd gojson
$ go test

Building CLI

$ go build -o _build/gojson ./gojson

Installing CLI

$ go install ./gojson

Formatting

$ gofmt -w -e -s -l .

Related Work

github.com/str1ngs/jflect

License

gojson is free software distributed under Version 3 of the GNU Public License.

As of the time of writing, this is the same license used for gcc (and therefore gccgo), so it is unlikely to restrict use in any way. Note that the GPL does not extend to any output generated by gojson; the GPL only applies to software which includes copies of gojson itself.

Issues
  • Changes how whole numbers are handled in YAML

    Changes how whole numbers are handled in YAML

    Summary

    This changes gojson’s behavior with YAML output so that it doesn’t try to infer whether YAML integers are floats.

    Also reformatted a docstring comment to appease the linter gods.

    Motivation

    Since YAML’s spec includes unambiguous tagging for integers and floats, we don’t need to use disambiguateFloatInt to turn whole numbers from floats to integers.

    Testing

    Added tests for this behavior as well!

    Hope this is helpful @ChimeraCoder – it unfortunately doesn’t really solve the problem of #44 :(

    opened by joshu-stripe 8
  • Removing the vendor directory

    Removing the vendor directory

    I personally think that the vendor directory should be removed, because it'll be either duplicated with the current YAML package that we currently have, or out of sync with it.

    Either case, it make little sense to keep it there, IMO.

    Please consider. Thx.

    opened by suntong 3
  • added yaml support

    added yaml support

    I made a few changes and I'm not sure if you want them or not. The biggest change is adding support for generating structs from either json or yaml. You can also specify what you want to tag the struct with (in case you want both json and yaml tags, or something else like bson or whatever). I updated the struct field naming to change names in all caps with underscores to be camel case. So TEST_FIELD_NAME becomes TestFieldName instead of TESTFIELDNAME.

    I'm not sure if the yaml support interests you or not, so feel free to ignore this if it doesn't and I'll keep my fork around for anyone who wants it.

    opened by darrennoble 3
  • Allow dashes in JSON keys

    Allow dashes in JSON keys

    Via cgtdk on #go-nuts, this error currently occurs:

    ### JSON DATA (THIS LINE NOT INCLUDED)
    {
      "query-continue": {
        "recentchanges": {
          "rcstart": "2013-03-18T20:20:30Z"
        }
      },
      "query": {
        "recentchanges": [
          {
            "type": "edit",
            "ns": 4,
            "title": "Wikipedia:Deletion review\/Log\/2013 March 18",
            "rcid": 563307455,
            "pageid": 38844023,
            "revid": 545266112,
            "old_revid": 545265931,
            "timestamp": "2013-03-18T20:20:30Z"
          },
          {
            "type": "edit",
            "ns": 0,
            "title": "Pope John IV",
            "rcid": 563307454,
            "pageid": 243546,
            "revid": 545266111,
            "old_revid": 545261452,
            "timestamp": "2013-03-18T20:20:30Z"
          }
        ]
      }
    }
    ### END JSON DATA
    
    ### RESULT OF USING GOJSON:
    [email protected]:~$ gojson -file="mwdata.json"
    panic: 3:7: expected ';', found '-'
    
    goroutine 1 [running]:
    main.main()
        /home/chris/code/go/src/github.com/ChimeraCoder/gojson/gojson.go:97 +0x126
    
    goroutine 2 [syscall]:
    created by runtime.main
        /build/buildd/golang-1.0.2/src/pkg/runtime/proc.c:221
    [email protected]:~$
    

    The easiest solution is probably to replace - with _.

    opened by ChimeraCoder 3
  • Ignore tests in vendor directory for older Go versions

    Ignore tests in vendor directory for older Go versions

    ./... matches the vendor/ directory in Go, which causes spurious test failures in the yaml package we use.

    This also drops tests for 1.5 and 1.6 on OS X, because the Travis OS X image for those is not working. 1.5 and 1.6 are still tested on Linux and Windows.

    opened by ChimeraCoder 2
  • Stringify first char of key name, should it be a int.

    Stringify first char of key name, should it be a int.

    This was reported to me as an error by someone using http://json2struct.herokuapp.com/. Specific example is parsing http status code information, e.g.:

    {
       "200": 1000,
       "3xx": 10,
        // ... etc ...
    }
    

    Expected results with this change would be:

    type Foo struct {
        Two00   int `json:"200"`
        ThreeXx int `json:"3xx"`
        // ... etc ...
    }
    
    opened by jmervine 2
  • Updating for use as an imported module in addition to cli.

    Updating for use as an imported module in addition to cli.

    Functionality is identical, however, build instructions are slightly different, see Makefile for details.

    Enables usage like that of https://github.com/jmervine/gojson-http.

    Thanks, J

    opened by jmervine 2
  • Generate better substruct names

    Generate better substruct names

    I just started using the -subStruct flag because I need to be able to append sub-structs to arrays. It works great, thanks!

    The name it gives the sub-structs could be better. I understand you'd want to avoid struct name collisions but even something like OuterStruct_InnerStruct would be better and more useful than the current _sub# method. Especially with IDE autocomplete. I can choose from 4 sub-structs but without going and seeing which is which I don't actually know. Also I'm assuming that if the order in the JSON file changed the sub indexing would get messed up and everything would break.

    I'll take a look at submitting a PR soon-ish of my idea.

    opened by bhechinger 1
  • use element's json path (without path separator) as struct name

    use element's json path (without path separator) as struct name

    As a proposal to #14, I think this will generate better struct names while avoiding the issue of different structs with the same name overriding each other.

    Let me know what you think :)

    opened by lobatt 1
  • Add -forcefloats flag

    Add -forcefloats flag

    Adds a -forcefloats flag. Numbers that are integral values are turned into type int64, except if the -forcefloat flag is passed, in which case numeric values are all interpreted as float64.

    This is functionally identical to https://github.com/ChimeraCoder/gojson/pull/22 by @e-dard.

    opened by ChimeraCoder 1
  • Parse floats as float64

    Parse floats as float64

    {
        "duration": 123.0,
        "timestamp" 1234
    }
    

    should provide

    type Trace struct {
       Duration float64
       Timestamp int64
    }
    

    instead of int, int

    opened by ChimeraCoder 1
  • Can we export output as struct variable

    Can we export output as struct variable

    I want to use this structure variable when the program is running, but the result is stored in the file. I can't predict it in advance, which will lead to compilation failure. Is there any way

    opened by gaofeng-lin 0
  • Add support to time.Time

    Add support to time.Time

    Hi, this is a request to extend the struct generation to support time fields. If the string matches the ISO 8601 / RFC 3339 patterns, like:

    • 2021-07-28T12:34:56Z
    • 2021-07-28T16:34:56-04:00

    Then the type should time.Time instead of string. Thanks for the consideration.

    opened by cassioconti 0
  • Allow omiting package name when generating structs

    Allow omiting package name when generating structs

    This depends on my previous PR and is branched of it: https://github.com/ChimeraCoder/gojson/pull/78

    I'm writing an integration to a thirdpary application. I'm using this library to generate structs using example inputs from the thirdparty application. The below code generates these structs and places them in one 1 file.

    The change I'm proposing allows these to be put into 1 file, otherwise I'd have to split every struct definition into separate files, this clutters a lot.

    The folder api_definitions has example output files from my thirdparty application, like customers.json and articles.json.

    ./generate.go

    package main
    
    import (
    	"io/ioutil"
    	"os"
    	"strings"
    
    	"github.com/ChimeraCoder/gojson"
    )
    
    //go:generate go run generate.go
    
    func main() {
    	const definitionsPath = "internal/third_pary/api_definitions"
    	const typesPath = "internal/third_pary/"
    
    	files, err := ioutil.ReadDir(definitionsPath)
    	if err != nil {
    		panic(err)
    	}
    	allStructs := []byte{}
    	packageIsWritten := false
    	for _, f := range files {
    		file, err := os.Open(definitionsPath + "/" + f.Name())
    		if err != nil {
    			panic(err)
    		}
    		packageName := "third_pary"
    		if packageIsWritten {
    			packageName = ""
    		}
    		structName := strings.Split(f.Name(), ".")[0]
    		output, err := gojson.Generate(file, gojson.ParseJson, structName, packageName, []string{"json"}, false, true)
    		if err != nil {
    			panic(err)
    		}
    		packageIsWritten = true
    		allStructs = append(allStructs, output...)
    	}
    
    	goStructFile, err := os.Create(typesPath + "/" + "api_types.go")
    	if err != nil {
    		panic(err)
    	}
    	_, err = goStructFile.Write(allStructs)
    	if err != nil {
    		panic(err)
    	}
    }
    
    
    #I then run this to generate my file with all the structs contained therein.
    go generate ./...
    
    opened by jhzn 0
  • -forcefloats flag does not work

    -forcefloats flag does not work

    It does not work. Input:

    echo '{"value": 9.00}' | gojson -forcefloats
    

    and the output is:

    type Foo struct {
    	Value int64 `json:"value"`
    }
    

    The -forcefloats flag does not work.

    opened by sheng-di 1
Owner
Aditya Mukerjee
Aditya Mukerjee
json to rpc example with envoy, go, grpc, redis

grpc-redis-envoy-example json to rpc example with envoy, go, grpc, redis Run Make sure you have docker installed locally Run the services docker-com

Charles Onunze 4 Feb 10, 2022
An example of how to parse json data using go....a typical step for preparing data prior to uploading to a db.

JSON parser using GO An example of parsing json data in go, when you already know the schema of the data Example input: { "num_listings":"36",

null 0 Jan 12, 2022
A convenient syntax to generate JSON (or YAML) for commandline

clon A convenient syntax to generate JSON (or YAML) for commandline "mumbo-jumbo". Syntax Overview Syntax resembles that of JSON with a few caveats: a

Luca Sepe 9 May 14, 2021
Generate Jsonnet definition for JSON representation of protobuf object

Generate Jsonnet definition for JSON representation of protobuf object

Manatsawin Hanmongkolchai 1 Nov 1, 2021
Get JSON values quickly - JSON parser for Go

get json values quickly GJSON is a Go package that provides a fast and simple way to get values from a json document. It has features such as one line

Josh Baker 10.5k Jun 30, 2022
JSON diff library for Go based on RFC6902 (JSON Patch)

jsondiff jsondiff is a Go package for computing the diff between two JSON documents as a series of RFC6902 (JSON Patch) operations, which is particula

William Poussier 172 Jun 23, 2022
Fast JSON encoder/decoder compatible with encoding/json for Go

Fast JSON encoder/decoder compatible with encoding/json for Go

Masaaki Goshima 1.6k Jun 26, 2022
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

High Performance, Kubernetes Native Object Storage 2 Jun 26, 2022
Json-go - CLI to convert JSON to go and vice versa

Json To Go Struct CLI Install Go version 1.17 go install github.com/samit22/js

Samit Ghimire 5 Mar 3, 2022
JSON Spanner - A Go package that provides a fast and simple way to filter or transform a json document

JSON SPANNER JSON Spanner is a Go package that provides a fast and simple way to

null 3 Jun 30, 2022
Example to validate performance using append or not in golang

benchtest-arr-go This code is a example to validate performance using append or not in golang result benchtests go test -benchmem -bench . goos: darwi

Renan Bastos 1 Jan 10, 2022
Abstract JSON for golang with JSONPath support

Abstract JSON Abstract JSON is a small golang package provides a parser for JSON with support of JSONPath, in case when you are not sure in its struct

Stepan Pyzhov 119 Jun 24, 2022
JSON query in Golang

gojq JSON query in Golang. Install go get -u github.com/elgs/gojq This library serves three purposes: makes parsing JSON configuration file much easie

Qian Chen 182 Apr 27, 2022
Arbitrary transformations of JSON in Golang

kazaam Description Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang. This functionality provides

Qntfy 230 Jun 17, 2022
Parsing JSON is a hassle in golang

GoJSON Parsing JSON is a hassle in golang. This package will allow you to parse and search elements in a json without structs. Install gojson go get g

swaraj18 25 Nov 12, 2021
Fast JSON serializer for golang.

easyjson Package easyjson provides a fast and easy way to marshal/unmarshal Go structs to/from JSON without the use of reflection. In performance test

Free and open source software developed at Mail.Ru 3.8k Jun 22, 2022
Fastest JSON interperter for golang

Welcome To JIN "Your wish is my command" Fast and Easy Way to Deal With JSON Jin is a comprehensive JSON manipulation tool bundle. All functions teste

eco 58 May 28, 2022
Fast Color JSON Marshaller + Pretty Printer for Golang

ColorJSON: The Fast Color JSON Marshaller for Go What is this? This package is based heavily on hokaccha/go-prettyjson but has some noticible differen

Tyler Brock 110 Jun 16, 2022
Golang port of simdjson: parsing gigabytes of JSON per second

This is a Golang port of simdjson, a high performance JSON parser developed by Daniel Lemire and Geoff Langdale. It makes extensive use of SIMD instructions to achieve parsing performance of gigabytes of JSON per second.

High Performance, Kubernetes Native Object Storage 1.3k Jun 25, 2022