JSONL graph tools - Graph is represented as JSONL of nodes and edges.

Overview

JSONL graph tools

Convenient to use with jq

Go Reference Go Report Card codecov Tests Awesome

$ go install github.com/nikolaydubina/[email protected]
# get https://graphviz.org/download/

Graph is represented as JSONL of nodes and edges.

Node contains id and any fields

{
    "id": "github.com/gin-gonic/gin",
    "can_get_git": true,
    "can_run_tests": true,
    "can_get_github": true,
    "github_url": "https://github.com/gin-gonic/gin",
    "git_url": "https://github.com/gin-gonic/gin",
    "git_last_commit": "2021-04-21",
    "git_last_commit_days_since": 4,
    "git_num_contributors": 321,
    ...
}

Edge contains from and to with node ids

{
    "from": "github.com/gin-gonic/gin",
    "to": "golang.org/x/tools"
}

Why?

JSONL is a perfect fit for storing graphs

  • can append new nodes and endges by concatenating files
  • nodes and edges can have any data
  • schemaless
  • any subset of lines is a valid graph

Examples

To illustrate, I am using data from import-graph. If you pass color scheme, then values will be colored.

colored.svg ">
$ cat '
{"id":"github.com/gin-gonic/gin","can_get_git":true, ... }
{"id":"github.com/gin-contrib/sse","can_get_git":true,"can_run_tests":true ... }
...
{"from":"github.com/gin-gonic/gin","to":"golang.org/x/tools"}
{"from":"github.com/gin-gonic/gin","to":"github.com/go-playground/validator/v10"}
' | jsonl-graph -color-scheme=file://$PWD/docs/colors.json | dot -Tsvg > colored.svg

gin-color

By default, no coloring is applied.

basic.svg ">
$ cat '
{"id":"github.com/gin-gonic/gin","can_get_git":true, ... }
{"id":"github.com/gin-contrib/sse","can_get_git":true,"can_run_tests":true ... }
...
{"from":"github.com/gin-gonic/gin","to":"golang.org/x/tools"}
{"from":"github.com/gin-gonic/gin","to":"github.com/go-playground/validator/v10"}
' | jsonl-graph | dot -Tsvg > basic.svg

gin-nocolor

If nodes have less fields, then adjusting graph

small.svg ">
$ cat '
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"bufio"}
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"bytes"}
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"encoding/json"}
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"errors"}
{"from":"github.com/nikolaydubina/jsonl-graph/graph","to":"fmt"}
...
' | jsonl-graph | dot -Tsvg > small.svg

small

Issues
  • Add metro-style for graph without details

    Add metro-style for graph without details

    https://observablehq.com/@nitaku/tangled-tree-visualization-ii

    • can not use graphviz or dot
    • need to generate svg manually
    • optional: think if can extend with node details
    • optional: think is can integrate to UI website
    opened by nikolaydubina 0
  • Web UI

    Web UI

    • [ ] interactive web interface
    • [ ] collapse node
    • [ ] highlight node and edges
    • [ ] arragement aglrorith
    • [ ] edge shape algorithm
    • [ ] export / import color config to same tool as graphviz
    opened by nikolaydubina 0
Releases(v1.1.1)
Owner
Nikolay Dubina
Nikolay Dubina
pipe JSONL data to SIEM

SIEMSend UNIX philosophy inspired SIEM connector. The binary work is very simple: get a stream of JSON lines from stdin, send them in batches to a SIE

Ali Mosajjal 1 Mar 21, 2022
A tools to find the path of a specific key in deep nested JSON.

如何快速从深层嵌套 JSON 中找到特定的 Key #公众号 在爬虫开发的过程中,我们经常遇到一些 Ajax 加载的接口会返回 JSON 数据。

kingname 44 May 16, 2022
Fast JSON parser and validator for Go. No custom structs, no code generation, no reflection

fastjson - fast JSON parser and validator for Go Features Fast. As usual, up to 15x faster than the standard encoding/json. See benchmarks. Parses arb

Aliaksandr Valialkin 1.5k May 11, 2022
Fast and flexible JSON encoder for Go

Jettison Jettison is a fast and flexible JSON encoder for the Go programming language, inspired by bet365/jingo, with a richer features set, aiming at

William Poussier 126 May 13, 2022
/ˈdʏf/ - diff tool for YAML files, and sometimes JSON

dyff is inspired by the way the old BOSH v1 deployment output reported changes from one version to another by only showing the parts of a YAML file that change.

null 599 May 17, 2022
HuJSON: JSON for Humans (comments and trailing commas)

HuJSON - Human JSON The HuJSON decoder is a JSON decoder that also allows comments, both /* ... */ and // to end of line trailing commas on arrays and

Tailscale 374 May 15, 2022
tson is JSON viewer and editor written in Go

tson tson is JSON viewer and editor written in Go. This tool displays JSON as a tree and you can search and edit key or values. Support OS Mac Linux I

skanehira 127 Mar 17, 2022
A Small tool for SDWAN performance test and policy validation

sdwan-perf Sdwan-perf is based on golang and could support almost platform for performance and policy validation. SDWAN Performance Test Report +--

Mie~~~ 16 May 14, 2022
JSONata in Go Package jsonata is a query and transformation language for JSON

JSONata in Go Package jsonata is a query and transformation language for JSON. It's a Go port of the JavaScript library JSONata.

Blues Inc 24 Apr 27, 2022
Slow and unreliable JSON parser generator (in progress)

VivaceJSON Fast and reliable JSON parser generator Todo List parse fields parse types generate struct generate (keypath+key) to struct Value Mapping F

null 7 Mar 3, 2022
Package json implements encoding and decoding of JSON as defined in RFC 7159

Package json implements encoding and decoding of JSON as defined in RFC 7159. The mapping between JSON and Go values is described in the documentation for the Marshal and Unmarshal functions

High Performance, Kubernetes Native Object Storage 3 May 10, 2022
Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang

kazaam Description Kazaam was created with the goal of supporting easy and fast transformations of JSON data with Golang. This functionality provides

Qntfy 205 Sep 17, 2021
Golang JSON decoder supporting case-sensitive, number-preserving, and strict decoding use cases

Golang JSON decoder supporting case-sensitive, number-preserving, and strict decoding use cases

Kubernetes SIGs 12 Apr 3, 2022
Easy JSON parsing, stringifying, and accesing

Easy JSON parsing, stringifying, and accesing

Neutron 1 Nov 23, 2021
json encoding and decoding

jx Package jx implements encoding and decoding of json [RFC 7159]. Lightweight fork of jsoniter. go get github.com/go-faster/jx Usage and examples Roa

go faster 53 May 8, 2022
A tool to aggregate and mine data from JSON reports of Go tests.

teststat A tool to aggregate and mine data from JSON reports of Go tests. Why? Mature Go projects often have a lot of tests, and not all of them are i

Viacheslav Poturaev 3 Jan 3, 2022
Go encoder and decoder for the NetBPM/PNM image formats. Compatible with Go's image packages.

gpnm This package implements an encoder and decoder for PNM image formats. It can be used with Go's image library. It covers all formats as defined by

null 0 Nov 26, 2021
Search and output the value of JSON by it's path.

Search and output the value of JSON by it's path.

Alexander Dawson 0 Dec 19, 2021
A simple GO module providing CRUD and match methods on a User "entity" stored locally as JSON

A simple GO module providing CRUD and match methods on a User "entity" stored locally as JSON. Created for GO language learning purposes. Once finishe

Lorenzo Piersante 0 Feb 5, 2022