Command-line JSON processor

Related tags

jq
Overview

jq

jq is a lightweight and flexible command-line JSON processor.

Coverage Status, Unix: Build Status, Windows: Windows build status

If you want to learn to use jq, read the documentation at https://stedolan.github.io/jq. This documentation is generated from the docs/ folder of this repository. You can also try it online at jqplay.org.

If you want to hack on jq, feel free, but be warned that its internals are not well-documented at the moment. Bring a hard hat and a shovel. Also, read the wiki: https://github.com/stedolan/jq/wiki, where you will find cookbooks, discussion of advanced topics, internals, release engineering, and more.

Source tarball and built executable releases can be found on the homepage and on the github release page, https://github.com/stedolan/jq/releases

If you're building directly from the latest git, you'll need flex, bison (3.0 or newer), libtool, make, automake, and autoconf installed. To get regexp support you'll also need to install Oniguruma or clone it as a git submodule as per the instructions below. (note that jq's tests require regexp support to pass). To build, run:

git submodule update --init # if building from git to get oniguruma
autoreconf -fi              # if building from git
./configure --with-oniguruma=builtin
make -j8
make check

To build without bison or flex, add --disable-maintainer-mode to the ./configure invocation:

./configure --with-oniguruma=builtin --disable-maintainer-mode

(Developers must not use --disable-maintainer-mode, not when making changes to the jq parser and/or lexer.)

To build a statically linked version of jq, run:

make LDFLAGS=-all-static

After make finishes, you'll be able to use ./jq. You can also install it using:

sudo make install

If you're not using the latest git version but instead building a released tarball (available on the website), then you won't need to run autoreconf (and shouldn't), and you won't need flex or bison.

To cross-compile for OS X and Windows, see docs/Rakefile's build task and scripts/crosscompile. You'll need a cross-compilation environment, such as Mingw for cross-compiling for Windows.

Cross-compilation requires a clean workspace, then:

# git clean ...
autoreconf -i
./configure
make distclean
scripts/crosscompile <name-of-build> <configure-options>

Use the --host= and --target= ./configure options to select a cross-compilation environment. See also "Cross compilation" on the wiki.

Send questions to https://stackoverflow.com/questions/tagged/jq or to the #jq channel (http://irc.lc/freenode/%23jq/) on Freenode (https://webchat.freenode.net/).

Issues
  • Document the potential security issues when using `-r`/`-0`.

    Document the potential security issues when using `-r`/`-0`.

    Also document how -r does separators.

    Reported-by: @pcworld Reported-in: https://github.com/stedolan/jq/issues/1271#issuecomment-920835007

    opened by pabs3 0
  • Could you enable the sponsor button on this repo?

    Could you enable the sponsor button on this repo?

    I'm using jq for my SaaS and I thought it might be a not that bad idea to wire a few bucks to this project via GitHub sponsors.

    But it seems to be disabled on the repo.

    Would you be ok with turning it on?

    opened by fenske 1
  • Serious sync issues

    Serious sync issues

    Version: jq-1.6

    These commands:

    echo '{"version": "v1"}' > /tmp/eventsheet.json

    cat /tmp/eventsheet.json | jq --arg next_var "1" '. + {next_var: $next_var}' > /tmp/eventsheet.json2 && mv -f /tmp/eventsheet.json2 /tmp/eventsheet.json

    cat /tmp/eventsheet.json | jq '. + {next_var: 1}' > /tmp/eventsheet.json2 && mv -f /tmp/eventsheet.json2 /tmp/eventsheet.json

    cat /tmp/eventsheet.json | jq --arg vars "[]" '. + {vars: $vars}' > /tmp/eventsheet.json2 && mv -f /tmp/eventsheet.json2 /tmp/eventsheet.json

    cat /tmp/eventsheet.json | jq '. + {vars: []}' > /tmp/eventsheet.json2 && mv -f /tmp/eventsheet.json2 /tmp/eventsheet.json

    Have resulted in this:

    { "version": "v1", "next_var": 1, "vars": [] } }

    It have one more "}".

    The commands are correct, but it seems that jq have sync issues when writing to json files (excellent at reading but not so at writing).

    Its an Electron app, but I had similar write sync issues from this bash program: https://github.com/DaniellMesquita/Web3Updater

    @stedolan

    opened by DaniellMesquita 4
  • docs(manual): fix the typo errors in the manual

    docs(manual): fix the typo errors in the manual

    fix the manual typo errors in manual v1.6 and main manual file.

    opened by meizhaohui 0
  • Grammar railroad diagram

    Grammar railroad diagram

    Using this tool https://www.bottlecaps.de/convert/ we can copy and paste /jq/blob/master/src/parser.y in the textarea Input grammar: and click the button Convert and then click the button View Diagram to get a full grammar railroad diagram.

    opened by mingodad 0
  • Question about license.

    Question about license.

    I would like to create a winget package for jq, but I wanted to make sure I attributed the right license. The text looks like MIT. Is that in fact the case? If not, can you say specifically which license it is? Thank you! ๐Ÿ™๐Ÿป

    opened by akulbe 1
  • visibility of wiki pages

    visibility of wiki pages

    The visibility of wiki pages to search engines is generally limited; for example, the search result for "jq Cookbook" looks like this:

    jq Cookbook - GitHub
    https://github.com โ€บ stedolan โ€บ jq โ€บ wiki โ€บ Cookbook
    No information is available for this page.
    

    Even worse, Google sometimes moves references such as this to somewhere near infinity in the search results.

    To help alleviate the near invisibility of the Cookbook and FAQ to people using search engines, it would greatly help if hyperlinks to these pages could be added at the top of https://stedolan.github.io/jq/ and/or the jq Manual.

    Thanks.

    opened by pkoppstein 0
  • Release 1.6 does not have pre-autoreconf'ed configure script

    Release 1.6 does not have pre-autoreconf'ed configure script

    Describe the bug Release tarball jq-1.6.zip on releases page does not seem to have a configure script pre-autoreconf'ed.

    To Reproduce

    $ wget https://github.com/stedolan/jq/releases/download/jq-1.6/jq-1.6.zip
    $ unzip ./jq-1.6.zip
    $ file jq-jq-1.6/configure
    jq-jq-1.6/configure: cannot open `jq-jq-1.6/configure' (No such file or directory)
    

    Expected behavior A configure script already in the tarball as with previous releases and is mentioned in the README.

    Environment (please complete the following information):

    • OpenSUSE TW
    • jq-1.6
    opened by ashermancinelli 1
  • Fix typo

    Fix typo

    opened by henrebotha 0
Releases(jq-1.6)
  • jq-1.6(Nov 1, 2018)

    New in this release since 1.5:

    • Destructuring Alternation
    • New Builtins:
      • builtins/0
      • stderr/0
      • halt/0, halt_error/1
      • isempty/1
      • walk/1
      • utf8bytelength/1
      • localtime/0, strflocaltime/1
      • SQL-style builtins
      • and more!
    • Add support for ASAN and UBSAN
    • Make it easier to use jq with shebangs (8f6f28c)
    • Add $ENV builtin variable to access environment
    • Add JQ_COLORS env var for configuring the output colors

    Bug fixes:

    • Calling jq without a program argument now always assumes . for the program, regardless of stdin/stdout. (5fe0536)
    • Make sorting stable regardless of qsort. (7835a72)
    • Adds a local oniguruma submodule and the ./configure --with-oniguruma=builtin option to make it easier to build with oniguruma support on systems where you can't install system-level libraries. (c6374b6 and 02bad4b)
    • And much more!
    Source code(tar.gz)
    Source code(zip)
    jq-1.6.tar.gz(1.66 MB)
    jq-1.6.zip(1.84 MB)
    jq-linux32(2.65 MB)
    jq-linux64(3.77 MB)
    jq-osx-amd64(843.78 KB)
    jq-win32.exe(2.58 MB)
    jq-win64.exe(3.36 MB)
  • jq-1.5(Aug 16, 2015)

    Thanks to the 20+ developers who have sent us PRs since 1.4, and the many contributors to issues and the wiki.

    The manual for jq 1.5 can be found at https://stedolan.github.io/jq/manual/v1.5/

    Salient new features since 1.4:

    • regexp support (using Oniguruma)!

    • a proper module system

      import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::* namespace

      and

      include "foo/bar"; # import foo/bar.jq's defs into the top-level

    • destructuring syntax (. as [$first, $second, {$foo, $bar}] | ...)

    • math functions

    • an online streaming parser

    • minimal I/O builtions (inputs, debug)

      One can now write:

      jq -n 'reduce inputs as $i ( ... )'

      to reduce inputs in an online way without having to slurp them first! This works with streaming too.

    • try/catch, for catching and handling errors (this makes for a dynamic non-local exit system)

    • a lexical non-local exit system

      One can now say

      label $foo | ..... | break $foo

      where the break causes control to return to the label $foo, which then produces empty (backtracks). There's named and anonymous labels.

    • tail call optimization (TCO), which allows efficient recursion in jq

    • a variety of new control structure builtins (e.g., while(cond; exp), repeat(exp), until(cond; next)), many of which internally use TCO

    • an enhanced form of reduce: foreach exp as $name (init_exp; update_exp; extract_exp)

    • the ability to read module data files

      import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar

    • --argjson var '<JSON text>'

      Using --arg var bit me too many times :)

    • --slurpfile var "filename"

      Replaces the --argfile form (which is now deprecated but remains for backward compatibility).

    • support for application/json-seq (RFC7464)

    • a large variety of new utility functions, many being community contributions (e.g., bsearch, for binary searching arrays)

    • datetime functions

    • a variety of performance enhancements

    • def($a): ...; is now allowed as an equivalent of def(a): a as $a | ...;

    • test and build improvements, including gcov support

    Lastly, don't forget the wiki! The wiki has a lot of new content since 1.4, much of it contributed by the community.

    Source code(tar.gz)
    Source code(zip)
    jq-1.5.tar.gz(721.98 KB)
    jq-1.5.zip(729.88 KB)
    jq-linux32(1.52 MB)
    jq-linux32-no-oniguruma(1.24 MB)
    jq-linux64(2.88 MB)
    jq-osx-amd64(634.76 KB)
    jq-win32.exe(1.16 MB)
    jq-win64.exe(2.21 MB)
  • jq-1.1(Aug 8, 2015)

  • jq-1.0(Aug 8, 2015)

  • jq-1.2(Aug 8, 2015)

  • jq-1.5rc2(Jul 27, 2015)

    Thanks to the 20+ developers who have sent us PRs since 1.4, and the many contributors to issues and the wiki. We're nearing a 1.5 release, finally.

    Salient new features since 1.4:

    • regexp support (using Oniguruma)!

    • a proper module system

      import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::* namespace

      and

      include "foo/bar"; # import foo/bar.jq's defs into the top-level

    • destructuring syntax (. as [$first, $second, {$foo, $bar}] | ...)

    • math functions

    • an online streaming parser

    • minimal I/O builtions (inputs, debug)

      One can now write:

      jq -n 'reduce inputs as $i ( ... )'

      to reduce inputs in an online way without having to slurp them first! This works with streaming too.

    • try/catch, for catching and handling errors (this makes for a dynamic non-local exit system)

    • a lexical non-local exit system

      One can now say

      label $foo | ..... | break $foo

      where the break causes control to return to the label $foo, which then produces empty (backtracks). There's named and anonymous labels.

    • tail call optimization (TCO), which allows efficient recursion in jq

    • a variety of new control structure builtins (e.g., while(cond; exp), repeat(exp), until(cond; next)), many of which internally use TCO

    • an enhanced form of reduce: foreach exp as $name (init_exp; update_exp; extract_exp)

    • the ability to read module data files

      import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar

    • --argjson var '<JSON text>'

      Using --arg var bit me too many times :)

    • --slurpfile var "filename"

      Replaces the --argfile form (which is now deprecated but remains for backward compatibility).

    • support for application/json-seq (RFC7464)

    • a large variety of new utility functions, many being community contributions (e.g., bsearch, for binary searching arrays)

    • datetime functions

    • a variety of performance enhancements

    • def($a): ...; is now allowed as an equivalent of def(a): a as $a | ...;

    • test and build improvements, including gcov support

    Lastly, don't forget the wiki! The wiki has a lot of new content since 1.4, much of it contributed by the community.

    Source code(tar.gz)
    Source code(zip)
    jq-1.5rc2.tar.gz(697.31 KB)
    jq-1.5rc2.zip(739.09 KB)
    jq-linux-x86(1.39 MB)
    jq-linux-x86_64(3.04 MB)
    jq-osx-x86_64(634.89 KB)
    jq-win32.exe(1.16 MB)
    jq-win64.exe(2.21 MB)
  • jq-1.5rc1(Dec 31, 2014)

    Salient new features since 1.4:

    • regexp support (using Oniguruma)

    • an online streaming parser

      Included is the ability to control reading of inputs from the jq program, using the new input and inputs builtins.

      Finally we can write:

      jq -n 'reduce inputs as $i ( ... )' # reduce online!

      to reduce inputs without slurping them first. This works with streaming too.

    • try/catch, for catching and handling errors (this makes for a dynamic non-local exit system)

    • a lexical non-local exit system

      Using try/catch to break out of control structures was not a good thing. A lexical mechanism is.

      You can now say

      label $foo | ..... | break $foo

      where the break causes control to return to the label $foo, which then produces empty (backtracks). There's named and anonymous labels.

    • tail call optimization (TCO), which allows efficient recursion in jq

    • a variety of new control structure builtins (e.g., while(cond; exp), repeat(exp), until(cond; next))

    • an enhanced form of reduce: foreach exp as $name (init_exp; update_exp; extract_exp)

    • a proper module system

      import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::* namespace

    • the ability to read module data files

      import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar

    • --argjson var '<JSON text>'

      Using --arg var bit me too many times :)

    • --in-place / -i for in-place editing of files

    • support for application/json-seq.

    • a variety of new utility functions, many being community contributions

    • a variety of performance enhancements (e.g., constant folding)

    • def($a): ...; is now allowed as an equivalent of def(a): a as $a | ...;

    Lastly, don't forget the wiki! It has a lot of new content since 1.4, much of it contributed by the community.

    Source code(tar.gz)
    Source code(zip)
    jq-1.5rc1.tar.gz(630.66 KB)
    jq-linux-x86_64-static(3.26 MB)
    jq-win32.exe(1.46 MB)
    jq-win64.exe(1.29 MB)
    jq.1(77.38 KB)
Automatically generate Go (golang) struct definitions from example JSON

gojson gojson generates go struct definitions from json or yaml documents. Example $ curl -s https://api.github.com/repos/chimeracoder/gojson | gojson

Aditya Mukerjee 2.4k Sep 24, 2021
A modern UNIX ed (line editor) clone written in Go

ed (the awesome UNIX line editor) ed is a clone of the UNIX command-line tool by the same name ed a line editor that was nortorious for being and most

James Mills 45 May 29, 2021
Kong is a command-line parser for Go

Kong is a command-line parser for Go Introduction Help Help as a user of a Kong application Defining help in Kong Command handling Switch on the comma

Alec Thomas 663 Sep 25, 2021
A Commander for modern Go CLI interactions

Cobra is both a library for creating powerful modern CLI applications as well as a program to generate applications and command files. Cobra is used i

Steve Francia 23.2k Sep 17, 2021
Simplistic interactive filtering tool

peco Simplistic interactive filtering tool NOTE: If you are viewing this on GitHub, this document refers to the state of peco in whatever current bran

null 6.5k Sep 18, 2021
Query, update and convert data structures from the command line. Comparable to jq/yq but supports JSON, TOML, YAML, XML and CSV with zero runtime dependencies.

dasel Dasel (short for data-selector) allows you to query and modify data structures using selector strings. Comparable to jq / yq, but supports JSON,

Tom Wright 1.6k Sep 24, 2021
:cherry_blossom: A command-line fuzzy finder

fzf is a general-purpose command-line fuzzy finder. It's an interactive Unix filter for command-line that can be used with any list; files, command hi

Junegunn Choi 39.3k Sep 24, 2021
yq is a portable command-line YAML processor

yq a lightweight and portable command-line YAML processor. yq uses jq like syntax but works with yaml files as well as json. It doesn't yet support ev

Mike Farah 4.3k Sep 25, 2021
CLI - A package for building command line app with go

Command line interface Screenshot Key features Lightweight and easy to use. Defines flag by tag, e.g. flag name(short or/and long), description, defau

็Ž‹ไป•ๆ™‹ 607 Sep 20, 2021
Fully featured Go (golang) command line option parser with built-in auto-completion support.

go-getoptions Go option parser inspired on the flexibility of Perlโ€™s GetOpt::Long. Table of Contents Quick overview Examples Simple script Program wit

David Gamba 37 Sep 16, 2021
CONTRIBUTIONS ONLY: A Go (golang) command line and flag parser

CONTRIBUTIONS ONLY What does this mean? I do not have time to fix issues myself. The only way fixes or new features will be added is by people submitt

Alec Thomas 3.1k Sep 15, 2021
๐Ÿ”„ A command-line utility to export Protocol Buffers (proto) files to YAML, and JSON

proto2yaml ?? A command-line utility to export Protocol Buffers (proto) files to YAML, and JSON. Currently supported exports are for: Packages Service

Kristof Kowalski 3 Aug 23, 2021
A command line http test tool. Maintain the case via git and pure text

httptest A command line http test tool Maintain the api test cases via git and pure text We want to test the APIs via http requests and assert the res

wklken 11 Aug 24, 2021
A JSON diff utility

JayDiff A JSON diff utility. Install Downloading the compiled binary Download the latest version of the binary: releases extract the archive and place

Guillaume de Sagazan 84 Aug 20, 2021
Hosty is a command-line utility that allows for fast inspection and editing of /etc/hosts-like files

Hosty Description Hosty is a command-line utility that allows for fast inspection and editing of /etc/hosts-like files. It is written in golang and us

null 11 Sep 3, 2021
Another CLI framework for Go. It works on my machine.

Command line interface framework Go framework for rapid command line application development

Ulrich Kautz 108 May 24, 2021
Command line tool for adding Windows resources to executable files

go-winres A simple command line tool for embedding usual resources in Windows executables built with Go: A manifest An application icon Version inform

null 71 Sep 4, 2021
Pure Go line editor with history, inspired by linenoise

Liner Liner is a command line editor with history. It was inspired by linenoise; everything Unix-like is a VT100 (or is trying very hard to be). If yo

Peter Harris 814 Sep 13, 2021
A command-line tool and library for generating regular expressions from user-provided test cases

Table of Contents What does this tool do? Do I still need to learn to write regexes then? Current features How to install? 4.1 The command-line tool 4

Peter M. Stahl 4.5k Sep 25, 2021