Tool, language and decoders for inspecting binary data.



Tool, language and decoders for inspecting binary data.

fq demo

In most cases fq works the same way as jq but instead of reading JSON it reads binary data. The result is a JSON compatible structures where each value has a bit range, symbolic interpretations and know how to be presented in a useful way.

NOTE: fq is early in development and many things are missing, broken or do not make sense. That also means there is a great opportunity to help out!


  • Make binary formats accessible and queryable.
  • Nested formats and bit-oriented decoding.
  • Quick and comfortable CLI tool.
  • Bit and byte transformations and conversions.
  • Programmer's calculator.


Basic usage is fq . file.

For details see


Download release for your platform, unarchive it and move the executable to PATH etc.


# install latest release
brew install wader/tap/fq

Build from source

Make sure you have go 1.17 or later installed.

To install directly from git repository do:

# build and install latest release
go install[email protected]

# or build and install latest master
go install[email protected]

# copy binary to $PATH if needed
cp "$(go env GOPATH)/bin/fq" /usr/local/bin

To build and run tests from source directory:

make test fq
# copy binary to $PATH if needed
cp fq /usr/local/bin

Supported formats

aac_frame, adts, adts_frame, apev2, av1_ccr, av1_frame, av1_obu, avc_annexb, avc_au, avc_dcr, avc_nalu, avc_pps, avc_sei, avc_sps, bzip2, dns, dns_tcp, elf, ether8023_frame, exif, flac, flac_frame, flac_metadatablock, flac_metadatablocks, flac_picture, flac_streaminfo, gif, gzip, hevc_annexb, hevc_au, hevc_dcr, hevc_nalu, icc_profile, icmp, id3v1, id3v11, id3v2, ipv4_packet, jpeg, json, matroska, mp3, mp3_frame, mp4, mpeg_asc, mpeg_es, mpeg_pes, mpeg_pes_packet, mpeg_spu, mpeg_ts, ogg, ogg_page, opus_packet, pcap, pcapng, png, protobuf, protobuf_widevine, pssh_playready, raw, sll2_packet, sll_packet, tar, tcp_segment, tiff, udp_datagram, vorbis_comment, vorbis_packet, vp8_frame, vp9_cfm, vp9_frame, vpx_ccr, wav, webp, xing, zip

For details see

TODO and ideas




Thanks and related projects

This project would not have been possible without itchyny's jq implementation gojq. I also want to thank HexFiend for inspiration and ideas and stedolan for inventing the jq language.

Similar or related projects:

  • Add WASM format

    Add WASM format

    Hello, I have created this because I personally needed it. Tested against bunch of .wasm / .fqtest pairs in format/wasm/testdata/core. (The .wasm files are generated from the standard test suite i.e.

    I would be happy to receive any feedback.

    Thank you!

    opened by bitbears-dev 50
  • macho: Initial impl for macho support.

    macho: Initial impl for macho support.

    [WIP] Several issues remain for symbol mapping for flags and constant fields UNIX/THREAD LC .state field has to be visualized for PPC/64, ARM/64 X86-64

    I have created a working barebones implementation there are several improvements to be made here: We need to have a SymStr mapping for cpusubtype but it depends on cputype and an arbitrary number. Flag fields for individual messages are in uint32_t, I think it is user friendly to have those expanded to see which flags are set or not. LC_THREAD/LC_UNIXTHREAD commands have a map of Register Name to Register Value. The visualization for this must be implemented for major architectures as I have written in the commit message

    It correctly parses the fq binary on MacOS.

    Solves #43

    opened by Akaame 47
  • Avro: Add decoder

    Avro: Add decoder

    Full avro OCF support. Handles all primitive, complex, and logical types besides decimals.

    Able to handle deflate, snappy, and null codecs for blocks.

    Requirements for WIP removal:

    • ~Support common logical types (date, decimal, duration, time, timestamp)~
    • ~Add test case with all avro datatypes~
    • ~Evaluate viability of splitting avro datum into a subdecoder~
    • ~Cleanup OCF header decoding~
    • ~Cleanup schema decoding~
    • ~Finalize design around handling OCF codecs. (Currently only handles null codec, rest treat the datum as raw bytes and won't decode them)~
    opened by xentripetal 27
  • Make packages importable

    Make packages importable

    Currently it seems none of the packages are importable

    because all items are marked as private. Can you export some items so that the packages can be used?

    opened by 89z 22
  • doc: Add Nix instructions to readme

    doc: Add Nix instructions to readme

    fq is packaged in Nixpkgs as of, so these instructions will work within a few days (once the master channel advances.)

    opened by siraben 13
  • Incorrect version in some distribution packages

    Incorrect version in some distribution packages

    $ docker run --rm alpine:3.15 sh -c 'apk add -X fq && fq --version'
    (1/1) Installing fq (0.0.3-r0)
    Executing busybox-1.34.1-r3.trigger
    OK: 15 MiB in 15 packages

    Currently version is set by goreleaser/make when building using go build ldflag -X main.version=.... To make it work with distributions that builds manually using go build etc we probably have to include the version in the source somehow.

    • Create some version file using goreleaser when packaging source archives. Wont fix issue if distribution clones the repo.
    • Convince package maintainers to build using make and pass version as some argument? too much hassel i think
    • Manually change/commit some version file before release
    • Some other way?
    opened by wader 12
  • mp4: Properly use trun data offset

    mp4: Properly use trun data offset

    Each trun has it's own data offset, before the last offset was wrongly used for all truns. Could also cause sample ranges to be beyond EOF.

    tenc: Decode default constant iv

    Fixes #292

    opened by wader 11
  • Elf parsed as mp3

    Elf parsed as mp3

    Is that expected to see such result of parsing elf (/bin/ls) as mp3?

    $ file /bin/ls
    /bin/ls: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/, BuildID[sha1]=2f15ad836be3339dec0e2e6a3c637e08e48aacbd, for GNU/Linux 3.2.0, stripped
    $ fq d /bin/ls


    opened by alex-bender 10
  • doc: use nix-env instead of nix-shell to install

    doc: use nix-env instead of nix-shell to install

    As mentioned on, nix-shell -p is not actually installing the package, this PR updates it to use nix-env -iA instead, this way it's behavior is the same as brew install and yay -S, making fq available system wide.

    opened by portothree 10
  • Differentiate color/format for item count and index

    Differentiate color/format for item count and index

    In the lovely color coded output, the same color/format ("key: [value]") is used to present both item count and item index. This is no big deal and I guess it's because all integers get this color, but slightly confusing.

    opened by Doctor-love 10
  • Adds Windows Scoop instructions for fq in README.

    Adds Windows Scoop instructions for fq in README.

    Adds instructions to installing fq from scoop which I added recently.

    image (ignore the warnings, this is a fresh install of Windows)

    We had some difficulty with licenses as fq isn't officially marked as MIT (even though the LICENSE file text is MIT).

    opened by thushan 8
  • Adds support for Apple Binary Plist, version 00

    Adds support for Apple Binary Plist, version 00

    This adds support for decoding Apple Binary Plists. The only well documented version is 00, and is therefore the only one supported here. I have tested this on both large and small binary plists, including ones with nested dictionaries.

    opened by dgmcdona 10
  • OOM when working with 1 GB PostgreSQL heap (table) file

    OOM when working with 1 GB PostgreSQL heap (table) file

    What version are you using (fq -v)?

    $ fq -v
    0.0.9 (linux amd64)

    How was fq installed?

    fq is build from source.

    My branch:

    Can you reproduce the problem using the latest release or master branch?

    I can reproduce problem on my branch, where PostgreSQL parsers are implemented.

    1 GB file:

    What did you do?

    I am trying to parse a heap (relation, table) file of maximum size (1 GB for PostgeSQL). fq consumes 90-100 times memory than file size. For 80 mb file fq requires 7,5 GB RAM.

    time fq -d pgheap -o flavour=postgres14 ".Pages[0].PageHeaderData.pd_linp[0, 1, 2, -1] | tovalue" 16397
    real    0m50.794s
    user    1m11.962s
    sys     0m8.994s

    Kenel messages:

    sudo dmesg | tail -2
    [193541.830725] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=user.slice,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/session-1.scope,task=fq,pid=454783,uid=1000
    [193541.830748] Out of memory: Killed process 454783 (fq) total-vm:31508780kB, anon-rss:26629332kB, file-rss:272kB, shmem-rss:0kB, UID:1000 pgtables:58860kB oom_score_adj:0

    1 GB File:

    ls -alh 16397
    -rw-r----- 1 pavel pavel 1.0G Aug 31 08:38 16397

    Memory profiler results:

    go tool pprof
    File: postgres.test
    Type: alloc_space
    Time: Aug 30, 2022 at 6:16pm (MSK)
    Entering interactive mode (type "help" for commands, "o" for options)
    (pprof) top 40
    Showing nodes accounting for 31041.34MB, 97.35% of 31886.71MB total
    Dropped 270 nodes (cum <= 159.43MB)
    Showing top 40 nodes out of 87
          flat  flat%   sum%        cum   cum%
    16563.23MB 51.94% 51.94% 17983.80MB 56.40%*D).TryFieldScalarFn.func1
     5007.15MB 15.70% 67.65%  5007.15MB 15.70%*D).fieldDecoder
     2168.92MB  6.80% 74.45%  3948.79MB 12.38%*D).FillGaps
     2109.38MB  6.62% 81.06%  2109.38MB  6.62%*D).AddChild (inline)
     1327.83MB  4.16% 85.23%  1327.83MB  4.16%
      872.54MB  2.74% 87.96% 26209.93MB 82.20%*D).FieldStruct
      587.02MB  1.84% 89.81%   587.02MB  1.84%  internal/reflectlite.Swapper
      513.59MB  1.61% 91.42%   513.59MB  1.61%  fmt.Sprintf
      425.01MB  1.33% 92.75%  1012.03MB  3.17%*Value).postProcess.func1
      324.51MB  1.02% 93.77%   324.51MB  1.02% (inline)
         288MB   0.9% 94.67%   372.51MB  1.17%*D).TryFieldScalarU16.func1
      236.02MB  0.74% 95.41%   696.61MB  2.18%
      174.50MB  0.55% 95.96%  9928.56MB 31.14%
      104.50MB  0.33% 96.29%  2701.76MB  8.47%
          61MB  0.19% 96.48%  1440.14MB  4.52%
       56.53MB  0.18% 96.65%  1510.17MB  4.74%
          48MB  0.15% 96.80%   311.03MB  0.98%*compiler).compileCallInternal
          48MB  0.15% 96.95%   460.59MB  1.44%
       46.51MB  0.15% 97.10%   410.05MB  1.29%*compiler).compileFuncDef
          24MB 0.075% 97.18%   369.99MB  1.16%*compiler).compileTerm
       15.05MB 0.047% 97.22% 31404.72MB 98.49%*env).Next
           7MB 0.022% 97.25%   169.67MB  0.53%*compiler).compileCall
           7MB 0.022% 97.27%   186.53MB  0.58%*compiler).compileIf
           5MB 0.016% 97.28%   358.54MB  1.12%*compiler).compile
        4.51MB 0.014% 97.30%   222.25MB   0.7%*Interp).Eval.func2
        4.50MB 0.014% 97.31%   375.04MB  1.18%*compiler).compileQuery
        3.50MB 0.011% 97.32%   205.43MB  0.64%*compiler).compileFunc
           3MB 0.0094% 97.33% 26212.93MB 82.21%
           3MB 0.0094% 97.34% 26212.93MB 82.21%*D).FieldArray
        1.50MB 0.0047% 97.35% 26198.29MB 82.16%
        0.50MB 0.0016% 97.35%   545.91MB  1.71%
        0.50MB 0.0016% 97.35%   164.87MB  0.52%*compiler).compileCallPc
             0     0% 97.35% 26212.93MB 82.21%
             0     0% 97.35% 26080.73MB 81.79% (inline)
             0     0% 97.35%  2322.01MB  7.28%
             0     0% 97.35% 26212.93MB 82.21%
             0     0% 97.35%  2235.26MB  7.01%
             0     0% 97.35%  5454.55MB 17.11%
             0     0% 97.35%   887.09MB  2.78%
             0     0% 97.35%  1419.14MB  4.45%

    I check with disassembler. TryFieldScalarFn.func1 is unnamed lambda in TryFieldScalarFn.

    opened by pnsafonov 1
  • Rebrand as

    Rebrand as "jq for formats"?

    One of reasons fq is not called bq is because i was aiming for decoding formats not only binary formats. Is "jq for formats" good or should be clearer? Suggestions welcome.

    fq can now do things like this:

    $ fq -n -d html '[inputs | {(input_filename): .html.head.title}?] | add' *.html
      "Media Internals.html": "Media Internals",
      "make_2.html": "GNU make - An Introduction to Makefiles",
      "search.html": "Search · decrypt"
    $ cat file.csv | fq -d csv '.[1:] | map(.[1] | tonumber) | add'
    $ fq -rd bencode 'torepr | del(.info.pieces) | {torrent: .} | toxml({indent: 2})' debian-live-10.7.0-amd64-gnome.iso.torrent
      <comment>&#34;Debian CD from;</comment>
      <creation date>1607188931</creation date>
        <piece length>1048576</piece length>
    $ echo 'https://host?a=b' | fq -Rr 'fromurl | .query.b = ("hello" | tobase64) | tourl'
    $ fq '.svg | [."-height", ."-width"]' file.svg

    At the moment some formats are not real format (can't be used with -d) ex url and base64, they are just jq functions. Maybe they should be formats in addition to being functions? so you could do echo ... | fq -d url ....

    opened by wader 4
  • Add MBR format

    Add MBR format

    Related to issue #23

    This is a first pass, it breaks down the MBR into the code_area, the partition_table, and the signature (magic number denoting the end of the boot record)

    Future work

    • Disassemble the 16-bit opcodes in the code area for example
    objdump -D -Mintel,i8086 -b binary -m i386 format/mbr/testdata/mbr.bin | head -15
    format/mbr/testdata/mbr.bin:     file format binary
    Disassembly of section .data:
    00000000 <.data>:
       0:	fa                   	cli
       1:	bc 00 7c             	mov    sp,0x7c00
       4:	31 c0                	xor    ax,ax
       6:	8e d0                	mov    ss,ax
       8:	8e c0                	mov    es,ax
       a:	8e d8                	mov    ds,ax
       c:	52                   	push   dx
       d:	be 00 7c             	mov    si,0x7c00


    MBR is short for Master Boot Record. It is the legacy method of booting, the code area is limited to 446 bytes, and it's all 16-bit x86 opcodes. Old BIOS code knows how to call into it and execute the instructions there. MBR also has a 64-byte partition table that can store up to 4 partitions. The GPT partitioning scheme obsoletes this, and allows for more partitions, but that requires UEFI to work.

    For more information on MBR, see:

    opened by tlehman 7
  • v0.0.9(Aug 25, 2022)


    • New wasm WebAssembly Binary Format decoder by Takashi Oguma @bitbears-dev
      # show part of code_section
      $ fq '.sections[4].content.code.x[0].code.e | d' add.wasm
          │00 01 02 03 04 05 06 07 08 09│0123456789│.sections[4].content.code.x[0].code.e[0:4]:
          │                             │          │  [0]{}: in
      0x3c│                           20│          │    opcode: "local.get" (0x20)
      0x46│01                           │.         │    x: 1 (valid)
          │                             │          │  [1]{}: in
      0x46│   20                        │          │    opcode: "local.get" (0x20)
      0x46│      00                     │  .       │    x: 0 (valid)
          │                             │          │  [2]{}: in
      0x46│         6a                  │   j      │    opcode: "i32.add" (0x6a)
          │                             │          │  [3]{}: in
      0x46│            0b               │    .     │    opcode: "end" (0xb)
      # count opcode usage
      $ fq '.sections[] | select(.id == "code_section") | [.. | .opcode? // empty] | count | map({key: .[0], value: .[1]}) | from_entries' add.wasm
        "end": 1,
        "i32.add": 1,
        "local.get": 2
      # list exports and imports
      $ fq '.sections | {import: map(select(.id == "import_section")[].nm.b), export: map(select(.id == "export_section").content.ex.x[].nm.b)}' add.wasm
        "export": [
        "import": []
    • Decode value display now shows address bar on new format or buffer. Should make it easier to spot changes and read hex and ASCII view. #365
      Examples of PCAP with different formats and TCP stream buffers:
             │00 01 02 03 04 05 06 07 08 09│0123456789│      packet{}: (ether8023_frame)
      0x00668│   00 0a 95 67 49 3c         │ ...gI<   │        destination: "00:0a:95:67:49:3c" (0xa9567493c)
      0x00668│                     00 c0 f0│       ...│        source: "00:c0:f0:2d:4a:a3" (0xc0f02d4aa3)
      0x00672│2d 4a a3                     │-J.       │
      0x00672│         08 00               │   ..     │        ether_type: "ipv4" (0x800) (Internet Prot...
             │00 01 02 03 04 05 06 07 08 09│0123456789│        payload{}: (ipv4_packet)
      0x00672│               45            │     E    │          version: 4
             │00 01 02 03 04 05 06 07 08 09│0123456789│          payload{}: (tcp_segment)
      0x00686│               00 50         │     .P   │            source_port: "http" (80) (World Wide ...
             │                             │          │  ipv4_reassembled[0:0]:
             │                             │          │  tcp_connections[0:1]:
             │                             │          │    [0]{}: tcp_connection
             │                             │          │      client{}:
             │                             │          │        ip: ""
             │                             │          │        port: 34059
             │                             │          │        has_start: true
             │                             │          │        has_end: true
             │                             │          │        skipped_bytes: 0
             │00 01 02 03 04 05 06 07 08 090123456789│
        0x000│47 45 54 20 2f 74 65 73 74 2f│GET /test/│        stream: raw bits
        0x000│65 74 68 65 72 65 61 6c 2e 68│ethereal.h│
        *    │until 0x1bc.7 (end) (445)    │          │
    • Add --unicode-output/-U argument to force use of Unicode characters to improve output readability. #377
      • For example useful when piping to less and you want fancy unicode and colors:
        fq -cU d file | less -r
    • to_entries now preserves struct field order. #340
    • Experimental `raw string` literal support. Work the same as golang raw string literals. Useful in REPL when pasting things etc but should probably be avoided in jq scripts. #371
    • Properly fail lexing of invalid binary, octal and hex number literals. #371
    • REPL completion now include all functions. Before some functions with multiple argument counts were skipped. #375
    • Switch to new gopacket fork with speedup and bug fixes. Remove SLL2 workarounds in fq. #386

    Decoder changes

    • csv Correctly handle decode values when tocsv normalize to strings. Before array and object ended up being JSON serialized to strings. #341
      • Normalize to strings is done so that non-string scalars can be used:
        $ fq -n '[[1,true,null,"a"]] | tocsv'
    • dns DNS over UDP format was accidentally used to probe TCP streams #366
    • elf Remove redundant program_header struct
    • flac
      • Add 32 bit samples support. #378 Thanks @ktmf01
      • Properly decode/checksum samples in partitions with raw samples. #379 Thanks @ktmf01
        Now successfully decodes all test cases from
    • jsonl Add decoder. Decodes JSON lines. There is also fromjsonl and tojsonl. #374
    • macho
      • Split FAT Macho decoding into macho_fat format which also fixed handling of file offsets in sections. #362
      • Decode symbol and string sections. #352
    • matroska Remove new lines in descriptions. Messes up tree. #369
    • mp3_frame
      • Support LSF (low sampling frequency) frames. #376
      • Skip trying to figure out what main data is for current frame and not. Was incorrect and doing it properly probably requires hoffman decoding. #376
    • pcap Support files with nanosecond precision. Has a different magic. #382
    • prores_frame Add basic decoder. Decodes container and fram header. #396 Thanks @Doctor-love for test files
    • tar Fix regression when decoding number fields. Now ok again to be empty string. #372
    • wasm Add WebAssembly Binary Format decoder. #383 Thanks to Takashi Oguma @bitbears-dev
    • yaml Fail on trailing data. Before it succeeded with the last value. #373
    • zip
      • Don't require PK header as there seems to be zip files with prepended data. #359
      • Correctly limit amount of backwards search for EOCD (end of content directory). #363
    • xml Correctly handle decode values when toxml normalize to strings. Before array and object ended up being JSON serialized to strings. #341
    • xml
      • Change attribute prefix to @ instead of - and make it an option attribute_prefix. #401
      • Skip default namespace in element names. #389
      • Always include attributes and children even when empty in array mode. Makes it a lot easer to work with as you can assume .[1] will be attributes and so on. #357
      • Normalize to strings is done so that non-string scalars can be used:
        $ fq -nr '{a: {"@boolean": true, "@number": 123, "@null": null}} | toxml'
        <a boolean="true" null="" number="123"></a>
      • Allow and ignore trailing <?procinstr?> and improve trailing data error message. #368
      • Correctly sort if any #seq is found and also properly sort negative #seq. #384


    • 0cd846a1 *extra: Rename extra to just ex and refactor to use generics
    • fb583e2c Add 32 bps FLAC to test
    • c1d5b2b1 Add sample size entry to list for 32bps flac streams
    • 3f209c46 Fix decoding of FLAC raw entropy partition
    • 25061aca Update docker-golang from 1.18.4 to 1.18.5
    • 0de2c906 Update docker-golang from 1.18.5 to 1.19.0
    • 7b8d95bf Update github-go-version from 1.18.4, 1.18.4, 1.18.4 to 1.18.5
    • 103991f7 Update github-go-version from 1.18.5, 1.18.5, 1.18.5 to 1.19.0
    • 4255b87a Update github-golangci-lint from 1.47.2 to 1.47.3
    • 198305ec Update github-golangci-lint from 1.47.3 to 1.48.0
    • fa9fec30 Update github-golangci-lint from 1.48.0 to 1.49.0
    • f579e9c3 Update make-golangci-lint from 1.47.2 to 1.47.3
    • c8069d22 Update make-golangci-lint from 1.47.3 to 1.48.0
    • 004eb564 Update make-golangci-lint from 1.48.0 to 1.49.0
    • abcc7366 add ULEB and SLEB to known words for spell check
    • 9238251b ci: Skip -race for windows and macos
    • 913f5780 columnwriter,dump: Add Column interface and refactor into BarColumn and MultiLineColumn
    • 5d9ffead decode,scalar: Map empty string also else sym might ends up nil
    • 326dada7 decode: Add LEB128 readers
    • 502f451c decode: Refactor to use scalar type assert helper
    • 840292ba decode: Simplify compound range sort behaviour
    • 15f7c67a dev,fuzz: Add some useful retrigger snippets
    • 46dca8cd dns: Don't use dns (udp) format for tcp also
    • c233215a dns: Rename isTCP to hasLengthHeader
    • ed424783 doc,interp: Update and add more examples
    • f247edb5 doc: Update README demo a bit with new features
    • 3613b6d4 elf: Remove redundant program_header struct
    • 8a19978b flac: Make gen script generate correct fqtest files
    • 2bfbe9a9 flac_frame: Cleanup some dev lefterovers and todos
    • 64b23659 fqtest: Run tests in parallell
    • af35b284 gojq: Preserve keys order for to_entries when used with JQValue
    • 804ad1e2 gojq: Update fq fork
    • add3dcfd gojq: Update fq fork, fix scope argcount issue
    • d898732c gojq: Update fq fork, new scope function, rawstring, stricter integers
    • 394717ca gopacket: Switch/update to new fork, remove SLL2 hack
    • 4eae7ffd interp,doc: Add -R raw string slurp hint to -s help
    • d8792fd1 interp,dump: Correctly flush columns if data will be shown
    • 29005c70 interp,dump: Show address bar for root, nested roots and on format change
    • c7559b59 interp: Add --unicode-output/-U to force use of unicode
    • 9e447c9a interp: Use RegisterFS instead of format files
    • 701c67c1 jsonl: Add decoder, also tojsonl encoder
    • bc6cffde lint,decode,fuzz:: Fix nilerr warnings, one real one should be ignored for now
    • 3c21b058 lint: Fix ioutil deprecation, reformat for new doc standard
    • b2d4e6d9 macho: Decode cmd symtab symbols
    • 725c8e83 macho: Split into macho/macho_fat, fix offset issue and add string decoding
    • 2e407386 matroska: Strip newlines in description
    • cf15661e mp3_frame: Add LSF support and fix incorrect main data handling
    • 74c7dc4e pcap: Add ns support and add header field
    • 8fc43533 prores_frame: Add basic container and frame header decoder
    • dc32ac08 script: Use strings.Builder to collect output
    • 0d44b937 tar: Some number fields can be empty
    • 545dac8c test: Update tests, go 1.19 uses \xff instead of \u00ff
    • ce438872 wasm: make doc
    • 074c22c9 wasm: add -timeout 20m for go test to workaround ci test fail
    • cd037c51 wasm: add comment to clarify lazy initialization
    • f73965d2 wasm: add wasm to probe list
    • 00869b37 wasm: avoid race condition
    • db8021c9 wasm: define and use constants for some insturctions
    • bcc0dfd9 wasm: fix comment format
    • 289ddf59 wasm: fix lint issues
    • 3fca7cc0 wasm: fix lint issues
    • cbb5a8ed wasm: further simplification
    • 934ed9a8 wasm: initial version
    • e5cf1731 wasm: make the godoc formatter happy
    • b0f3fec8 wasm: remove nolint:unparam which is no longer needed
    • e1691dec wasm: remove unused function
    • ae4529c4 wasm: run golangci-lint run --fix
    • fead68de wasm: tidy up
    • 3298d181 wasm: to be able to probe
    • 2eb17505 wasm: update tests
    • d5d9e738 wasm: use FieldULEB128() / FieldSLEB128() defined in the upstream
    • 7401d141 wasm: use WRITE_ACTUAL=1 to generate .fqtest files
    • 2037b86a wasm: use map, not switch
    • ae08bf70 wasm: use s.ActualU() instead of s.Actual.(uint64)
    • 63f4a726 wasm: use scalar.UToSymStr for simplicity
    • 0ad5a8ec wasm: use underscores for symbol values
    • fa20c74c xml,csv,interp: Handle JQValue when string normalizing
    • f4e01372 xml,html: Always include attrs and children in array mode
    • 9a5fcc89 xml: Allow trailing
    • 71900c2a xml: Correctly sort if one #seq is found and allow negative seq numbers
    • 716323ce xml: Even more namespace fixes
    • f24d685a xml: Keep track of default namespace and skip it element names
    • 095e1161 xml: Switch from "-" to "@" as attribute prefix and make it an option
    • 3623eac3 yaml: Error on trailing yaml/json
    • d607bee1 zip: Correctly limit max EOCD find
    • 19b70899 zip: Skip header assert as there are zip files with other things appended
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(556 bytes)
    fq_0.0.9_linux_amd64.tar.gz(3.30 MB)
    fq_0.0.9_linux_arm64.tar.gz(3.00 MB) MB) MB) MB) MB)
  • v0.0.8(Jul 24, 2022)


    • Add support for some common structured serialization formats: #284 #335
      • XML, toxml, fromxml options for indent, jq mapping variants (object or array) and order preservation
      • HTML, fromhtml options for indent, jq mapping variants (object or array) and order preservation
      • TOML, totoml, fromtoml
      • YAML, toyaml, fromyaml
      • jq-flavored JSON (optional key quotes and trailing comma) tojq, fromjq options for indent #284
        # query a YAML file
        $ fq '...' file.yml
        # convert YAML to JSON
        # note -r for raw string output, without a JSON string with JSON would outputted
        $ fq -r 'tojson({indent:2})' file.yml
        $ fq -nr '{hello: {world: "test"}} | toyaml, totoml, toxml, tojq({indent: 2})'
            world: test
          world = "test"
          hello: {
            world: "test"
        $ echo '<doc><element a="b"></doc>' | fq -r '.doc.element."-a"'
        $ echo '<doc><element a="b"></doc>' | fq -r '.doc.element."-a" = "<test>" | toxml({indent: 2})'
          <element a="&lt;test&gt;"></element>
      • CSV, tocsv, fromcsv options for separator and comment character
        $ echo -e  '1,2\n3,4' | fq -rRs 'fromcsv | . + [["a","b"]] | tocsv'
    • Add support for binary encodings
      • Base64. tobase64, frombase64 options for encoding variants.
        $ echo -n hello | base64 | fq -rRs 'frombase64 | tostring'
      • Hex string. tohex, fromhex
    • Add support for text formats
      • XML entities toxmlentities, fromxmlentities
      • URL tourl, fromurl
        $ echo -n 'https://host/path/?key=value#fragment' | fq -Rs 'fromurl | ., (.host = "changed" | tourl)'
          "fragment": "fragment",
          "host": "host",
          "path": "/path/",
          "query": {
            "key": "value"
          "rawquery": "key=value",
          "scheme": "https"
      • URL path encoding tourlpath, fromurlpath
      • URL encoding tourlencode, fromurlencode
      • URL query tourlquery, fromurlquery
    • Add support for common hash functions:
      • MD4 tomd4
      • MD5 tomd5
        $ echo -n hello | fq -rRs 'tomd5 | tohex'
      • SHA1 tosha1
      • SHA256 tosha256
      • SHA512 tosha512
      • SHA3 224 tosha3_224
      • SHA3 256 tosha3_256
      • SHA3 384 tosha3_384
      • SHA3 512 tosha3_512
    • Add support for common text encodings:
      • ISO8859-1 toiso8859_1, fromiso8859_1
      • UTF8 tutf8, fromutf8
      • UTF16 toutf16, fromutf16
      • UTF16LE toutf16le, fromutf16le
      • UTF16BE toutf16be, fromutf16be
        $ echo -n 00680065006c006c006f | fq -rRs 'fromhex | fromutf16be'
    • Add group function, same as group_by(.) #299
    • Update/rebase readline dependency (based on @tpodowd #305 #308
      • Less blinking/redraw in REPL
      • Lots of small bug fixes
    • Update/rebase gojq dependency #247
      • Fixes JQValue destructing issue (ex: <some object JQValue> as {$key})
    • Major rewrite/refactor how native function are implemented. Less verbose and less error-prone as now shared code takes care of type casting and some argument errors. #316
    • Add tojson($opts) that support indent option. tojson still works as before (no indent).
      $ echo '{a: 1}' | fq -r 'tojson({indent: 2})'
        "a": 1
    • Rename --decode-file (will still work) to --argdecode be be more consistent with existing --arg* arguments. #309
    • On some decode error cases fq can now keep more of partial tree making it easier to know where it stopped #245
    • Build with go 1.18 #272

    Decoder changes

    • bitcoin Add Bitcoin blkdat, block, transcation and script decoders #239
    • elf Use correct offset to dynamic linking string table #304
    • tcp Restructure into separate client/server objects and add skipped_bytes (number of bytes with known missing ACK), has_start (has first byte in stream) and has_end (has last byte in stream) per direction #251
      • Old:
            │00 01 02 03 04 05 06 07│01234567│.tcp_connections[0]{}: tcp_connection
            │                       │        │  source_ip: ""
            │                       │        │  source_port: 34059
            │                       │        │  destination_ip: ""
            │                       │        │  destination_port: "http" (80) (World Wide Web HTTP)
            │                       │        │  has_start: true
            │                       │        │  has_end: true
       0x000│47 45 54 20 2f 74 65 73│GET /tes│  client_stream: raw bits
       0x008│74 2f 65 74 68 65 72 65│t/ethere│
       *    │until 0x1bc.7 (end) (44│        │
       0x000│48 54 54 50 2f 31 2e 31│HTTP/1.1│  server_stream: raw bits
       0x008│20 32 30 30 20 4f 4b 0d│ 200 OK.│
       *    │until 0x191.7 (end) (40│        │
      • New:
            │00 01 02 03 04 05 06 07│01234567│.tcp_connections[0]{}: tcp_connection
            │                       │        │  client{}:
            │                       │        │    ip: ""
            │                       │        │    port: 34059
            │                       │        │    has_start: true
            │                       │        │    has_end: true
            │                       │        │    skipped_bytes: 0
       0x000│47 45 54 20 2f 74 65 73│GET /tes│    stream: raw bits
       0x008│74 2f 65 74 68 65 72 65│t/ethere│
       *    │until 0x1bc.7 (end) (44│        │
            │                       │        │  server{}:
            │                       │        │    ip: ""
            │                       │        │    port: "http" (80) (World Wide Web HTTP)
            │                       │        │    has_start: true
            │                       │        │    has_end: true
            │                       │        │    skipped_bytes: 0
       0x000│48 54 54 50 2f 31 2e 31│HTTP/1.1│    stream: raw bits
       0x008│20 32 30 30 20 4f 4b 0d│ 200 OK.│
       *    │until 0x191.7 (end) (40│        │
    • zip Add 64-bit support and add uncompress option #278
    • matroska Update and regenerate based on latest spec and also handle unknown ids better #291
    • mp4 Changes:
      • Fix PSSH decode issue #283
      • Add track for track_id references without tfhd box
      • Makes it possible to see samples in fragments without having an init segment. Note it is possible to decode samples in a fragment file by concatenating the init and fragment file ex: cat init frag | fq ....
      • Add senc box support #290
      • Don't decode encrypted samples #311
      • Add track_id to tracks #254
      • Add fairplay PSSH system ID #310
      • Properly handle trun data offset #294
      • Skip decoding of individual PCM samples for now #268
      • Add mvhd, tkhd, mdhd and mehd version 1 support #258
      • Make sure to preserve sample table order #330
    • fairplay_spc Add basic FairPlay Server Playback Context decoder #310
    • avc_pps Correctly check for more rbsp data


    • 210940a4 Update docker-golang from 1.18.1 to 1.18.2
    • fbeabdc3 Update docker-golang from 1.18.2 to 1.18.3
    • 51a414db Update docker-golang from 1.18.3 to 1.18.4
    • 3017e8b4 Update github-go-version from 1.18.1, 1.18.1, 1.18.1 to 1.18.2
    • c597f7f7 Update github-go-version from 1.18.2, 1.18.2, 1.18.2 to 1.18.3
    • dd283923 Update github-go-version from 1.18.3, 1.18.3, 1.18.3 to 1.18.4
    • d10a3616 Update github-golangci-lint from 1.45.2 to 1.46.0
    • 75b5946c Update github-golangci-lint from 1.46.0 to 1.46.1
    • 3ffa9efb Update github-golangci-lint from 1.46.1 to 1.46.2
    • 4be8cb91 Update github-golangci-lint from 1.46.2 to 1.47.0
    • 1b8f4be8 Update github-golangci-lint from 1.47.0 to 1.47.1
    • fc596a7a Update github-golangci-lint from 1.47.1 to 1.47.2
    • 62be9223 Update gomod-BurntSushi/toml from 1.1.0 to 1.2.0
    • 5db7397a Update make-golangci-lint from 1.45.2 to 1.46.0
    • 456742ea Update make-golangci-lint from 1.46.0 to 1.46.1
    • 06757119 Update make-golangci-lint from 1.46.1 to 1.46.2
    • 3d69e9d0 Update make-golangci-lint from 1.46.2 to 1.47.0
    • 2170925d Update make-golangci-lint from 1.47.0 to 1.47.1
    • c4199c0f Update make-golangci-lint from 1.47.1 to 1.47.2
    • 02f00be9 Update
    • 75169a65 asn1: Add regression test for range decode fix ##330
    • b0096bc1 avc_pps: Correct check if there is more rbsp data
    • 5d67df47 avro_ocf: Fix panic on missing meta schema
    • 417255b7 bitcoin: Add blkdat, block, transcation and script decoder
    • a6a97136 decode: Cleanup Try/ pairs
    • 3ce660a2 decode: Keep decode tree on RangeFn error
    • c4dd518e decode: Make compound range sort optional
    • 8bb4a6d2 decode: Range decode with new decoder to preserve bit reader
    • 342612eb dev: Cleanup linters and fix some unused args
    • 78aa96b0 dev: Cleanup some code to fix a bunch of new linter warnings
    • 3570f1f0 doc: Add more related tools
    • 7aff654a doc: Clarify decode, slurp and spew args
    • 0863374f doc: Correct bencode spec URL
    • 10cc5518 doc: Improve and cleanup text formats
    • b1006119 doc: Typos and add note about Try* functions
    • c27646a6 doc: Update and shorten a bit
    • b0388722 doc: Use singular jq value to refer to jq value
    • a980656c doc: go 1.18 and improve intro text a bit
    • a64c28d9 dump: Skip JQValueEx if there are not options
    • 40481f66 elf,fuzz: Error on too large string table
    • f66a359c elf: Use correct offset to dynamic linking string table
    • 64f3e5c7 fairplay: Add basic SPC decoder and PSSH system id
    • cae288e6 format,intepr: Refactor json, yaml, etc into formats also move out related functions
    • e9d9f8ae fq: Use go 1.18
    • 377af133 fqtest: Cleanup path usage
    • 2464ebc2 fuzz: Replace built tag with FUZZTEST env and use new interp api
    • 0f78687b gojq: Fix JQValue index and destructuring issue and rebase fq fork
    • 59c7d0df gojq: Rebase fq fork
    • c57dc17d gojq: Rebase fq fork
    • 9a7ce148 gojq: Update rebased fq fork
    • c1a0cda5 gojq: Update rebased fq fork
    • 32361dee gojqextra: Cleanup gojq type cast code
    • 9b2e474e gojqextra: Simplify function type helpers
    • fd302093 hevc_vps,fuzz: Error on too many vps layers
    • efa5e23a icc_profile: Correctly clamp align padding on EOF
    • 1ddea1ad interp,format: Refactor registry usage and use function helpers
    • a3c33fc1 interp: Add group/0
    • 95e61965 interp: Add internal is helpers
    • 3b717c3b interp: Add to/from for some common serialzations, encodings and hashes
    • 6b088000 interp: Cast jq value to go value properly for encoding functions
    • f5be5180 interp: Cleanup and clarify some format naming
    • c7701851 interp: Extract to/from map/struct to own package
    • 8dde3ef5 interp: Fix crash when including relatve path when no search paths are set
    • 735c443b interp: Improve type normalization and use it for toyaml and totoml
    • 81a014ce interp: Make empty _finally fin error on error
    • 2dc509ab interp: Refactor dump and revert #259 for now
    • ab8c728a interp: Rename --decode-file to --argdecode to be more consistent
    • dff3cc11 interp: dump: Fix column truncate issue with unicode bars
    • 5109df4a interp: dump: Show address bar for nested roots
    • 80214921 interp: help: Fix incorrect options example
    • 76714349 mapstruct: Handle nested values when converting to camel case
    • c92f4f13 matroska: Update ebml_matroska.xml and allow unknown ids
    • c2a359bd mod: Update{crypto,net}
    • 3780375d mp3: Use d.FieldValueU and some cleanup
    • 7b27e506 mp4,bitio: Fix broken pssh decoding and add proper reader cloning to bitio
    • 6b00297e mp4,senc: Refactor current track/moof tracking and add senc box support
    • 8228ecae mp4: Add track id field and add track for tfhd with unseen track_id
    • ea2cc3c2 mp4: Don't decode encrypted samples
    • c6d0d89c mp4: Don't range sort samples, keep sample table order
    • 7d25fbfd mp4: Properly use trun data offset
    • ba844eb0 mp4: Skip fields for pcm samples for now
    • 0e02bb66 mp4: iinf: Only assume sub boxes for version 0
    • 2e328180 mp4: mvhd,tkhd,mdhd,mehd: Add version 1 support
    • 44bab274 readline: Rebase on top of tpodowd's redraw/completion fixes PR
    • a5122690 readline: Rebase on top of tpodowd's update PR
    • 54dcdce9 readline: Update fq fork
    • 6e7267d2 readme: add MacPorts install details
    • 76161a1b scalar,mp4,gzip,tar: Add timestamp to description
    • 9133f0e5 scalar: Add *Fn type to map value and clearer naming
    • 34cf5442 tcp: Split into client/server structs and add skipped_bytes and has_start/end per direction
    • 1aaaefb0 wav,bencode,mpeg_ps_packet,id3v1: Random fixes
    • 47350e46 zip: Add uncompress=false test and some docs
    • e6412744 zip: Add zip64 support and uncompress option
    • aa694e3f zip: s/Decompress/Uncompress/
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(556 bytes)
    fq_0.0.8_linux_amd64.tar.gz(3.20 MB)
    fq_0.0.8_linux_arm64.tar.gz(2.93 MB) MB) MB) MB) MB)
  • v0.0.7(May 1, 2022)


    • Format specific options
      • Formats can now have own options
      • Example to skip decoding of samples in a mp4 file use:
      • fq -d decode_samples=false d file.mp4 or ... | mp4({decode_samples: false})
      • To see supported options for a formats see formats documentation, use fq -h mp4 or help(mp4) in the REPL.
    • gojq fork rebase:
      • Many performance improvements from upstream
      • Assign to a JQValue will now shallowly turn it into a jq value and then be assigned.
      • Refactor and rewrote large parts to make it easier to rebase and maintain in the future.

    Decoder changes

    • amf0 Add Action Message Format 0 decoder #214
    • hevc_pps Add H.265/HEVC Picture Parameter Set decoder #210
    • hevc_sps Add H.265/HEVC Sequence Parameter Set decoder #210
    • hevc_vpc Add H.265/HEVC Video Parameter Set decoder #210
    • mp3 Add max_unique_header_config and max_sync_seek options #242
    • mp4 Simplify granule structure a bit #242
    • mp4 Add decode_samples and allow_truncate options #242
    • flac_frame Add bits_per_sample option #242
    • icmpv6 Add Internet Control Message Protocol v6 decoder #216
    • id3v2 Add v2.0 PIC support
    • ipv6_packet Add Internet protocol v6 packet decoder #216
    • macho Remove redundant arch struct level and cleanup some sym values #226
    • macho Add raw fields for section and encryption info #238
    • mp4 Add more HEIF boxes support #221
    • mpeg_pes Support MPEG1 #219
    • rtmp Add Real-Time Messaging Protocol decoder. Only plain RTMP for now. #214
    • matroska Symbol name cleanup #220
    • tcp Better port matching and make it possible to know if byte stream has start/end. #223
    • udp Better port matching #223


    • 010f6430 Update docker-golang from 1.17.8 to 1.18.0
    • 05096f50 Update docker-golang from 1.18.0 to 1.18.1
    • e5f61e22 Update github-go-version from 1.17.7, 1.17.7, 1.17.7 to 1.18.0
    • fdfc5c5b Update github-go-version from 1.18.0, 1.18.0, 1.18.0 to 1.18.1
    • 4ea362e3 Update github-golangci-lint from 1.44.2 to 1.45.0
    • 2a90485b Update github-golangci-lint from 1.45.0 to 1.45.2
    • d9195ac4 Update gomod-mapstructure from 1.4.3 to 1.5.0
    • cf88bc11 Update make-golangci-lint from 1.44.2 to 1.45.0
    • 3a0799cb Update make-golangci-lint from 1.45.0 to 1.45.2
    • 34cbe487 amf0: Decode strings in more detail
    • b2a865ea avc_sps: Add chroma format name mapping
    • b35b1804 decode,format: Add d.FieldFormatOrRaw(Len)
    • f4480c6f decode,interp: Support for format specific options
    • 5ff67e4c formats: Sym and field name cleanup to be more jq friendly
    • 3c029925 github: Update action versions
    • 02a97fa3 gojq: Rebase fq fork
    • 2e240447 gojq: Rebase fq fork
    • 518f6af4 gojq: Rebase fq fork
    • 88f791e0 gojq: Rebase fq fork
    • 8c918702 gojq: Rebase fq fork
    • adde8c70 gojq: Rebase fq fork
    • d79afeb3 gojq: Rebase fq fork
    • dd0d97ea gojq: Rebase fq fork (speedup and fix range with JQValue)
    • afd724bf gojq: Rebase fq fork. Fixes JQValue path tracking when iterating
    • 74978c9d hevc: Add hevc_vps, hevc_sps and hevc_pps decoders
    • c0202483 hevc_vpc,hevc_sps: Use same nameing for profile as in spec
    • 09385c61 id3v2: Add 2.0 PIC support
    • 9cb4b57a interp,cli: Handle ctrl-c properly
    • 607202bb interp: Don't truncate last display column
    • 6f03471d interp: Paths with a array as root was missing start dot
    • dabad850 interp: Proper display column truncate
    • e8678ca8 interp: Remove opts refactor leftover
    • d376520f interp: Remove to*range pad argument and fix stdout padding issue
    • 087d1241 interp: Simpler and more efficient hexdump
    • 21ad628a interp: dump: Show field name for compound values in arrays
    • e8dc7112 ipv6,icmpv6: Add decoder
    • d6c31dac macho: Add section and encryption_info raw data fields
    • 5424eed7 macho: Cleanup syms and remove redundant fat_arch struct
    • f8d79a57 matroska: More sym cleanup
    • f34ebd83 mp4: Add more HEIF boxes
    • f8fd6b7f mp4: Add more HEIF boxes
    • 39ba5c4d mpeg_pes: Support mpeg1 and some cleanup
    • d8aaf303 rtmp,amf0: Add decoders
    • 788b0ac1 rtmp,amf0: Improve decoders, aac asc, chunk stream interrupt, fix amf0 ecma arrays
    • 5d25bbc2 tcp,udp: Refactor and make port matching better
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(556 bytes)
    fq_0.0.7_linux_amd64.tar.gz(2.67 MB)
    fq_0.0.7_linux_arm64.tar.gz(2.44 MB) MB) MB) MB) MB)
  • v0.0.6(Mar 11, 2022)

    Added macho decoder (thanks @Akaame), nicer REPL interrupt, error and prompt, add slurp/spew functions and explode for binary.

    Added fq talk slides from Binary Tools Summit 2022 to


    • Major query rewrite refactor to share code for slurp-ish functions repl, slurp and future help system. #178
    • REPL improvements:
      • Much improved eval and output interrupt. Should fix more or less all issue with un-interruptable long outputs. It is still possible to get "hangs" if some decode value ends up being expanded into a huge string etc. #191 #192
      • Prompt paths now has colors support. #181
      • Shows an arrow on parse error.
      • Faster on multi inputs. #165
    • Speedup interpeter by skipping redundant includes. #172
    • gojq fork rebase: #179
      • Fixes try ... | ... catch precedence issue.
      • tonumber now supports non-base-10 numbers.
    • Add slurp/spew to collect outputs and outputs them later.
      • 1,2,3 | slurp("a") collects, later do spew("a") to output them. Also a global array $a will be available. #178
    • Add explode for binary. #188
      • "a" | tobits | explode return bits [0,1,1,0,0,0,0,1].
      • "åäö" | tobytes | explode return utf8 bytes instead of codepoints [195,165,195,164,195,182].
    • Add optional sub topic to --help: #177
      • Replace --formats with --help formats. #181
      • Add --help options to see all default option values. #181
    • Remove var, use slupr instead.

    Decoder changes

    • macho Add decoder. Thanks @Akaame #43
    • mp4 Support colr box. #176


    • ee5e4718 Update docker-golang from 1.17.7 to 1.17.8
    • ca04cc20 Update github-golangci-lint from 1.44.0 to 1.44.1
    • 5c6e1d32 Update github-golangci-lint from 1.44.1 to 1.44.2
    • 1b8e6936 Update make-golangci-lint from 1.44.0 to 1.44.1
    • 9d5ba826 Update make-golangci-lint from 1.44.1 to 1.44.2
    • cd2cbef6 decode: Some cleanup
    • 9e4f2641 dev: Add .jq-lsp.jq to add additional builtins for jq-lsp
    • c6a90cfc doc,asn1_ber: Add more documentation
    • c53bd777 doc: Add bts2022 video
    • b97776c9 doc: Add fq bts2022 presentation
    • d334c2d4 doc: Add href in supported format list
    • c95b0d6d doc: Forgot make doc
    • a202df9a doc: Improve and fix some typos
    • 9ec1d357 doc: Improve project description
    • 758b2d0e doc: Regenerate after macho merge
    • 920629f5 doc: Regenerate and fix macho section size
    • d3397cf9 doc: Tweak format diagram
    • d47e04c4 fixup! macho: CPU_SUBTYPE_MULTIPLE and TYPE_ALL are 0xff_ff_ff_ff
    • 27e76157 format: Simplify torepr, no need for _f function
    • 206dcd02 fuzz: Include more testdata seed files
    • be6f0093 gojq: Rebase fq fork and add support for non-10 base for tonumber
    • 33efb02a interp,repl: Add path and value colors to prompt
    • 41551de3 interp,repl: Improved eval and output interrupt
    • dff7e7da interp: Cleanup binary regexp overloading and add explode
    • fe8183b5 interp: Color parse in jq
    • 6f10745a interp: Fix interrupt regression after query rewrite refactor
    • f66f115e interp: Make _finally handle null, call fin once and last
    • eeb59152 interp: Make help output less wide
    • 9dc59e5d interp: Move _is_decode_value to jq
    • 0bc11719 interp: Move opts eval to options.jq
    • 3f50bb90 interp: Rework formats and options help
    • 03f450f8 interp: Skip redundant includes
    • c5918d23 macho: CPU_SUBTYPE_MULTIPLE and TYPE_ALL are 0xff_ff_ff_ff
    • 5c974209 macho: TS string to UTC
    • 04eae939 macho: add basic docs
    • 5e95d1c3 macho: add cpuSubTypes
    • 2638f419 macho: add darwin_amd64 test
    • 5c5bd879 macho: add fqtest actualization
    • bf214d5e macho: add nolint suppression to const defs
    • 333a3243 macho: add scalar.Hex mapper to addr fields
    • a86e7043 macho: add section type parsing
    • 90b94631 macho: adopt plural-singular scheme for FieldStructArrayLoop
    • b78ed02f macho: barebones decoder impl
    • e199d219 macho: basic impl for ar and fat file parsing
    • 66feebc5 macho: change parseFlags impl for ordered results
    • b5fe9ce6 macho: change registry description
    • 20e5be3f macho: delete ar decoder code
    • efdd0bf5 macho: discard lc_ and lowercase command names
    • b0911af2 macho: docs review changes
    • a29bfca5 macho: expand filetypes and header flags
    • fb0654ec macho: fix FieldUTF8NullFixedLen for segname
    • d1f093ce macho: fix fat header decode bug
    • 0d648928 macho: fix null in segname sectname
    • 9eb71dc6 macho: generate doc via make doc
    • 3991c51a macho: handle unknown lc_commands better
    • ef2919b3 macho: introduce arm and fat tests
    • 98c9840d macho: linting changes for ar parse
    • 1feb81c9 macho: little-endian to little_endian
    • 141a8e84 macho: mach_header_X to header
    • 9206d9d8 macho: magicToHex to scalar.Hex
    • 2021b054 macho: make actual
    • 70b84cde macho: ntools fix LC_MAIN fix
    • 78699f3a macho: parse flags individually
    • 4016ad0b macho: parse segment section flags
    • 5a48cb30 macho: refactor prebound_dylib
    • 2e7767cd macho: remake docs
    • 33347503 macho: reuse ar decoder
    • 228757b9 macho: review fixes
    • 5ee9a23c macho: review fixes
    • e3daee7d macho: simplify thread state decoder
    • 70c9d519 macho: thread state visualization
    • a4789dc1 macho: timestamp mapper
    • 2ccb8087 macho: update test cases v to dv
    • 74abe990 macho: update tests
    • 12eb7cc5 macho: use FieldUTF8NullFixedLen
    • 5f4ad410 macho: use FieldUTF8NullFixedLen for segname
    • f8690e6c mp4: Add colr box support
    • b157751a mp4: Reformat and use dv in test
    • 0a043f90 repl,interp: Refactor repl and slurp
    • ca8cdadb repl: Add comments and query from/to helper
    • 9cb4205b repl: Correct error arrow position in color mode
    • e238f292 repl: Speedup multi input to sub-repl
    • 56ae4a0c test: Make expect cli test more robust
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(556 bytes)
    fq_0.0.6_linux_amd64.tar.gz(2.57 MB)
    fq_0.0.6_linux_arm64.tar.gz(2.41 MB) MB) MB) MB) MB)
  • v0.0.5(Feb 13, 2022)

    Improved binary slicing and bit reading, avro_ocf decoder (thanks @xentripetal), asn1_ber decoder, renamed display aliases, new grep_by and paste function.


    • Big internal bit reader refactor. Now much more consistent code and fixes some issues reading and decoding of binary arrays and binary slices. #123
      • Bit reading and IO have been moved to a bitio package.
      • Non-simple bit reading have been move out of bitio to decode package.
      • [0,1,1,0,0,1,1,0,0,1,1,1,0,0,0,1 | tobits] | tobytes | tostring returns "fq".
      • [.frames[0], .frames[-1]] | mp3 decode mp3 based on first and last frame of other mp3.
    • Add grep_by that recursively selects using a filter condition and ignores errors. #102
      • grep_by(.type == "trak") finds all objects where .type is "trak" (all mp4 track boxes).
      • grep_by(tonumber | . >= 40 and . <= 100) find all numbers between 40 and 100.
      • grep_by(format == "jpeg") find all jpegs.
    • Add paste function to read string from stdin util ^D (EOF). Can be used to paste in REPl etc. #143
      • paste | frompem | asn1_ber | repl wait for PEM encoded text on stdin (paste and press ^D), decode it with asn1_ber and start a sub-REPL with the result.
      • paste | fromjson decode pasted JSON.
      • eval(paste) eval pasted jq expression.
    • Cleanup display aliases. Remove v and f, add da, dd, dv and ddv. #112
      • d/d($opts) display value and truncate long arrays and buffers
      • da/da($opts) display value and don't truncate arrays
      • dd/dd($opts) display value and don't truncate arrays or buffers
      • dv/dv($opts) verbosely display value and don't truncate arrays but truncate buffers
      • ddv/ddv($opts) verbosely display value and don't truncate arrays or buffers
    • Refactor radix into toradix($base)/fromradix($base). #139
    • Remove number_to_bytes. Can be done with tobytes. #139
    • Change tobytes to zero pad most significant bits to byte alignment if needed. #133
    • Add tobytes/tobits variant that takes an argument to add extra padding. #133
      • 0xf | tobytes 8 bit binary with last 4 bits set
      • 0xf | tobytes(4) 32 bit binary with last 4 bits set
      • 0xf | tobits(12) 12 bit binary with last 4 bits set
    • Rename fq type buffer to binary as it makes more sense. #133
    • Add topem/frompem to work with PEM encoding. #92
    • Add Windows scoop install. #137 Thanks @thushan
    • Add display, decode value, binary and binary array documentation. #118 #136 #133
    • Add decode API documentation. #149
    • Improved REPL completion for keys. #144
    • Add -o force=<bool> option that sets force decode option. Same as mp4({force: true}). #126

    Decoder changes

    • avro_ocf Add decoder. #38 Thanks @xentripetal
      • Full avro OCF support. Handles all primitive, complex, and logical types besides decimals.
      • Able to handle deflate, snappy, and null codecs for blocks.
    • asn1_ber Add decoder. #92
      • Also decodes CER and DER (X.690) but with no additional validation at the moment.
      • Support all types but real type is currently limited to range for 64 bit interger/float.
      • Has torepr support.
      • No schema support.
    • aac_frame Only decode object types we know about. #130
    • mp3 Shorter sync find heuristics. #104
    • mp4 Add stz2 support
    • mp4 Add pnot (preview container) and jP (JPEG 2000) signature. #125

    Also thanks to @Doctor-love for keeping things tidy.

    • 6fc1efd9 Add test case with all data types
    • ae4a6243 Adds Windows Scoop instructions for fq.
    • 4b809a73 Change avro codec to funcs
    • 66ca1f10 Change tests to use new verbose syntax
    • 7345b8c7 Cleanup
    • 07ddf36f Cleanup for linting
    • 4508241b Cleanup snappy
    • 0909fb6d Comment on snappy decompression
    • 21cfc70c Dates need to specify UTC too
    • 75b84961 Fix lint
    • 7a8e3ca2 Hook into registry, add codecs
    • 251053ef Initial pass on logical types
    • 2605bce4 Lint and add basic doc
    • ee184075 Parse header using avro decoders. Still not certain this is the best idea. Will get opinions before finalizing.
    • ab50088d Polish of problem template and clarifying questions
    • 27789f2d Regenerate docs
    • 5a1d35e7 Remove redudant question and fix typo
    • 31c4c0d3 Support snappy and deflate codecs
    • 0300c955 Take heading off doc to match make doc format
    • 6f57cdbf Timestamps should be UTC
    • 06085a26 Undo change to doc/file.mp4. I have no idea how this got changed in the first place? Maybe some macos shenanigans.
    • d137a72a Update docker-golang from 1.17.6 to 1.17.7
    • 267e30ec Update github-go-version from 1.17.6, 1.17.6 to 1.17.7
    • 1e859cda Update github-golangci-lint from 1.43.0 to 1.44.0
    • 16849c8f Update linting
    • d02d8968 Update make-golangci-lint from 1.43.0 to 1.44.0
    • 68e85a2d Use existing scalar description helper
    • 3bab3d65 aac_frame: Only try decode object types we know about
    • 0829c167 asn1_ber: Add decoder
    • 0312c92c asn1_ber: Add more doc and multiple outputs for frompem
    • 06245d12 binary,decode,doc: Rename buffer to binary and add some documentation
    • 7c521534 bitio,decode: Refactor bitio usage and make buffer slicing more correct
    • 0d74e879 bitio,doc: Even more cleanup
    • d854ed57 bitio: Cleanup documentation a bit
    • 82aeb355 bitio: More doc cleanup
    • 01ecde64 bump: Add snappy config
    • de64a99e cleanup some docs, change enum to mapper, error zigzag on more than 8 bytes
    • 6cd1c38f decode,scalar: Add scalar.Str{Uint/Int/F}ToSym to parse numbers
    • 4ab6381d decode: Add scalars args to FieldRootBitBuf
    • be71eb01 decode: Rename LenFn() to LimitedFn, add FramedFn and document
    • 7bc25219 doc,interp: Add some example usages to cli help
    • 8e47fb1a doc,matroska: Fix filesname in example
    • c15f5283 doc: Add format links to format table
    • b86da7ae doc: Add inital decoder API documentation
    • 49c90f89 doc: Add macOS security notes and move supported format up a bit
    • 06b67e4b doc: Add more license details
    • a8664ed5 doc: Add per format documentation
    • 09552628 doc: Add snappy license
    • 36307857 doc: Cleanup and add more decode value and binary documentation
    • 710c29b2 doc: Color edges in diagram based on dest
    • f0ce7179 doc: Document display and some more jq hints
    • b3504680 doc: More decode API details and polish
    • 6b51b067 doc: More display alias leftover fixes
    • dd3e40fb doc: Unbreak formats_digaram.jq since radix change
    • c52a1a23 doc: Use f($a; $b) instead of jq f/2 notation
    • 233d86a3 fq: Add arch and os to --version
    • b8efd8e5 fuzz: Fuzz all formats
    • e1bdfdf8 fuzz: List seed numbers and make it build again
    • 6090b65e fuzz: Make it compile again and run one format per fuzz
    • aea48847 github: Add basic issue template
    • b55ca2cd gojq: Rebase fq branch
    • 47c978e4 goreleaser: Use zip for macos
    • 85371173 id3v2: Should assert not validate magic
    • d6ca4818 initial work for avro OCF files
    • ca68e6a1 interp: Add Platform() method to OS interface
    • e792598c interp: Add grep_by/1 to recursively match using a filter
    • 0a1a5610 interp: Add missing default opts for tovalue
    • 48a19cb8 interp: Add paste function to allow pasting text into REPL etc
    • fc0aacb6 interp: Cleanup display aliases, now: d, da, dd, dv, ddv
    • bf7a483f interp: Fix handling of group decode error from stdin
    • 26d9650b interp: Refactor radix* into toradix($base)/fromradix($base)
    • 366f6b18 interp: Support force decode as -o force=true
    • 77ab667b interp: Use absolute path in errors
    • c31ec2a3 interp: Use correct sym color
    • 898dfec1 lint: Fix typeassert and case exhaustive warnings
    • d5401166 make doc
    • bf170be8 make: Cleanup some not very used targets
    • 8d2d88f4 mp3: Decrease max sync seek length between frames to 4k
    • d555c324 mp4,fuzz: Fatal error on infinite sgpd box entries
    • 45b00aab mp4: Add stz2 support
    • 092609be mp4: Add video preview (pnot) and JPEG 2000 (jP) signatures
    • febce5a5 mpeg_spu: Fatal error on infinite loop
    • c58ba28d mpeg_spu: Fatal error on unknown cmd
    • d1943dad pcapng,fuzz: Fix infinite loop by fatal error on block length <= 0
    • 2ab395a0 protobuf: Add note about sub message decoding
    • af053811 repl,interp: Make stdio work during completion
    • bd9be2c5 repl: Fix completion of non-underscore extkeys
    • 69c745d3 simplify scalar usage
    • 778a1a41 zip: Assert signature not validate

    (Some commits have been removed from list for clarity)

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(556 bytes)
    fq_0.0.5_linux_amd64.tar.gz(2.55 MB)
    fq_0.0.5_linux_arm64.tar.gz(2.39 MB) MB) MB) MB) MB)
  • v0.0.4(Jan 21, 2022)


    • Add a torepr/0 function that converts decoded value into what it represents. Initially works for:bencode, cbor, bson and msgpack #74 Example usage: fq torepr file.cbor fq -i torepr file.cbor fq torepr.field file.cbor fq 'torepr | .field | ...' file.cbor
    • Add stderr/0 function for jq compatibility #58
    • Bitwise operators band, bor etc are now normal functions instead of operators. Was done to be syntax compatible with jq #61 Uses the same convention as jq math functions: Unary uses input 123 | bnot, more than one argument all as arguments band(123; 456)
    • Decode API now supports null values #81
    • Decode API now supports arbitrary large integers #79
    • TCP reassembly now supports streams with missing SYN/ACK #57
    • Update readline package to version with less dependencies #83
    • Make REPL prompt more jqish #70

    Decoder changes

    • bencode Add decoder #64
    • cbor Add decoder #75
    • msgpack Add decoder #69
    • mp4 Much improved sample decode #82 #78
    • png Decode PLTE and tRNS chunks #59
    • tar Don't assume there is a end marker and support more than 2 blocks #86 #87

    Also thanks to @Doctor-love for keeping things tidy.


    • af8e7ef bencode: Add decoder
    • 0b0f28e cbor: Add decoder
    • 1383b41 decode,interp: Add arbitrary large integer support (BigInt)
    • 548a065 decode,interp: Finish up nil value support
    • ff5c0b8 decode: Error on negative number of bits when reading numbers
    • cf8a50c decode: Use stable sort for values to not change order or values with same range start
    • b4694b6 doc,dev: Add some more decoder implementation help
    • 0c1716b doc: Add alpine and go run
    • 809210b doc: Add more dev tips
    • 59b8803 doc: Document dev dependencies and related PRs/issues etc
    • 6ca4767 doc: Improve formats graph a bit
    • 8e9700d doc: Improve readme a bit and add torepr example
    • 0cf486d elf: fix all-platforms naming typo
    • 263f1ae flac: Don't allow zero subframe sample size
    • 729a6ca formats: Sort and make lists less likely to cause collision
    • 78c0775 fq: Embed version in source
    • aa7adb6 fq: Update version to 0.0.4
    • 7461264 fuzz: Skip other tests when fuzzing
    • be0ef80 interp,fq: Make bit operators normal functions
    • a3cfcd0 interp: Add stderr again for jq compat
    • 149cb3f interp: Add torepr/0 that converts decode value into what it reptresents
    • b3a0980 interp: Document bit opts funcs and add some error tests
    • 8d10423 make: Fix quote issue in release script
    • 4a1e859 mp4: Improved stsz handling
    • 61bf2ce mp4: Refactor sample decode into something more sane
    • a6bf62c msgpack: Add decoder
    • edad481 num,mathextra: Rename num package to mathextra
    • bfc977b png: Decode PLTE and tRNS chunks and cleanup syms a bit
    • 36d2891 readline: Update to verison with less deps
    • 9770b00 repl: Make prompt for array and iter more jqish
    • ba1edef tar: Allow more than 2 zero end blocks at end
    • 5921d76 tar: Don't assume there is a end marker
    • edd0ae1 tcp,flow: By default allow missing syn/ack for now

    (Some commits have been removed from list for clarity)

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(562 bytes)
    fq_0.0.4_linux_amd64.tar.gz(2.51 MB)
    fq_0.0.4_linux_arm64.tar.gz(2.35 MB)
    fq_0.0.4_macos_amd64.tar.gz(2.62 MB)
    fq_0.0.4_macos_arm64.tar.gz(2.59 MB) MB) MB)
  • v0.0.3(Jan 6, 2022)


    • Now works on Windows #52
    • Now builds and runs on 32-bit CPUs #30 @danfe
    • print/0, println/0 function now properly convert input to string if needed. #54
    • match functions now don't try to be smart with buffers, use. tobytes etc instead. Less confusing and also unbreak grep:ing decode values. #53 Now this works: fq 'grep("^strtab$") | parent | {name, string}' /bin/ls
    • Add Arch package #19 @orhun @ulrichSchreiner @dundee
    • Add Nix package #22 @siraben @jtojnar @portothree
    • Add FreeBSD port @danfe

    Decoder changes

    • bson Add Binary JSON deccoder
    • ar Add Unix archive decoder
    • bsd_loopback_frame Add BSD lookback frame decoder (used in pcap files)
    • elf Now does a two-pass decode to correctly handle string table references
    • elf Decode more sections: symbol tables and hashes
    • matroska Assert sane tag size only for strings
    • pcap Don't fail if incl_len > spanlen

    Also thanks to @Doctor-love @mathieu-aubin for keeping things tidy.


    • 628f0f4 bson: Add decoder
    • 46b59d0 crc: Unbreak build on 32-bit arch
    • 681dbc2 elf,ar: Add ar decoder, improved elf decoder
    • e5c620d github,ci: Add windows, macos and 32-bit linux
    • 52dddbb goreleaser: Use draft release to allow release note changes
    • e365f22 interp: Cleanup stdio usage and functions
    • 55b1d5c interp: Move _registry to decode
    • b6515c8 interp: Remove buffer smartness for regexp match functions
    • b867113 matroska: Assert sane tag size only for strings
    • b9aef39 pcap,pcapng,bsd_loopback_frame: Add decoder, refactor link frame into a group
    • af23eb8 pcap: Don't fail if incl_len > spanlen
    • a41f0d4 windows: Correct @builtin include path join
    • bf9e13c windows: Unbreak tests

    (Some commits have been removed from list for clarity)

    Source code(tar.gz)
    Source code(zip)
    checksums.txt(562 bytes)
    fq_0.0.3_linux_amd64.tar.gz(2.49 MB)
    fq_0.0.3_linux_arm64.tar.gz(2.33 MB)
    fq_0.0.3_macos_amd64.tar.gz(2.60 MB)
    fq_0.0.3_macos_arm64.tar.gz(2.57 MB) MB) MB)
  • v0.0.2(Dec 22, 2021)


    • 00f34c2 Update docker-golang from 1.17.3 to 1.17.4
    • 05b179c Update docker-golang from 1.17.4 to 1.17.5
    • c721ac8 Update github-go-version from 1.17.3, 1.17.3 to 1.17.4
    • befe783 Update github-go-version from 1.17.4, 1.17.4 to 1.17.5
    • 75aa475 decode: Generate Try?Scalar* methods for readers too
    • 9b683cd deocde: Cleanup some io panic(err)
    • e1e8a23 doc: Add color/unicode section and move config section
    • ee023d7 doc: Add some more related and similar projects
    • d02c7c4 doc: Add some more usage examples
    • be46d5f doc: Cleanup todo and add some dev notes
    • 47deb4d doc: Fix interpretation typo
    • fe68b51 doc: Improve readme text a bit
    • 916cb30 doc: Improve usage examples as bit
    • 5cb3496 docker: Fix broken build, copy fq.go
    • c2131bb flac: Cleanup scalar usage and fix incorrect sample rates
    • 1500fd9 gojq: Update to rebased fq fork
    • 3601fe3 gzip: fuzz: Don't uncompress on unknown compress method (nil create reader fn)
    • f4f6383 interp: Add ._index for values in arrays
    • 4558192 interp: Cleanup buffer code and implement ExtType()
    • e823475 interp: Fix help a bit
    • fa350c6 interp: Move display to jq
    • f7c7801 interp: Rework buffer regex support
    • 6ed2e2e interp: dump: Indicate arrays using jq-syntax
    • 9aec91a interp: match: Fix issue with regexp meta characters when matching using a buffer
    • e5e81e7 make: Fix prof build issue
    • e91b22b matroska,ebml: Use scalar and require sane tag size
    • eb9698f mp4,ctts: Seem more usable to treat sample count/offset as signed
    • c149732 mp4,trun,fuzz: Limit number of constant sample entries
    • 406263b mp4: Add comment about hdlr.component_name prefix byte
    • 28a3b71 mp4: Cleanup sample decode code
    • 6278529 mp4: fuzz: Make sure stsz has sane number of entries on constant sample size
    • 9f08af3 mpeg,aac: Factor out escape value decoding
    • dc1aea3 opus: Cleanup endian usage and fix incorrect preskip decode
    • 2b2320d pcap,flows: fuzz: Handle broken packets more nicely
    • 1d7ace3 pcap,pcapng,tcp: Use capture length not original length
    • b525d0b pcap: fuzz: Skip ssl2 packet if too short
    • 713ffe4 scalar: Add Require* and Require/Validate/AssertRange*
    • f348002 sll2: fuzz: Limit address length to max 8 bytes
    • eb4718f tar: Cleanup api usage
    • 45026eb tar: Cleanup constant usage a bit more
    • 91cc6d8 tar: Fix size decode regression after cleanup
    • 07a2ebe tiff,fuzz: Fatal error on infinite ifd loops
    • 91217e8 tiff: Fix endian typo and cleanup todos
    • 3850968 udp: Use proper udp payload format var name
    • 6a8d77b vorbis_comment: Cleanup endian usage and naming a bit
    • 57e9f41 vorbis_packet: Cleanup endian usage
    • f5a4d26 vscode: Use tabsize 2 for jq files
    • dd883b3 wav: Cleanup endian usage
    • e260830 webp: Cleanup endian usage
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(562 bytes)
    fq_0.0.2_linux_amd64.tar.gz(2.48 MB)
    fq_0.0.2_linux_arm64.tar.gz(2.32 MB)
    fq_0.0.2_macos_amd64.tar.gz(2.58 MB)
    fq_0.0.2_macos_arm64.tar.gz(2.55 MB) MB) MB)
  • v0.0.1(Dec 3, 2021)


    • 4242bf6 *_annexb: Refactor into avc/hevc_annexb
    • e86b45b Add *grep/1/2 and find/1/2
    • 36fd74a Add comment how raw byte regexp matching works
    • d1be167 Add decode struct each order test
    • 7f36f70 Add to/bytes/bits[range]
    • 571bf29 Change project title
    • 95ec5e1 Cleanup and rename s/BufferView/BufferRange
    • 9797cdc Cleanup dev/snippets.jq
    • 565f18d Correct avc_au format variable name
    • 6a1fa04 Decode hvc1 as hevc samplesa also
    • 74bad2d Fix broken value.fqtest
    • 514739a Give proper error on missing short flag
    • 57f0ec1 Improve cli help a bit
    • 9704659 Init
    • f33b310 Refactor decode.Value gojq bindings
    • 3d90b6d Remove fixed comment
    • e4e269b Rename and move cli test to pkg/cli
    • 1bd34bf Same args error behavior as jq
    • 3693667 Start of configurable json bit buffer formats
    • 590ee52 Update ci-golang from 1.17.0 to 1.17.1
    • 72a3f69 Update ci-golangci-lint from 1.42.0 to 1.42.1
    • 078cf29 Update docker-golang from 1.17.1 to 1.17.2
    • f5a8484 Update docker-golang from 1.17.2 to 1.17.3
    • 1371bc7 Update docker-golangci-lint from 1.42.1 to 1.43.0
    • f1ad700 Update github-go-version from 1.17.1, 1.17.1 to 1.17.2
    • 314bd17 Update github-go-version from 1.17.2, 1.17.2 to 1.17.3
    • 53a8d91 Update github-golangci-lint from 1.42.1 to 1.43.0
    • 898cd26 Update golang from 1.17.0 to 1.17.1
    • 31cd26e Update golangci-lint from 1.42.0 to 1.42.1
    • d4b2d58 Update gomod-mapstructure from 1.4.2 to 1.4.3
    • 1decf85 Update make-golangci-lint from 1.42.0 to 1.42.1
    • 233aaa1 Update make-golangci-lint from 1.42.1 to 1.43.0
    • 15e9f6f ansi: Correct background reset code
    • 1ccab2d apev2: Add test
    • 27e4770 apev2: Fatal if > 1000 tags
    • 3bf1a57 avc: Cleanup and add color names etc
    • 777191f avc: Correct sign expgolomb decode
    • 20021f4 bitio: Handle < 0 nbits
    • 7c4b0b3 bitio: Simplify by embedding reader
    • f600f2e build: Require go 1.17
    • 77f97aa builtin: Add chunk_by, count_by and debug
    • 27ba359 bump: Add action and cleanup names
    • 71e87e6 bump: Cleanup config, add config for release.yml
    • 8f2f524 bump: Move bump config to where it's used
    • 184df0a cli: A bit clearner array and iter prompt
    • d350971 cli: Add --decode-file VAR PATH support
    • 17104f0 cli: Add --options to make help a bit nicer
    • 6356a84 cli: Add --raw-string
    • 93fd097 cli: Add -M -C support and default to color if tty
    • 8dc0f06 cli: Add completion tests
    • 2010cac cli: Add error test
    • 706b2f2 cli: Add exit 2 (like jq) for no args
    • 569b631 cli: Add output join tests
    • 49f541c cli: Add proper repl iterator support
    • 3304f29 cli: Add string_input options test
    • 0dd848d cli: Better filenames in errors
    • 22eb53d cli: Cleanup MaybeLogFile
    • c14c29a cli: Cleanup and more commens
    • 10d7ed7 cli: Fix error filename on script error
    • 2b8d11d cli: Fix indent
    • 2d4eb9c cli: Fix non-string variables and var(; f) variant to delete etc
    • f5ffd32 cli: Implement --arg, --argjson and --rawfile
    • ae5566a cli: Include paths and some refactor
    • ec98fd3 cli: Make --argjson and --decode-file error similar
    • ac8cfca cli: Make --raw-string work with input/0 and inputs/0
    • b33f2cd cli: Make profile build optional and move it to cli
    • e2ff2a2 cli: Move help/0 to inter.jq, better help for -n
    • d0bb9a5 cli: Nicer grammar for --null-input help
    • 78eb737 cli: Nicer usage and indent input iteration
    • 394e2b3 cli: Only compelete at end or whitespace
    • 8d1fafe cli: Only show fq info for --help
    • 2684ed2 cli: Prepare completion for better variables support
    • e666380 cli: REPL and multiple files
    • 75cf46f cli: Refactor options code
    • f8ab00e cli: Remove unused eval debug arg
    • 5c8fb5f cli: Revert accidental history path change
    • 55cd45f cli: Simplify code
    • 2874bc7 cli: Unbreak colors in windows
    • 48517c7 cli: Unbreak part of completion
    • 518b725 cli: Use format/0 to check if value is a format
    • 21bef18 cli: User defined global vars
    • 538f4ff cli: add -nul-output compat
    • 85d1719 cli: jq compat, multiple -L
    • c8f0264 cli: more jq compat
    • f893295 cli: rename fq.jq to interp.jq
    • 1436fdc completion: Better and _internal handling
    • c7416e6 decode, interp: More buffer reuse
    • 6ee7977 decode,format: Allow root array
    • 58ba84f decode,interp: Add RecoverableErrorer interface instead of enumerate
    • b66ed32 decode,interp: Make fuzzing work again and cleanup fatal/error code
    • 5052bae decode,interp: Refactor to allow decode/fillgap a range
    • d4142b8 decode,png: Add FieldFormatReaderLen, refactor out zlib to format
    • 826c509 decode: Add Generated header for scalar
    • 3db11d3 decode: Add UTF8Fn functions and trim some null terminated strings
    • c083a9e decode: Fix MapRawToScalar regression
    • c17483d decode: Fix accidental rename
    • 6fba1a8 decode: Fix bitbuf root handling a bit
    • d1e1cd9 decode: Fix walk root depth issue causing dump to indent incorrectly
    • 1b32b42 decode: Major decode API refactor
    • 7af191d decode: Move io helper into *D
    • 26d615b decode: Move name/description into DecodeOptions
    • 986d5ec decode: Move registry package to decode/registry and add a format group type
    • ede2e77 decode: Nicer scalar template and add doc
    • 6207fcc decode: Pass context to be able to cancel properly
    • 5d98a69 decode: Refactor Error/Fatal into printf functions
    • 9f55b6e decode: Refactor and add symbol struct and mapping
    • 2fc0a71 decode: Refactor scalar usage
    • 8eaba88 decode: Refactor walk code a bit, add WalkRoot* to stay inside one root
    • f40320b decode: Remove D.Scalar* and add d.(Try)FieldScala*Fn instead
    • c155c89 decode: Rename format *Decode to *Format
    • f801cc0 decode: Rename s/FieldTryFormat/TryFieldFormat for consistency
    • 776a6b3 decode: Reuse read buffer per decode to speed things up
    • 9d116df decode: Rework use of TryFieldReaderRangeFormat
    • d48ebc1 decode: Simplify Compound.Children
    • 473b224 decode: Simplify and move format arg into DecodeOptions
    • 0480a2f decode: Some format decode and sub buffer work
    • a49e924 decode: Use for text decoding
    • af3e6b1 dev: Add format_summary.jq
    • 4c6de82 dev: Add summary how start to dump tree works
    • a926c8f dev: Some debug notes
    • b17a715 difftest: Remove accidental log
    • 1e1ad14 difftest: Sync code
    • 3cea849 dns: Cleanup a bit
    • d469edf doc,make: Correctly strip out graphviz version from svg
    • 15d85e1 doc: Add find/1 find/2
    • 26ea6d8 doc: Cleanup
    • a131210 doc: Cleanup and add note about . argument
    • 0a97f86 doc: Cleanup and note about repl limit
    • 8440e8a doc: Cleanup todo and dev a bit
    • 15b6d64 doc: Cleanup up a bit
    • 1047d90 doc: Document io packages a bit
    • 1a0089e doc: Fix typo and some improvments
    • 97c7403 doc: Fix typo in README
    • 0814206 doc: Fix typos and old examples
    • 0e8c82a doc: Improve readme goals a bit
    • 2f9d93d doc: Improved readme and cleanup todo
    • 242525f doc: Move formats to own file
    • 583bc38 doc: Regenerate demo.svg
    • a050adc doc: Regenerate demo.svg
    • 18e3e20 doc: Regenerate demo.svg with newer ansisvg
    • 1f61704 doc: Regenerate svg after ansisvg monospace update
    • a7459b3 doc: Some basic usage and cleanup
    • 6fa5ae8 doc: Some fixes
    • 00b7c18 doc: Some more doc work
    • 07c7daa doc: Some rewording
    • 5a82224 doc: Update
    • 21a74fa doc: Update
    • d6d3265 doc: Update README a bit
    • df5bd19 doc: Update TODO
    • fb13fe5 doc: Update todo
    • eee3c4e doc: Use unicode pipe in demo
    • b9b0326 doc: fq - jq for files
    • 790198c docker: Make image build again and install expect
    • 2387ec8 docker: Use golang 1.17
    • f8e5944 dump,json: Properly figure if compound or not
    • ee972f4 dump: Add ascii header
    • 69c6d15 editlit.jq: Update after mp4 field renames
    • 27095b5 es: Decode MPEG ASC if audio stream
    • 761c411 exif: Add note about JPEGInterchangeFormat
    • 344f628 flac: Calculate correct md5 when total samples count is zero
    • ce044ba flac: Cleanup
    • 2af08da flac: Fix block_size regression
    • 4f0bf92 flac: Make md5_calculated be a buffer
    • 797bd4d flac: Refactor flac_metadatablock into flac_metadatablocks
    • 1f26d4f flac_frame: Correctly read escaped samples and also a bit less allocations
    • bc9951c flac_frame: Fail if trying to decode outside block size
    • 723542a flac_frame: Make utf8Uint reader more correct and robust
    • b643e22 flac_frame: Support non-8 bit align sample size
    • 2daa738 flac_frame: Use d.Invalid for posssible errors
    • c1d9b4d flac_metadatablocks: typ > 127 can't happen, add app id
    • 984ba1a flac_metadatablocks: type >= 127 is invalid
    • 2b35d28 flac_picture,mp4: avif images and data fallback is image format fails
    • 509b8f8 flac_picture: Add picture_type names
    • 4f8d037 format,decode: Some crc and endian refactor
    • aa38ccf format,interp: Use MustGroup and add probe order test
    • 577c0f5 format: Add panic if register after resolve
    • 606c0b6 format: Add vorbis-comment-picture test, add .gitignore and cleanup
    • 798141a format: Cleanup comments
    • c0eebcc format: Remove unused ProtoBufType
    • 4b48828 format: Rename source file to match format name
    • d1b514e format: Some claeanup
    • ec97eca format: Split default.go into format.go and shared.go
    • 25f5ad7 fq,cli: Rename chunk to streaks, cleanup
    • 905c0ab fq: Add chunk/1
    • 46d37ef fq: Add cli sanity test
    • b849895 fq: Add truncate array support to dump/display
    • 8cb380e fq: Generate decode alises code
    • dfcefc1 fq: Make format/0 native for performance
    • ba273be fq: Make relative include work with @builtin etc
    • d23edaa fq: Rename bits/0 bytes/0 to tobits tobytes, remove string/0
    • a7a58c8 fq: Rename main.go to fq.go
    • 834f4a5 fq: use jq functions for all display alises
    • 31d7611 fqtest: Add env support and isterminal/width/height support
    • 285356d fqtest: Cleanup and dont assert when WRITE_ACTUAL
    • 86b34a3 fqtest: Fix section regexp
    • dee10db fqtest: No need to escape empty stdout
    • 6916880 fqtest: Refactor our script part to own package
    • 01d8a90 funcs: Add delta/0 delta_by/0
    • 96f7a75 funcs: Fix typo add count/0
    • 962d84d funcs: Make intdiv truncate to int
    • 08ec4f0 funcs: Remove unsued string function
    • d5c084c funcs: chunk_by comment
    • 80eaa46 funcs: format helper
    • c770a75 funcs: make in_bytes/bits_range more generic
    • 6ff5aca gif: Support GIF87a
    • 0d87018 github: Install expect to cli test
    • 8e3d9d8 github: bump: Checkout with bump token so it's used when push
    • 0c7fa09 gojq: Initial update support
    • 1888bb2 gojq: Remove div operator
    • d7dbe7c gojq: Update fq branch to fix mod (-1 % 256) difference
    • bf5c222 gojq: Update fq fork
    • 093ee71 gojq: Update fq fork
    • 1d15c1d gojq: Update fq fork
    • 3044fef gojq: Update fq fork
    • 5a5f01e gojq: Update fq fork
    • bfec366 gojq: Update fq fork
    • 4104a18 gojq: Update fq fork
    • 31a5047 gojq: Update fq fork
    • 0b90558 gojq: Update fq fork
    • 8277b79 gojq: Update fq fork, support JQValue alt //
    • a34784d gojq: Update fq gojq fork
    • 7ad3d25 gojq: Update fq gojq fork
    • f5164ee gojq: Update fq gojq fork
    • ac7568d gojq: Update gojq fq fork
    • f828ae1 gojq: Update gojq fq fork
    • 845bc6b gojq: Update gojq fq fork
    • d365ab7 gojq: Update gojq fq fork
    • 6420928 gojq: Update gojq fq fork
    • c8776ab gojq: Update gojq fq fork
    • f4cd7bf gojq: Update jq fork
    • b75da30 gojq: Update jq fork
    • 03af2b5 gojq: Update rebased fq branch
    • c3b7b5c gojq: Update to rebased fq branch
    • 2d6573d gojq: Update to rebased fq branch
    • fdb811e gojqextra,decode: Add generic lazy JQValue
    • aab32cf gojqextra,interp: Add lazy string to speed usage of decode value buffer where string is not used
    • 9035278 gojqextra: Move errors to own file
    • 86092e6 golangci: unused: Assume go 1.17
    • ac106d9 goreleaser: Cleanup
    • ee611a4 gzip,bzip2: Calculate CRC
    • 5344c7e icc: Add mluc support
    • 50d00d8 icc_profile: Add element alignment bytes field
    • 59e7fd4 icc_profile: Clamp alignment to end instead of check last tag (might not be last in buffer)
    • a5b802b icc_profile: Correct alignment byte count
    • 57a1207 id3v2: Support GEOB tags
    • f55b1af inet: Add tcp and ipv4 reassembly
    • 26c594f input: make -R mimic jq on io error
    • d9b45ba internal: _global_var returns new value instead of _global_state
    • f9f8660 interp,decode: Add force option to ignore asserts
    • 6a15625 interp,decode: Refactor out Scalar from Value and merge Array/Struct into Compound
    • cc5f405 interp,format: Update tests after decode refactor/tosym/toactual
    • 9f2dddc interp,gojqextra: Make buffers values even more lazy and error early on non-scalar calls
    • bf19588 interp: Add ansi helper
    • 7298a4c interp: Add buffer match support to find and grep
    • 0d693aa interp: Add line between usage and args help
    • c997536 interp: Add root, buffer_root, format_root, parent and parents
    • 69e4eea interp: Better error if format/group is not found
    • 7423f45 interp: Cleanup output types
    • 8d442b8 interp: Cleanup stdin reading and add more option tests
    • b641c77 interp: Cleanup unfinished/broken preview
    • 67898cb interp: Cleanup, use BufferRange for _open, progress for all decode
    • 0660ff0 interp: Clear up confusing --rawfile (add a jq alias)
    • 3fafee8 interp: Clearer help for -d
    • 879bb56 interp: Correctly check if _decode_value, add more has/1 tests
    • 2e964fa interp: Disable progress after decode is done
    • 80b5d66 interp: Document inputs and repl/cli details
    • 13fae09 interp: Don't print context cancel
    • 96cc128 interp: Eval options in jq instead of calling jq from go
    • 178032e interp: Fix $opts shadowing in decode
    • 4eccb1e interp: Fix broken aes_ctr, should return buffer instead of []byte
    • 110c86b interp: Fix broken dynamic width/height
    • 80a6997 interp: Fix file leak and always cache for now
    • cfdd922 interp: Fix prompt issue with format
    • 93322bc interp: Implement format/0 in jq
    • 8e5442f interp: Limit how often decode progress fn is called
    • f1fcbe5 interp: Make has/1 work for _ext keys
    • 826c8bd interp: Make include abs path work again
    • 6034ad7 interp: More sure stdOS stops the signal forward gorutine
    • 6cacc9b interp: Move *CtxWriter to ioextra, some comments cleanup
    • afb1050 interp: Move _decode_value to value.jq
    • 3e7e133 interp: Move formats func def to jq
    • 5cd5633 interp: Move jq function impls closer to where they belong
    • 996be0f interp: Move more options code into options.jq
    • 976e992 interp: Move progress logic to jq
    • 528e6b9 interp: Refactor and use mapstructure
    • 472c1ef interp: Refactor out string_input to own function
    • 54e121c interp: Refactor repl inputs a bit
    • 618c1ea interp: Refactor/Rename BufferView
    • d6d9484 interp: Remove --options, probably just confusing
    • b024316 interp: Remove accidental extra space after path in prompt
    • ff143d0 interp: Remove redundant decode arg
    • abcecb8 interp: Remove unused []byte type
    • 01a407a interp: Rename s/bufferrange/buffer and cleanup
    • 567bc4b interp: Rename to* funcs to match
    • 36e5562 interp: Rename value.* to decode.* as it makes more sense
    • 0cce5ec interp: Reorganize, move out repl and options, more functions to funcs.jq
    • eedfd16 interp: Replace find with overloaded match that support buffer
    • 58bf069 interp: Return []byte value as a buffer for now
    • 1c3c65b interp: Rework buffer, still confusing
    • 97f7317 interp: Rework repl prompt code and fix some whitespace issues again
    • 4af5739 interp: Rework string/buffer for decode values
    • ff2077b interp: Simplify Function, aliases done in jq now
    • 1fe5d95 interp: Some better naming and typos
    • 1b7b2f9 interp: Use gojqextra.NonUpdatableTypeError
    • 6ce4ba9 interp: Use snake_case for all options
    • 9cba69e interp: Use todescription in tests
    • cf26b1f interp: _readline: Use _repeat_break, add test
    • 07b4210 interp: add topath/0 and make todescription return null if there is none
    • 16d1f45 interp: find buffer should always use ByteRuneReader
    • 3ff0c9b lint: Enable errcheck adnd revive
    • 13e98d4 lint: Fix unused bufferRange and toBufferView
    • 13d5cbd lint: Remove usused nolint, should somehow tell about decode.Copy
    • ad850d6 lint: want explicit types in gojqextra.go
    • 90c19c6 make,test: Move testjq to own script and reuse fq binary
    • 052b9c0 make: Build with -s -w same as goreleaser
    • 1557e14 make: CGO_ENABLED=0 for static build
    • 8019212 make: Enable -race for tests
    • 263a77f make: Mark actual and cover as phony
    • b0694f5 make: Move build flags to vars
    • 75b59db make: Move doc generate to helper script
    • fcbfc29 make: Move into pkg/interp
    • 5bf4bc7 make: Rename testwrite to actual
    • 40e26e8 make: Rename to testjq
    • ab8080f make: Reorganize test targets
    • 98a8bae make: Silence git if no repo version found
    • 95b9c32 make: doc/formats.svg: Ignore graphviz verison to get less diff
    • 30ad643 make: go build args should not be quoted
    • f1507f7 mod: Use proper path and dont use replace
    • c4a3120 mp3,README: some cleanup
    • da386ea mp3: Be more relaxed with zero padding, just warn
    • 1325e5c mp3: Cleanup comments
    • e104748 mp3: Continue try find frames on error
    • 046f2fd mp3: Don't allow more than 64k between frames
    • 527f917 mp3: Error if > 5 unique header configs
    • 4344b62 mp3: Only look for supported mp3 sync headers
    • 8a4f66f mp3: Probe order after formwats with raw samples and similar sync headers
    • 60df04b mp3_frame: Only supports layer 3, fixes some probe issues
    • ed21f36 mp3_frame: Rename samples_per_frame to sample_count
    • db586eb mp4,matroska: Add *_path/0 variant that uses format_root
    • 9ac17bd mp4: Add comment about future truncate to size option
    • 2e71fa1 mp4: Add smhd box
    • 278e909 mp4: Add tapt, prof, enof and clap boxes
    • 161dcaf mp4: Better fragmented mp4 support
    • e47888e mp4: Fix 64bit size regression
    • 0801882 mp4: Fix field name typo for sample_composition_time_offsets_present
    • f322e78 mp4: More _time decoding
    • 6b8d26c mp4: Properly decode tfra
    • f2c1327 mp4: Use ISOBMFF naming and some more tests
    • 589207d mp4: Use descriptor field for all descriptor boxes
    • c7d45ff mp4: add pssh_playready format
    • e6cb708 mpeg: Nicer sym and description
    • f4b11b4 mpeg_annexb: Add format
    • 6a8ba31 number_to_bytes: Force int to make it work with bigint
    • 88eade9 ogg: Add flac support
    • 7f76986 ogg: Cleanup bitio in format out, maybe later
    • 7b7faaf pcap: Add pcap, pcapng, ether8023, ipv4, udp, udp
    • fc76907 png: Add proper color type
    • 5c733ad readline: Update fq fork
    • 4cee498 readme: Nicer demo
    • ffb5adf registry: Move to pkg/registry, feels better
    • dcceaa4 repl: Fix help and error message a bit
    • 44d8b66 repl: Give error if repl is used in non-repl mode
    • 1d0ebb5 repl: Handle directives, add tests
    • c9777aa repl: completion in jq
    • ccf6cab repl: use map in _query_slurp_wrap
    • 74b5750 shquote: Remove unnecessary sb.Reset()
    • 44251ca snippet: Add mp4_matrix_structure_rotation
    • 79a1aea snippets: Add urldecode
    • 684193a snippets: add changes/1
    • fda1dda snippets: urlencode: only 0-9a-f
    • 5ad048d tar: Fix 0 trim regression
    • 49d2e61 tar: Unbreak num parsing and add test
    • c8fad57 tiff: Fix reading of mluc tags with multiple records
    • b55f24a todo: Add ignore range check idea
    • bc1b3bf todo: Add note about symbols and iprint improvments
    • 1d83554 todo: Add note about test and capture with buffer
    • f839317 todo: Add some known issues
    • 4d94c9a todo: Clenaup a bit
    • 17a708f todo: Remove fixed repl item
    • 0af4c2b todo: Update about readline
    • d03a1c9 todo: add echo '{} {} {}' | jq difference
    • f9622c2 vorbis_comment: Fix field name typo
    • be0fdbe vp9: Add profile and fix reserved_zero field collision
    • 646f902 vpx_ccr: Add color names
    • b0ad3f2 w
    • 45afbe6 wip
    • d838d2f zip: Add format decoder
    • 9029143 zip: Fix nested decode for none compress
    Source code(tar.gz)
    Source code(zip)
    checksums.txt(562 bytes)
    fq_0.0.1_linux_amd64.tar.gz(2.48 MB)
    fq_0.0.1_linux_arm64.tar.gz(2.29 MB)
    fq_0.0.1_macos_amd64.tar.gz(2.59 MB)
    fq_0.0.1_macos_arm64.tar.gz(2.56 MB) MB) MB)
Mattias Wadman
Mattias Wadman
GoLang - Produces a binary suitable for use in shell scripts and cron jobs for rotating IAM credentials.

AWS-Rotate-IAM-Key aws-rotate-iam-key makes it easy to rotate your IAM keys whether they be in your ~/.aws/credentials file or else where. This work i

WarnerMedia Open Source 0 Feb 9, 2022
Analyze the binary outputted by `go build` to get type information etc.

Analyze the binary outputted by go build to get type information etc.

Masaaki Goshima 15 May 20, 2022
An experiment building a custom binary protocol for a calculator

Overview Implementation of a calculator service built on a custom protocol on top of TCP Details The server is in main.go, and the client is in client

Preetha 1 Nov 28, 2021
Transfer decimal ipv4 to binary ipv4

transfer decimal ipv4 to binary ipv4. Download: git clone

yifan 1 Jun 8, 2022
Exercise for solve problem data processing, performance and something wrong in passing data

Citcall Exercise Exercise for solve problem data processing, performance and something wrong in passing data Pengolahan data data processing - Readme

Muhammad Fazri 0 Nov 25, 2021
CodePlayground is a playground tool for go and rust language.

CodePlayground CodePlayground is a playground tool for go and rust language. Installation Use homebrews to install code-playground. brew tap trendyol/

Trendyol Open Source 47 Mar 5, 2022
dropspy is a (POC-quality) reworking of the C-language dropwatch tool in Go, with some extra features.

dropspy is a (POC-quality) reworking of the C-language dropwatch tool in Go, with some extra features. 28 Sep 10, 2022
Buffer Compactor is a tool to allow for buffering for a duration and compacting data on keys.

Buffer Compactor is a tool to allow for buffering for a duration and compacting data on keys. It uses a badgerDB and sortedset in order to coridinate a time-delayed queue that also aggregates updates sharing the same key in a extremely peformant manner.

Parker Roan 0 Feb 8, 2022
Generic mapStringInterface tool for extracting of data for CSV output

Generic mapStringInterface tool for extracting of data for CSV output

tbal999 1 Nov 2, 2021
A comphrehensive tool for converting between data table formats

tblconv A simple tool for converting one "table" format into another. Supported Formats CSV source output Excel source output SQL source output CLI Fe

null 0 May 30, 2022
The new home of the CUE language! Validate and define text-based and dynamic configuration

The CUE Data Constraint Language Configure, Unify, Execute CUE is an open source data constraint language which aims to simplify tasks involving defin

null 3k Sep 25, 2022
Lithia is an experimental functional programming language with an implicit but strong and dynamic type system.

Lithia is an experimental functional programming language with an implicit but strong and dynamic type system. Lithia is designed around a few core concepts in mind all language features contribute to.

Valentin Knabel 6 Jun 14, 2022
A stack oriented esoteric programming language inspired by poetry and forth

paperStack A stack oriented esoteric programming language inspired by poetry and forth What is paperStack A stack oriented language An esoteric progra

null 0 Nov 14, 2021
A toy language parser, lexer and interpreter written in Golang

Monkey - A toy programming language Monkey is a toy programming language used to learn how to write a lexer, parser and interpreter. The language is i

Adam Petrovic 1 Nov 16, 2021
Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.

Advent of Code 2021 Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved

Kemal Ogun Isik 0 Dec 2, 2021
Zach Howell 0 Jan 4, 2022
Various Dungeons and Dragons Tools. Written in go as an aid to learning the language.

dnd_tools Various Dungeons and Dragons Tools. Written in go as an aid to learning the language. Some tools are generic, while others will target eithe

SREiously 0 Jan 28, 2022
Some utilities for Persian language in Go (Golang)

persian Some utilities for Persian language in Go (Golang). Installation go get API .ToPersianDigits Converts all English d

هلو | آموزش زبان با بازی 65 Sep 2, 2022
Unit tests generator for Go programming language

GoUnit GoUnit is a commandline tool that generates tests stubs based on source function or method signature. There are plugins for Vim Emacs Atom Subl

Max Chechel 64 Sep 27, 2022