🏡 Gee is tool of stdin to each files and stdout

Overview


gee
Standard input to each files and stdout. similar to tee, But there's a little bit more and Write in go

πŸ”– Introduction

Gee is tool of stdin to each files and stdout. It is similar to the tee command, but there are more functions for convenience. In addition, it was written as go. which provides output to stdout and files. In this process, it has various processing functions for lines such as replace, prefix, and suffix, so it can be used as a pipeline configuration or as a utility for testing. For more information, see the usage and case of study below!

πŸš€ Installation

from source

$ go get github.com/hahwul/gee

with homebrew

$ brew tap hahwul/gee
$ brew install gee

β˜„οΈ Usage

$ ~/go/bin/gee -h (if you install from source)
$ gee -h
Usage: ./gee [flags] [file1] [file2] ...
(If you do not specify a file, only stdout is output)

Flags:
  -append
    	Append mode for files
  -chunked int
    	Chuked files from line (e.g output / output_1 / output_2)
  -debug
    	Show debug message!
  -distribute
    	Distribution to files
  -find string
    	Find string in line (colorize red)
  -format string
    	Change output format (json, md-table, html-table) (default "line")
  -grep string
    	Greping with Regular Expression (like grep)
  -grepv string
    	Inverse greping with Regular Expression (like grep -v)
  -prefix string
    	Prefix string
  -replace string
    	Replace string in line with '-find' option
  -rmnl
    	Remove newline(\r\n)
  -split string
    	Split string within line. (to line , to table, to md-table)
  -suffix string
    	Suffix string
  -uncolor
    	Uncolorize stdout
  -version
    	Version of gee
  -with-line
    	With line number (colorize blue)
  -with-time
    	With timestamp (colorize green)

πŸ“š Case of Study

gee with prefix and suffix

$ cat urls | gee -prefix "curl -i -k " -suffix " -H 'Auth: abcd'" curls.sh
curl -i -k https://www.hahwul.com/?q=123 -H 'Auth: abcd'
curl -i -k http://testphp.vulnweb.com/listproducts.php?cat=asdf&ff=1 -H 'Auth: abcd'
curl -i -k https://xss-game.appspot.com/level1/frame  -H 'Auth: abcd'

Find and replace

$ cat raw.txt | gee -find keep-alive
$ cat raw.txt | gee -find keep-alive -replace close

https://twitter.com/hahwul/status/1360495560843689989

Specify the maximum length of the file and save it in multiple files.

$ wc -l http.txt
2278

$ cat http.txt | gee -chunked 500 output

https://twitter.com/hahwul/status/1360495565633540097

Distribute each line sequentially to multiple files.

$ wc -l http.txt
2278

$ cat http.txt | gee -distribute alice.txt bob.txt charlie.txt

https://twitter.com/hahwul/status/1360495570922704897

🌟 Contribute

Contributions are always welcome. Please write/modify the code and send the PR.
Please read Contribute for more information.

🌸 Contributors

Comments
  • gee -distribute doesnt evenly distribute

    gee -distribute doesnt evenly distribute

    @hahwul I love gee. Im trying to work it more and more into axiom https://github.com/pry0cc/axiom.

    take a file with x number of lines and distribute for example

    seq 1 6 > file | gee -distribute 1 2 3
    
    cat 1 | wc -l
    3
    cat 2 | wc -l
    2
    cat 3 | wc -l
    1
    

    Gee should instead distribute evenly, expected result:

    cat 1 | wc -l
    2
    cat 2 | wc -l
    2
    cat 3 | wc -l
    2
    
    bug 
    opened by 0xtavian 4
  • Warning: Calling bottle :unneeded is deprecated! There is no replacement.

    Warning: Calling bottle :unneeded is deprecated! There is no replacement.

    Please report this issue to the hahwul/jwt-hack tap (not Homebrew/brew or Homebrew/core): /usr/local/Homebrew/Library/Taps/hahwul/homebrew-jwt-hack/Formula/jwt-hack.rb:9

    opened by hahwul 1
  • File write bug

    File write bug

    파일 κ²½λ‘œμ— - κ°€ μžˆλŠ” 경우 파일둜 μ €μž₯λ˜μ§€ μ•ŠμŒ

    정상

    $ wc -l docs/http.txt | cut -d " " -f 1 | ~/go/bin/gee -debug 33.txt  
    [DEBUG][MSG] Running on Debug mode
    [DEBUG][FILES] [33.txt]
    [DEBUG][MSG] Starting gee
    1132841
    [DEBUG][MSG] Finish gee, graceful shutdown...
    

    비정상

    $ wc -l docs/http.txt | cut -d " " -f 1 | ~/go/bin/gee -debug 3-3.txt
    [DEBUG][MSG] Running on Debug mode
    [DEBUG][FILES] []
    [DEBUG][MSG] Starting gee
    1132841
    [DEBUG][MSG] Finish gee, graceful shutdown...
    

    특수문자 λ³„λ‘œ λͺ¨λ‘ ν…ŒμŠ€νŠΈν•΄λ΄€κ³ , - 만 ν˜„μƒμž¬ν˜„

    bug 
    opened by hahwul 1
  • Spec

    Spec

    • [x] write to multiple files and stdout (#2)
    • [ ] ~input only stdin / only stderr / all (#3) -stdin -stdout -stdall~
    • [x] append to the given FILEs, do not overwrite (#4)-append
    • [x] write chunked files (size / line) (#5) -chunked-line and ~-chunked-size~
    e.g 
    + limit 10MB and filename=output , input 14MB
    - output-1 10MB 
    - output-2 4MB 
    
    • [x] with line prefix (#6) -p and -prefix
    • [x] with line suffix (#7) -s and -suffix
    • [x] with timestamp (#8) -with-time
    • [x] with linenumber (#9) -with-line
    • [x] remove newline(\r\n) (#10) -rmnl
    • [ ] ~to table format(markdown) (#11) --table-md~
    • [ ] ~to table format(html) (#12) --able-html~
    • [x] regex (grepping and ungrepping) (#13) -regex and -regexv
    • [x] split in line for prefix/suffix/table-* (#14) -split
    • [x] format -format (line md-table html-table json) (#14)
    • [x] replace in line-replace (#21)
    • [x] colorize and uncolor option -uncolor
    • [x] Distribution files (#15) -distribute
    e.g 
    + with --distribute and files(a.txt / b.txt / c.txt)
    input: a-l
    a.txt:
    a
    d
    g
    j
    
    b.txt:
    b
    e
    h
    k
    
    c.txt:
    c
    f
    i
    l
    
    • [x] Make docs page (#23)
    develope 
    opened by hahwul 1
  • -inject flag

    -inject flag

    -inject flag is -prefix + -suffix

    e.g

    $ cat 1234.txt
    1
    2
    3
    4
    
    $ cat 1234.txt | gee -inject "This is %%INJECT%% line"
    This is 1 line
    This is 2 line
    This is 3 line
    This is 4 line
    
    develope plan 
    opened by hahwul 0
  • Update module logrusorgru/aurora to v3

    Update module logrusorgru/aurora to v3

    WhiteSource Renovate

    This PR contains the following updates:

    | Package | Type | Update | Change | |---|---|---|---| | logrusorgru/aurora | require | major | v2.0.3+incompatible -> v3.0.0 |


    Release Notes

    logrusorgru/aurora

    v3.0.0

    Compare Source


    Renovate configuration

    :date: Schedule: At any time (no schedule defined).

    :vertical_traffic_light: Automerge: Disabled by config. Please merge this manually once you are satisfied.

    :recycle: Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

    :no_bell: Ignore: Close this PR and you won't be reminded about this update again.


    • [ ] If you want to rebase/retry this PR, check this box

    This PR has been generated by WhiteSource Renovate. View repository job log here.

    opened by renovate[bot] 0
  • Configure Renovate

    Configure Renovate

    WhiteSource Renovate

    Welcome to Renovate! This is an onboarding PR to help you understand and configure settings before regular Pull Requests begin.

    :vertical_traffic_light: To activate Renovate, merge this Pull Request. To disable Renovate, simply close this Pull Request unmerged.


    Detected Package Files

    • Dockerfile (dockerfile)
    • .github/workflows/contributors.yml (github-actions)

    Configuration Summary

    Based on the default config's presets, Renovate will:

    • Start dependency updates only once this onboarding PR is merged
    • Separate major versions of dependencies into individual branches/PRs
    • Do not separate patch and minor upgrades into separate PRs for the same dependency
    • Upgrade to unstable versions only if the existing version is unstable
    • Raise PRs immediately (after branch is created)
    • If semantic commits detected, use semantic commit type fix for dependencies and chore for all others
    • Keep existing branches updated even when not scheduled
    • Disable automerging feature - wait for humans to merge all PRs
    • Ignore node_modules, bower_components, vendor and various test/tests directories
    • Autodetect whether to pin dependencies or maintain ranges
    • Rate limit PR creation to a maximum of two per hour
    • Limit to maximum 20 open PRs at any time
    • Group known monorepo packages together
    • Use curated list of recommended non-monorepo package groupings
    • Ignore spring cloud 1.x releases
    • Ignore http4s digest-based 1.x milestones

    :abcd: Would you like to change the way Renovate is upgrading your dependencies? Simply edit the renovate.json in this branch with your custom config and the list of Pull Requests in the "What to Expect" section below will be updated the next time Renovate runs.


    What to Expect

    It looks like your repository dependencies are already up-to-date and no Pull Requests will be necessary right away.


    :question: Got questions? Check out Renovate's Docs, particularly the Getting Started section. If you need any further assistance then you can also request help here.


    This PR has been generated by WhiteSource Renovate. View repository job log here.

    opened by renovate[bot] 0
Releases(v1.1.2-action)
Owner
HAHWUL
Security Engineer and Developer and H4cker
HAHWUL
app-services-go-linter plugin analyze source tree of Go files and validates the availability of i18n strings in *.toml files

app-services-go-linter app-services-go-linter plugin analyze source tree of Go files and validates the availability of i18n strings in *.toml files. A

Red Hat Developer 2 Nov 29, 2021
Split text files into gzip files with x lines

hakgzsplit split lines of text into multiple gzip files

Luke Stephens (hakluke) 6 Jun 21, 2022
Easily create Go files from stub files

go-stubs Easily create .go files from stub files in your projects. Usage go get github.com/nwby/go-stubs Create a stub file: package stubs type {{.Mo

Sam Newby 4 Jan 27, 2022
Finder is a tool to sort and organize your files.

Finder ?? Finder is a tool to sort and organize your files. Installation ?? Currently, we only support the installation via go directly as shown below

Nicklas Frahm 0 Jan 23, 2022
This is a tool to extract TODOs, NOTEs etc or search user provided terms from given files and/or directories.

ado This is a tool to extract TODOs, NOTEs etc or user provided terms from given files and/or directories. DEPRECIATED: My project seek has cleaner co

Meelis Utt 0 Aug 11, 2022
A dead simple tool to rename your files for smooth web access!

ffw - Friendly Files for the Web Easily rename files from a folder to be compatible with the web Run ffw and that's it! Installation on macOs brew tap

Vincent Gschwend 2 Jan 31, 2022
A blazingly-fast simple-to-use tool to find duplicate files on your computer, portable hard drives etc.

A fast and simple tool to find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.

Manu Manjunath 215 Sep 7, 2022
A tool for moving files into directories by file extensions

The tool for moving files into directories by file extensions Example before moving structure: moving into same extension dir result: moving into diff

Uladzislau Jum 0 Dec 6, 2021
go tool for working with /etc/hosts files

hostsfile This library, and the associated command line binary, will help you manipulate your /etc/hosts file. Both the library and the binary will le

Kevin Burke 120 Jul 27, 2022
Vaala archive is a tar archive tool & library optimized for lots of small files.

?? Vaar ?? Vaala archive is a tar archive tool & library optimized for lots of small files. Written in Golang, vaar performs operations in parallel &

Qing Moy 14 Sep 12, 2022
QueryCSV enables you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to a CSV file

QueryCSV enable you to load CSV files and manipulate them using SQL queries then after you finish you can export the new values to CSV file

Mohamed Shapan 100 Dec 22, 2021
Golang PDF library for creating and processing PDF files (pure go)

UniPDF - PDF for Go UniDoc UniPDF is a PDF library for Go (golang) with capabilities for creating and reading, processing PDF files. The library is wr

UniDoc 1.7k Sep 22, 2022
Go (golang) library for reading and writing XLSX files.

XLSX Introduction xlsx is a library to simplify reading and writing the XML format used by recent version of Microsoft Excel in Go programs. Current s

Geoffrey J. Teale 5.4k Sep 23, 2022
Golang library for reading and writing Microsoft Excelβ„’ (XLSX) files.

Excelize Introduction Excelize is a library written in pure Go providing a set of functions that allow you to write to and read from XLSX / XLSM / XLT

360 Enterprise Security Group, Endpoint Security, inc. 12.9k Sep 25, 2022
Easily create & extract archives, and compress & decompress files of various formats

archiver Introducing Archiver 3.1 - a cross-platform, multi-format archive utility and Go library. A powerful and flexible library meets an elegant CL

Matt Holt 3.7k Sep 26, 2022
A Go filesystem package for working with files and directories

Stowage A Go filesystem package for working with files and directories, it features a simple API with support for the common files and directories ope

null 19 May 28, 2021
🌳 πŸ“‚ The utility displays a tree of directories and files(symlinks in future).

dirTree The utility displays a tree of directories and files. usage: dirTree [-f] How it works with directory, where I wrote this project for example

Hutiev Alexander 2 Aug 12, 2021
Allows parsing CSV files into custom structs and implements required fields that can't be empty

Welcome to Go Custom CSV Parser ?? Allows parsing CSV files into custom structs and implements required fields that can't be empty ?? Homepage Install

Elmedin TurkeΕ‘ 2 Nov 9, 2021
Utility for working with files and folders stored on Google Drive

skicka Utility for working with files and folders stored on Google Drive. Note: skicka is not an official Google product! Intro skicka makes it easy t

null 0 Nov 15, 2021