A fast and powerful alternative to grep

Related tags

Command Line sift
Overview

sift

A fast and powerful open source alternative to grep.

Features

sift has a slightly different focus than most other grep alternatives. Code search, log search / digital forensics and data processing are the main use cases, but the primary goal is to provide safe defaults and to make it easily configurable for a specific use case. Among the features are:

  • Stable releases, cross platform support
  • Safe defaults: sift searches everywhere if not configured otherwise
  • Complete & working .gitignore support
  • High performance for many uses cases
  • Support for adding custom file types to narrow down searches
  • Multiline support
  • Support for big files: >50GB, >5,000,000,000 lines and >5,000,000,000 matches successfully tested

Sift allows easy customization. Example: Configure sift to

  • Ignore case
  • Show line numbers
  • Skip binary files
  • Respect .gitignore files:

sift -i -n --binary-skip --git --write-config

The configuration can be overridden for specific directories.

sift understands conditions to process complex formats or support code audits.

Example: Search for .php files containing a call to mysql_query, that is preceded by $_GET or $_POST (accessing external input), but the preceding 5 lines do not contain a call to an escape function:

sift -x php mysql_query --preceded-within "5:_(GET|POST)" --not-preceded-within "5:escape"

Please go to sift-tool.org for more information.

Installation

Download Binaries

You can download binaries for the current version at https://sift-tool.org/download.

sift is available for Linux, Windows, OS X and *BSD.

Install from Package Repositories

Arch Linux

Download and install the binary from http://sift-tool.org/download:

$ yaourt -S sift-bin

Or build and install sift from source at https://github.com/svent/sift:

$ yaourt -S sift

OS X

Using Homebrew:

$ brew install sift

Install with Working Go Environment

If you have a working go environment, you can install sift using "go get":

go get github.com/svent/sift

Contributing

Feature Requests

If there is a feature or option you would like to see in sift, please open an issue and describe what you are missing. Where possible, please include an example (input file, expected output etc.) to better convey your idea.

Bugs / Unexpected Behavior

If you found a bug, please check the open issues and the limitations and restrictions described in the documentation. If you cannot find any documentation about it, please open a new issue, name the sift version you used and describe the steps to reproduce the problem.

Pull requests

Please do not send pull requests and open an issue instead as accepting substantial contributions cannot be done correctly without some legal hassle. Moreover, this allows me to consider already planned features while implementing smaller changes.

License

Copyright (C) 2014-2016 Sven Taute

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 3 of the License.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.

Comments
  • Request: support to ignore files/dirs from .gitignore

    Request: support to ignore files/dirs from .gitignore

    I'd love sift to understand .gitignore or similar files for other VCS. Doc suggest using sift --exclude-dirs '.git' --write-config so I could probably manually transfer contents of .gitignore to the sift config, but the problem is: .gitconfig is different for every project while sift config seems to be the same for all projects.

    enhancement 
    opened by jodaka 10
  • Suppress very long lines error

    Suppress very long lines error

    Is there any way to suppress these errors?

    Error: cannot process data from file 'some/file.txt': file contains very long lines (no newline in 262144 bytes)
    

    I'm getting tons of these errors, and it makes it hard to see the actual results.

    enhancement 
    opened by mwean 7
  • Add .sift.config location relative to the executable.

    Add .sift.config location relative to the executable.

    Currently sift scans for config file in home directory and current working directory. Could you incorporate the directory sift.exe is in? It would improve portability.

    opened by vovcacik 6
  • [WORKAROUND] Lookbehind and lookahead possible?

    [WORKAROUND] Lookbehind and lookahead possible?

    I/O

    echo Saving to: 'hello.sql' | sift --only-matching (?<=').+(?=')
    The filename, directory name, or volume label syntax is incorrect.
    

    Expected

    hello.sql
    

    It seems error occurs because cmd.exe fails to process special chars, right?

    However the following piece confuses me whether issue is about cmd.exe of Sift itself.

    echo Saving to: 'hello.sql' | sift --only-matching -e "(?<=').+(?=')"
    Error: cannot parse pattern: error parsing regexp: invalid or unsupported Perl syntax: `(?<`
    
    opened by sergeevabc 5
  • mutiple options with --dirs doesn't work in Windows

    mutiple options with --dirs doesn't work in Windows

    Not Working: sift --dirs core text // Output: matching strings but skipped --dirs restriction. sift --dirs core -x js text // Output: Nothing

    Tested in sift 0.3.2 Tested in following shells:

    • Windows Command Line
    • Windows GIT Bash
    • Powershell
    need-feedback 
    opened by KayLeung 5
  • Benchmarks are non-sense

    Benchmarks are non-sense

    Hello,

    I took a look at your benchmarks and they don't quite make sense. You claim you managed to read 35GB of data split over 32 files in under 0.5s? How is this possible? You'd need read speeds of over 70GB/s to achieve this. On normal machine with PCI-Express 3.0 x16 you have ~4GB/s bandwidth, and this is the fastest interface in the computer.

    Can you elaborate how you measured these magic results and what setup you used? Thank you

    opened by lunemec 5
  • Exit code 1 and no results when run without `.` path indication in Neovim

    Exit code 1 and no results when run without `.` path indication in Neovim

    Hi,

    I noticed a somewhat weird problem when running sift in Neovim. Initially I thought it's connected with vim-grepper so I opened an issue there.

    Marco Hinz noticed that running sift in Neovim doesn't work even without vim-grepper when there's no search path indication.

    Basically: :!sift -n foo yields shell returned 1 and no results. :!sift -n foo . works just fine. There is no problem with 'vanilla' Vim.

    I tracked the issue to a4bc2b9 - sift built at this point yields the aforementioned error, e109c32 works ok.

    My sift binaries were built with go version go1.5.1 linux/amd64 (git checkout COMMIT_ID; rm ./sift; go build).

    opened by moon-musick 4
  • Feature - Add option to get a global result count

    Feature - Add option to get a global result count

    Sift already includes an option (--count) to display a simple count of results per file. Unfortunately, there's no easy way to then get a single count of all found matches.

    Ack currently has this ability by combining the --count and --no-filename options, which is quite honestly very awkward, but it works.

    The alternative is to pipe the Sift output to a script that adds all of the lines in the output. Unfortunately, that's quite awkward to do in a linux shell, let alone Windows:

    screen shot 2016-01-08 at 11 08 05 am

    If this feature/option were to be added to Sift, I'd personally suggest not following the Ack way of doing things, as the dual options are awkward and hard to discover, but that's just me. :stuck_out_tongue:

    opened by Rican7 4
  • Request: Allow --type, --ext and --files

    Request: Allow --type, --ext and --files

    It would be great to select files like (for example):

    sift --type=cc,cpp --files=CMakeLists.txt --files=Makefile --ext=mak,def
    

    Currently, one can use multiple --type, --file and --ext, but not mixed. Maybe, allowing creating new types in .sift.conf?

    I'm very close to fully replace ack with sift! Great work.

    opened by MiguelLatorre 4
  • Feature - Add `-g PATTERN` like option to enable smart file listing

    Feature - Add `-g PATTERN` like option to enable smart file listing

    Both Ack and The Silver Searcher have a -g PATTERN command that enables for fast file listing based on expression matching the file path itself, while still keeping in mind configuration settings (like ignoring certain file types from showing in results).

    While this -g flag enables these applications to perform quite differently (matching against path/filename rather than actual contents), it allows them to be used in interesting contexts, such as the driver to CtrlP indexing. More info regarding the benefits of this use case are spelled out in @thoughtbot's blog post "Faster Grepping in Vim".

    opened by Rican7 4
  • Feature - Add option to ignore the config file

    Feature - Add option to ignore the config file

    Ack has a --noenv option to allow for ignoring any configuration/settings defined in the environment or in the .ackrc file. Since sift has a similar ability to configure it's usage through a config file, it would be a nice addition to be able to ignore it temporarily.

    This is very useful for testing queries with different settings or for relying on a default behavior in a script.


    PS: Great tool! I love it!

    opened by Rican7 4
  • Sift breaks on macOS Monterey

    Sift breaks on macOS Monterey

    Sift and macOS 12 do not get along:

    [email protected] Desktop % sift
    fatal error: runtime: bsdthread_register error
    
    runtime stack:
    runtime.throw(0x1ed7c0, 0x21)
    	/opt/go-1.7.1/src/runtime/panic.go:566 +0x95 fp=0x7ff7bfeff8f0 sp=0x7ff7bfeff8d0
    runtime.goenvs()
    	/opt/go-1.7.1/src/runtime/os_darwin.go:88 +0xa0 fp=0x7ff7bfeff920 sp=0x7ff7bfeff8f0
    runtime.schedinit()
    	/opt/go-1.7.1/src/runtime/proc.go:450 +0x9c fp=0x7ff7bfeff960 sp=0x7ff7bfeff920
    runtime.rt0_go(0x7ff7bfeff990, 0x1, 0x7ff7bfeff990, 0x1000, 0x1, 0x7ff7bfeffad8, 0x0, 0x7ff7bfeffadd, 0x7ff7bfeffb05, 0x7ff7bfeffb3e, ...)
    	/opt/go-1.7.1/src/runtime/asm_amd64.s:145 +0x14f fp=0x7ff7bfeff968 sp=0x7ff7bfeff960
    [email protected] Desktop %
    

    More than happy to provide whatever is necessary to help debug and test.

    Sift 0.9.0 standalone download. Brew version seems to work fine.

    opened by reafian 2
  • Request:  replace to allow escaped characters (\t, \n, etc)

    Request: replace to allow escaped characters (\t, \n, etc)

    Currently, a call to swift with echo 'abcde' | sift '(.*)c(.*)' --replace='$1\n$2' results in ab\nde. Instead of a carriage return character, it includes a '\' char followed by a 'n' char.

    Expected result: replace \n with a carriage return. ab de

    opened by mmicheo 0
  • --binary-skip doesn't skip archives

    --binary-skip doesn't skip archives

    Hello! Thanks for this great utility.

    Today I was trying to search for something in repos where I have binaries, archives and loads of source code, and even though the sift skipped the binaries, it didn't skip the archives.

    Could somebody look at this issue please?

    Thank you!

    opened by onionka 0
  • Is this a bug with--exclude-dirs?

    Is this a bug with--exclude-dirs?

    If I do --exclude-dirs "subdir/subdir2", then the --exclude-dirs gets ignored. It looks like I cannot have / in the directory. Is this expected?

    opened by ghbren 0
  • Command line option to NOT follow symlinks

    Command line option to NOT follow symlinks

    There's a configuration knob to switch this behavior, but on command line one can only enable sift to follow symlinks (--follow). Would it be feasible to add a counter-part command line option to disable this behavior?

    Use case: most of the time I'd prefer to follow symlinks but there are some quite rare cases when I do not want this behavior. Editing ~/.sift.conf to achieve this is much more toil rather than putting e.g. --no-follow on command line.

    Thanks in advance.

    opened by yermulnik 0
Releases(v0.9.0)
  • v0.9.0(Oct 23, 2016)

    Version 0.9.0 introduces several new options/features:

    • --conf
    • --field-sep
    • --byte-offset
    • Support for custom types

    A detailed description can be found in this blog post: https://svent.github.io/2016/10/23/sift-version-0.9-released/

    This version also removes the cgo dependency while improving the performance for line counting and case conversions.

    The latest version of can sift can be downloaded here: https://sift-tool.org/download

    Source code(tar.gz)
    Source code(zip)
  • v0.8.0(Feb 8, 2016)

    Version 0.8.0 introduces several new features:

    • search using smart case via option -s or --smart-case
    • the options ipath and exclude-ipath are availabe as case insensitive alternatives to path and exclude-path
    • follow symlinks via --follow
    • show column numbers for matches via --column (useful for editor integration etc.)

    The latest version of can sift can be downloaded here: https://sift-tool.org/download

    Source code(tar.gz)
    Source code(zip)
  • v0.7.1(Jan 7, 2016)

    Version 0.7.1 fixes a bug in the new --targets option that can lead to messed up output in some environments.

    The latest version of can sift can be downloaded here: https://sift-tool.org/download

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(Jan 4, 2016)

    This release contains a small performance improvement and implements several feature requests/issues:

    • #46 (Implement option --word-regexp/-w to match on word boundaries only)
    • #49 (Improve handling of line length errors)
    • #51 (Fix handling of custom output separator for multiline matches)
    • #53 (Implement option --only-matching)
    • #47 (Implement option --targets to list selected files)

    The last one enables fast file listing based on sifts filtering capabilities without actually searching the files. As @Rican7 mentioned in the issue, this allows e.g. to use sift as driver for CtrlP (a vim plugin).

    The result is much more powerful than just implementing the option -g known from ack/the silver searcher - you can now use sift to list files and combine all filtering options, not just filter the path based on a regular expression.

    An example based on the linux kernel source code (version 3.18.2): sift --targets --git --path net -x c --files '*test*' --exclude-dirs wireless

    This lists all files that:

    • contain "net" in the path ("net" is used as a regular expression here, like in the mentioned -g option)
    • are c code files (extension *.c)
    • contain "test" in the filename

    And while searching, sift will

    • respect .gitignore files
    • not descend into directories named "wireless"

    While your average use case should be much less complex, this shows the flexibility of the solution.

    You can download the new binaries here: https://sift-tool.org/download

    Source code(tar.gz)
    Source code(zip)
Ddg - DuckDuckGo terminal search that plays nice with grep and fzf

ddg duckduckgo terminal search for use with fzf (or grep) What? With this tool y

Cauê Felchar 3 Apr 1, 2022
vgrep - a user-friendly pager for grep

vgrep is a pager for grep, git-grep, ripgrep and similar grep implementations, and allows for opening the indexed file locations in a user-specified e

Valentin Rothberg 590 Nov 18, 2022
🚀 Platform providing a powerful and fast public script parsing API dedicated to the Skript community.

SkriptMC-Parser is currently a prototype in the early stages of development of a system that allows the Skript community to test their scripts via a public API for potential errors or warnings. This is a quick and easy way to check your scripts without having to set up a Spigot server on your environment.

Romain 0 Mar 3, 2022
archy is an static binary to determine current kernel and machine architecture, with backwards compatible flags to uname, and offers alternative output format of Go runtime (i.e. GOOS, GOARCH).

archy archy is an simple binary to determine current kernel and machine architecture, which wraps uname and alternatively can read from Go runtime std

xargs-dev 3 Mar 18, 2022
giter8 alternative in Go

go-giter8 go-giter8 implements a simple library capable of handling giter8 templates. Using as library Add to go.mod module github.com/yourusername/yo

Gympass 5 Sep 5, 2022
An alternative syntax to generate YAML (or JSON) from commandline

yo An alternative syntax to generate YAML (or JSON) from commandline. The ultimate commanline YAML (or JSON) generator! ... I'm kidding of course! but

Luca Sepe 10 Jul 30, 2022
🐘 Cross-platform, neofetch alternative for fetching system info.

elefetch ?? Cross-platform, neofetch alternative for fetching system info. Installation go get: go get -u github.com/burntcarrot/elefetch Binaries Bin

Aadhav Vignesh 1 Jan 23, 2022
💾 A visual du(1) alternative for human beings

dux A disk usage analyzer. Like du(1), but displays the results as an interactive treemap. testdata 111B (8 files) (4) ┌testdata/ 111B────────────

Jens Green Olander 1 Sep 6, 2022
A powerful modern CLI and SHELL

Grumble - A powerful modern CLI and SHELL There are a handful of powerful go CLI libraries available (spf13/cobra, urfave/cli). However sometimes an i

梁辰风 0 Dec 30, 2021
Mcli - A mininal and very powerful cli library for Go

mcli mcli is a minimal but powerful cli library for Go. m stands for minimal and

Shawn Wang 9 Nov 18, 2022
Flag is a simple but powerful command line option parsing library for Go support infinite level subcommand

Flag Flag is a simple but powerful commandline flag parsing library for Go. Documentation Documentation can be found at Godoc Supported features bool

null 123 Sep 26, 2022
`tmax` is a powerful tool to help you get terminal cmd directly.

The positioning of tmax is a command line tool with a little artificial intelligence. If you frequently deal with the terminal daily, tmax will greatly improve your work efficiency.

Jeremy 32 Oct 15, 2022
A powerful cli tool to implement gin annotation ⭐

gin-annotation A powerful cli tool to implement gin annotation Chinese Document Features Using code generating technology by operating golang AST Rout

Tang WenXuan 49 Mar 24, 2022
Building powerful interactive prompts in Go, inspired by python-prompt-toolkit.

go-prompt A library for building powerful interactive prompts inspired by python-prompt-toolkit, making it easier to build cross-platform command line

Masashi Shibata 4.7k Dec 2, 2022
A powerful little TUI framework 🏗

Bubble Tea The fun, functional and stateful way to build terminal apps. A Go framework based on The Elm Architecture. Bubble Tea is well-suited for si

Charm 16.1k Dec 3, 2022
Powerful CLI written in GO to generate projects in various technologies

Barca CLI is a project generator written in GO and its purpose is to build and configure HTTP servers, web proxy, SPA/PWA, Blog and custom landing page. It's easy, fast and productive.

Barca 2 Aug 26, 2022
A lightweight but powerful OCR tool.

这个项目是什么? LOCR(Lightweight OCR)是一款轻量级的文字识别工具, 结合第三方截图工具, 可以快速的对图片文字进行识别。 为什么有这个项目 在日常学习的工作中, 难免会遇到一些文字的复制粘贴任务。但由于一些限制,我们无法复制想要的文字,只能一个字一个字的敲出来。而随着近几年OC

lingyin 8 Nov 1, 2022
Sloc, Cloc and Code: scc is a very fast accurate code counter with complexity calculations and COCOMO estimates written in pure Go

Sloc Cloc and Code (scc) A tool similar to cloc, sloccount and tokei. For counting physical the lines of code, blank lines, comment lines, and physica

Ben Boyter 4k Dec 6, 2022
Fast, realtime regex-extraction, and aggregation into common formats such as histograms, numerical summaries, tables, and more!

rare A file scanner/regex extractor and realtime summarizor. Supports various CLI-based graphing and metric formats (histogram, table, etc). Features

Chris LaPointe 173 Dec 1, 2022