A package to allow one to concurrently go through a filesystem with ease

Overview

skywalker GoDoc Build Status Build status codecov Go Report Card

Skywalker is a package to allow one to concurrently go through a filesystem with ease.

Features

  • Concurrency
  • BlackList filtering
  • WhiteList filtering
  • Filter by Directory
  • Filter by Extension
  • Glob Filtering (provided by gobwas/glob)

For matching to work properly across platforms. Please use /. In gobwas/glob the \ is an escape character (so you can escape *, ?, etc...) making it difficult to know if you want to escape a character or go into directory.

Example

package main

import (
    "fmt"
    "sort"
    "strings"
    "sync"

    "github.com/dixonwille/skywalker"
)

type ExampleWorker struct {
    *sync.Mutex
    found []string
}

func (ew *ExampleWorker) Work(path string) {
    //This is where the necessary work should be done.
    //This will get concurrently so make sure it is thread safe if you need info across threads.
    ew.Lock()
    defer ew.Unlock()
    ew.found = append(ew.found, path)
}

func main() {
    //Following two functions are only to create and destroy data for the example
    defer teardownData()
    standupData()

    ew := new(ExampleWorker)
    ew.Mutex = new(sync.Mutex)

    //root is the root directory of the data that was stood up above
    sw := skywalker.New(root, ew)
    sw.DirListType = skywalker.LTBlacklist
    sw.DirList = []string{"sub"}
    sw.ExtListType = skywalker.LTWhitelist
    sw.ExtList = []string{".pdf"}
    err := sw.Walk()
    if err != nil {
        fmt.Println(err)
        return
    }
    sort.Sort(sort.StringSlice(ew.found))
    for _, f := range ew.found {
        fmt.Println(strings.Replace(f, sw.Root, "", 1))
    }
}
Issues
  • Files in subfolders whit list

    Files in subfolders whit list

    opened by dixonwille 1
  • Error if Root DNE

    Error if Root DNE

    Currently, the Walking function panics if the path does not exist.

    bug 
    opened by dixonwille 1
  • It would be nice to reverse walk

    It would be nice to reverse walk

    This feature should only touch directories starting from one location and work backward. This can be a nice to have when looking for config files in you CWD tree. Not sure how helpful it is to have it concurrent, but at least it will be there if it is needed.

    enhancement 
    opened by dixonwille 0
Releases(v1.7.0)
Owner
Will Dixon
Will Dixon
s3fs provides a S3 implementation for Go1.16 filesystem interface.

S3 FileSystem (fs.FS) implementation.Since S3 is a flat structure, s3fs simulates directories by using prefixes and "/" delim. ModTime on directories is always zero value.

Jacek Szwec 121 Nov 28, 2021
A FileSystem Abstraction System for Go

A FileSystem Abstraction System for Go Overview Afero is a filesystem framework providing a simple, uniform and universal API interacting with any fil

Steve Francia 4.1k Dec 5, 2021
An implementation of the FileSystem interface for tar files.

TarFS A wrapper around tar.Reader. Implements the FileSystem interface for tar files. Adds an Open method, that enables reading of file according to i

Eyal Posener 48 Nov 18, 2021
Takes an input http.FileSystem (likely at go generate time) and generates Go code that statically implements it.

vfsgen Package vfsgen takes an http.FileSystem (likely at go generate time) and generates Go code that statically implements the provided http.FileSys

null 935 Dec 5, 2021
memfs: A simple in-memory io/fs.FS filesystem

memfs: A simple in-memory io/fs.FS filesystem memfs is an in-memory implementation of Go's io/fs.FS interface. The goal is to make it easy and quick t

Peter Sanford 49 Nov 14, 2021
A Go io/fs filesystem implementation for reading files in a Github gists.

GistFS GistFS is an io/fs implementation that enables to read files stored in a given Gist. Requirements This module depends on io/fs which is only av

Jean Hadrien Chabran 122 Dec 6, 2021
A Small Virtual Filesystem in Go

This is a virtual filesystem I'm coding to teach myself Go in a fun way. I'm documenting it with a collection of Medium posts that you can find here.

Alyson 25 Nov 26, 2021
CRFS: Container Registry Filesystem

CRFS: Container Registry Filesystem Discussion: https://github.com/golang/go/issues/30829 Overview CRFS is a read-only FUSE filesystem that lets you m

Google 1.2k Dec 2, 2021
Encrypted overlay filesystem written in Go

An encrypted overlay filesystem written in Go. Official website: https://nuetzlich.net/gocryptfs (markdown source). gocryptfs is built on top the exce

null 2k Nov 26, 2021
Go filesystem implementations for various URL schemes

hairyhenderson/go-fsimpl This module contains a collection of Go filesystem implementations that can discovered dynamically by URL scheme. All filesys

Dave Henderson 197 Nov 29, 2021
filesystem for golang

filesystem filesystem for golang installation go get github.com/go-component/filesystem import import "github.com/go-component/filesystem" Usage sup

null 4 Jul 9, 2021
A set of io/fs filesystem abstractions and utilities for Go

A set of io/fs filesystem abstractions and utilities for Go Please ⭐ this project Overview This package provides io/fs interfaces for: Cloud providers

null 7 Oct 7, 2021
Tarserv serves streaming tar files from filesystem snapshots.

tarserv A collection of tools that allow serving large datasets from local filesystem snapshots. It is meant for serving big amounts of data to shell

Aurora 0 Nov 29, 2021
Fast, dependency-free, small Go package to infer the binary file type based on the magic numbers signature

filetype Small and dependency free Go package to infer file and MIME type checking the magic numbers signature. For SVG file type checking, see go-is-

Tom 1.4k Nov 27, 2021
Package cae implements PHP-like Compression and Archive Extensions.

Compression and Archive Extensions 中文文档 Package cae implements PHP-like Compression and Archive Extensions. But this package has some modifications de

ᴜɴᴋɴᴡᴏɴ 35 Sep 18, 2021
mateors zip and unzip package

Zip unzip package How to install this package? go get github.com/mateors/mzip How to make zip files? package main import "github.com/mateors/mzip" f

MD MOSTAIN BILLAH 14 Jul 3, 2021
Maybe is a Go package to provide basic functionality for Option type structures

Maybe Maybe is a library that adds an Option data type for some native Go types. What does it offer: The types exported by this library are immutable

Pablo Morelli 6 Oct 20, 2021
gsheet is a CLI tool (and Golang package) for piping csv data to and from Google Sheets

gsheet Table of Contents Introduction Why? Installation Authentication and Authorization What about OAuth authentication? CLI Usage Sheet commands Dri

chris 13 Aug 5, 2021
noioutil finds files using the "io/ioutil" package.

noioutil noioutil finds files using the "io/ioutil" package.

Leo Takaoka 2 Sep 11, 2021