A package to allow one to concurrently go through a filesystem with ease

Overview

skywalker GoDoc Build Status Build status codecov Go Report Card

Skywalker is a package to allow one to concurrently go through a filesystem with ease.

Features

  • Concurrency
  • BlackList filtering
  • WhiteList filtering
  • Filter by Directory
  • Filter by Extension
  • Glob Filtering (provided by gobwas/glob)

For matching to work properly across platforms. Please use /. In gobwas/glob the \ is an escape character (so you can escape *, ?, etc...) making it difficult to know if you want to escape a character or go into directory.

Example

package main

import (
    "fmt"
    "sort"
    "strings"
    "sync"

    "github.com/dixonwille/skywalker"
)

type ExampleWorker struct {
    *sync.Mutex
    found []string
}

func (ew *ExampleWorker) Work(path string) {
    //This is where the necessary work should be done.
    //This will get concurrently so make sure it is thread safe if you need info across threads.
    ew.Lock()
    defer ew.Unlock()
    ew.found = append(ew.found, path)
}

func main() {
    //Following two functions are only to create and destroy data for the example
    defer teardownData()
    standupData()

    ew := new(ExampleWorker)
    ew.Mutex = new(sync.Mutex)

    //root is the root directory of the data that was stood up above
    sw := skywalker.New(root, ew)
    sw.DirListType = skywalker.LTBlacklist
    sw.DirList = []string{"sub"}
    sw.ExtListType = skywalker.LTWhitelist
    sw.ExtList = []string{".pdf"}
    err := sw.Walk()
    if err != nil {
        fmt.Println(err)
        return
    }
    sort.Sort(sort.StringSlice(ew.found))
    for _, f := range ew.found {
        fmt.Println(strings.Replace(f, sw.Root, "", 1))
    }
}
Issues
  • It would be nice to reverse walk

    It would be nice to reverse walk

    This feature should only touch directories starting from one location and work backward. This can be a nice to have when looking for config files in you CWD tree. Not sure how helpful it is to have it concurrent, but at least it will be there if it is needed.

    enhancement 
    opened by dixonwille 0
Releases(v1.7.0)
Owner
Will Dixon
Will Dixon
s3fs provides a S3 implementation for Go1.16 filesystem interface.

S3 FileSystem (fs.FS) implementation.Since S3 is a flat structure, s3fs simulates directories by using prefixes and "/" delim. ModTime on directories is always zero value.

Jacek Szwec 130 May 17, 2022
A FileSystem Abstraction System for Go

A FileSystem Abstraction System for Go Overview Afero is a filesystem framework providing a simple, uniform and universal API interacting with any fil

Steve Francia 4.4k May 17, 2022
An implementation of the FileSystem interface for tar files.

TarFS A wrapper around tar.Reader. Implements the FileSystem interface for tar files. Adds an Open method, that enables reading of file according to i

Eyal Posener 50 May 17, 2022
Takes an input http.FileSystem (likely at go generate time) and generates Go code that statically implements it.

vfsgen Package vfsgen takes an http.FileSystem (likely at go generate time) and generates Go code that statically implements the provided http.FileSys

null 943 Apr 29, 2022
memfs: A simple in-memory io/fs.FS filesystem

memfs: A simple in-memory io/fs.FS filesystem memfs is an in-memory implementation of Go's io/fs.FS interface. The goal is to make it easy and quick t

Peter Sanford 58 Apr 28, 2022
A Go io/fs filesystem implementation for reading files in a Github gists.

GistFS GistFS is an io/fs implementation that enables to read files stored in a given Gist. Requirements This module depends on io/fs which is only av

Jean Hadrien Chabran 124 May 1, 2022
A Small Virtual Filesystem in Go

This is a virtual filesystem I'm coding to teach myself Go in a fun way. I'm documenting it with a collection of Medium posts that you can find here.

Alyson 31 Apr 18, 2022
CRFS: Container Registry Filesystem

CRFS: Container Registry Filesystem Discussion: https://github.com/golang/go/issues/30829 Overview CRFS is a read-only FUSE filesystem that lets you m

Google 1.2k May 19, 2022
Encrypted overlay filesystem written in Go

An encrypted overlay filesystem written in Go. Official website: https://nuetzlich.net/gocryptfs (markdown source). gocryptfs is built on top the exce

null 2.3k May 20, 2022
Go filesystem implementations for various URL schemes

hairyhenderson/go-fsimpl This module contains a collection of Go filesystem implementations that can discovered dynamically by URL scheme. All filesys

Dave Henderson 223 May 14, 2022
filesystem for golang

filesystem filesystem for golang installation go get github.com/go-component/filesystem import import "github.com/go-component/filesystem" Usage sup

null 4 Jul 9, 2021
A set of io/fs filesystem abstractions and utilities for Go

A set of io/fs filesystem abstractions and utilities for Go Please ⭐ this project Overview This package provides io/fs interfaces for: Cloud providers

null 8 Mar 24, 2022
Tarserv serves streaming tar files from filesystem snapshots.

tarserv A collection of tools that allow serving large datasets from local filesystem snapshots. It is meant for serving big amounts of data to shell

Aurora 1 Jan 11, 2022
Grep archive search in any files on the filesystem, in archive and even inner archive.

grep-archive Grep archive search for string in any files on the filesystem, in archive and even inner archive. Supported archive format are : Tar Form

Michel Prunet 0 Jan 26, 2022
Fast, dependency-free, small Go package to infer the binary file type based on the magic numbers signature

filetype Small and dependency free Go package to infer file and MIME type checking the magic numbers signature. For SVG file type checking, see go-is-

Tom 1.5k May 20, 2022
Package cae implements PHP-like Compression and Archive Extensions.

Compression and Archive Extensions 中文文档 Package cae implements PHP-like Compression and Archive Extensions. But this package has some modifications de

ᴜɴᴋɴᴡᴏɴ 35 Sep 18, 2021
mateors zip and unzip package

Zip unzip package How to install this package? go get github.com/mateors/mzip How to make zip files? package main import "github.com/mateors/mzip" f

MD MOSTAIN BILLAH 15 Apr 25, 2022
Maybe is a Go package to provide basic functionality for Option type structures

Maybe Maybe is a library that adds an Option data type for some native Go types. What does it offer: The types exported by this library are immutable

Pablo Morelli 8 Mar 28, 2022
gsheet is a CLI tool (and Golang package) for piping csv data to and from Google Sheets

gsheet Table of Contents Introduction Why? Installation Authentication and Authorization What about OAuth authentication? CLI Usage Sheet commands Dri

chris 14 Apr 20, 2022