A package to allow one to concurrently go through a filesystem with ease

Overview

skywalker GoDoc Build Status Build status codecov Go Report Card

Skywalker is a package to allow one to concurrently go through a filesystem with ease.

Features

  • Concurrency
  • BlackList filtering
  • WhiteList filtering
  • Filter by Directory
  • Filter by Extension
  • Glob Filtering (provided by gobwas/glob)

For matching to work properly across platforms. Please use /. In gobwas/glob the \ is an escape character (so you can escape *, ?, etc...) making it difficult to know if you want to escape a character or go into directory.

Example

package main

import (
    "fmt"
    "sort"
    "strings"
    "sync"

    "github.com/dixonwille/skywalker"
)

type ExampleWorker struct {
    *sync.Mutex
    found []string
}

func (ew *ExampleWorker) Work(path string) {
    //This is where the necessary work should be done.
    //This will get concurrently so make sure it is thread safe if you need info across threads.
    ew.Lock()
    defer ew.Unlock()
    ew.found = append(ew.found, path)
}

func main() {
    //Following two functions are only to create and destroy data for the example
    defer teardownData()
    standupData()

    ew := new(ExampleWorker)
    ew.Mutex = new(sync.Mutex)

    //root is the root directory of the data that was stood up above
    sw := skywalker.New(root, ew)
    sw.DirListType = skywalker.LTBlacklist
    sw.DirList = []string{"sub"}
    sw.ExtListType = skywalker.LTWhitelist
    sw.ExtList = []string{".pdf"}
    err := sw.Walk()
    if err != nil {
        fmt.Println(err)
        return
    }
    sort.Sort(sort.StringSlice(ew.found))
    for _, f := range ew.found {
        fmt.Println(strings.Replace(f, sw.Root, "", 1))
    }
}
You might also like...
filesystem for golang

filesystem filesystem for golang installation go get github.com/go-component/filesystem import import "github.com/go-component/filesystem" Usage sup

A set of io/fs filesystem abstractions and utilities for Go
A set of io/fs filesystem abstractions and utilities for Go

A set of io/fs filesystem abstractions and utilities for Go Please ⭐ this project Overview This package provides io/fs interfaces for: Cloud providers

Tarserv serves streaming tar files from filesystem snapshots.

tarserv A collection of tools that allow serving large datasets from local filesystem snapshots. It is meant for serving big amounts of data to shell

Grep archive search in any files on the filesystem, in archive and even inner archive.

grep-archive Grep archive search for string in any files on the filesystem, in archive and even inner archive. Supported archive format are : Tar Form

Warp across your filesystem in ~5 ms
Warp across your filesystem in ~5 ms

WarpDrive: the Go version. What does this do? Instead of having a huge cd routine to get where you want, with WarpDrive you use short keywords to warp

Fast, dependency-free, small Go package to infer the binary file type based on the magic numbers signature

filetype Small and dependency free Go package to infer file and MIME type checking the magic numbers signature. For SVG file type checking, see go-is-

Package cae implements PHP-like Compression and Archive Extensions.

Compression and Archive Extensions 中文文档 Package cae implements PHP-like Compression and Archive Extensions. But this package has some modifications de

mateors zip and unzip package

Zip unzip package How to install this package? go get github.com/mateors/mzip How to make zip files? package main import "github.com/mateors/mzip" f

Maybe is a Go package to provide basic functionality for Option type structures

Maybe Maybe is a library that adds an Option data type for some native Go types. What does it offer: The types exported by this library are immutable

Comments
  • It would be nice to reverse walk

    It would be nice to reverse walk

    This feature should only touch directories starting from one location and work backward. This can be a nice to have when looking for config files in you CWD tree. Not sure how helpful it is to have it concurrent, but at least it will be there if it is needed.

    enhancement 
    opened by dixonwille 0
Releases(v1.7.0)
Owner
Will Dixon
Will Dixon
s3fs provides a S3 implementation for Go1.16 filesystem interface.

S3 FileSystem (fs.FS) implementation.Since S3 is a flat structure, s3fs simulates directories by using prefixes and "/" delim. ModTime on directories is always zero value.

Jacek Szwec 137 Sep 13, 2022
A FileSystem Abstraction System for Go

A FileSystem Abstraction System for Go Overview Afero is a filesystem framework providing a simple, uniform and universal API interacting with any fil

Steve Francia 4.8k Sep 22, 2022
An implementation of the FileSystem interface for tar files.

TarFS A wrapper around tar.Reader. Implements the FileSystem interface for tar files. Adds an Open method, that enables reading of file according to i

Eyal Posener 50 May 17, 2022
Takes an input http.FileSystem (likely at go generate time) and generates Go code that statically implements it.

vfsgen Package vfsgen takes an http.FileSystem (likely at go generate time) and generates Go code that statically implements the provided http.FileSys

null 947 Aug 6, 2022
memfs: A simple in-memory io/fs.FS filesystem

memfs: A simple in-memory io/fs.FS filesystem memfs is an in-memory implementation of Go's io/fs.FS interface. The goal is to make it easy and quick t

Peter Sanford 70 Sep 17, 2022
A Go io/fs filesystem implementation for reading files in a Github gists.

GistFS GistFS is an io/fs implementation that enables to read files stored in a given Gist. Requirements This module depends on io/fs which is only av

Jean Hadrien Chabran 124 Jul 3, 2022
A Small Virtual Filesystem in Go

This is a virtual filesystem I'm coding to teach myself Go in a fun way. I'm documenting it with a collection of Medium posts that you can find here.

Alyson 31 Sep 10, 2022
CRFS: Container Registry Filesystem

CRFS: Container Registry Filesystem Discussion: https://github.com/golang/go/issues/30829 Overview CRFS is a read-only FUSE filesystem that lets you m

Google 1.3k Sep 25, 2022
Encrypted overlay filesystem written in Go

An encrypted overlay filesystem written in Go. Official website: https://nuetzlich.net/gocryptfs (markdown source). gocryptfs is built on top the exce

null 2.5k Sep 24, 2022
Go filesystem implementations for various URL schemes

hairyhenderson/go-fsimpl This module contains a collection of Go filesystem implementations that can discovered dynamically by URL scheme. All filesys

Dave Henderson 234 Sep 24, 2022