Simple wrapper around multiple fs.FS instances, recursively merging them together dynamically.

Overview

go-layerfs

Latest Release Build Status Coverage Status Go ReportCard GoDoc

This is a simple wrapper around multiple fs.FS instances, recursively merging them together dynamically.

If you have two directories, of which one is called examples/upper and the other examples/lower, you can layer upper over lower like this:

import (
	"os"
	"path/filepath"
	"github.com/dschmidt/go-layerfs"
)
upper, _ := filepath.Abs("examples/upper")
lower, _ := filepath.Abs("examples/lower")
fsys := layerfs.New(os.DirFS(upper), os.DirFS(lower))

If examples/upper looks like this

.
├── dir1
│   ├── f11.txt (content: foo)
│   └── f12.txt (content: foo)
├── f1.txt (content: foo)
└── f2.txt (content: foo)

and examples/lower looks like this

.
├── dir1
│   ├── f12.txt (content: bar)
│   └── f13.txt (content: bar)
├── f2.txt (content: bar)
└── f3.txt (content: bar)

then your fsys looks like this:

.
├── dir1
│   ├── f11.txt (content: foo)
│   ├── f12.txt (content: foo)
│   └── f13.txt (content: bar)
├── f1.txt (content: foo)
├── f2.txt (content: foo)
└── f3.txt (content: bar)

Example usage

You can run examples/file_server.go like this:

go run examples/file_server.go

2021/11/17 22:59:22 Listening on :8090...

Then requests via httpie should give you these results:

http GET http://localhost:8090/files
HTTP/1.1 200 OK
Content-Length: 123
Content-Type: text/html; charset=utf-8
Date: Wed, 17 Nov 2021 22:03:21 GMT
Last-Modified: Wed, 17 Nov 2021 21:55:53 GMT

<pre>
<a href="dir1/">dir1/</a>
<a href="f1.txt">f1.txt</a>
<a href="f2.txt">f2.txt</a>
<a href="f3.txt">f3.txt</a>
</pre>
http GET http://localhost:8090/files/f1.txt
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 3
Content-Type: text/plain; charset=utf-8
Date: Wed, 17 Nov 2021 22:05:29 GMT
Last-Modified: Wed, 17 Nov 2021 21:56:26 GMT

foo
http GET http://localhost:8090/files/f3.txt
HTTP/1.1 200 OK
Accept-Ranges: bytes
Content-Length: 3
Content-Type: text/plain; charset=utf-8
Date: Wed, 17 Nov 2021 22:05:56 GMT
Last-Modified: Wed, 17 Nov 2021 21:56:30 GMT

bar
You might also like...
This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances

Session Manager Plugin This plugin helps you to use the AWS Command Line Interface (AWS CLI) to start and end sessions to your managed instances. Sess

go-awssh is a developer tool to make your SSH to AWS EC2 instances easy.

Describing Instances/VPCs data, select one or multiple instances, and make connection(s) to selected instances. Caching the response of API calls for 1day using Tmpfs.

Automatically stop GCE instances
Automatically stop GCE instances

auto-stop-gce-instances Architecture How to deploy 1. Deploy Pub/Sub $ gcloud pubsub topics create $PUBSUB_NAME 2. Deploy Cloud Functions $ gcloud fu

Create AWS Auto Scaling groups from running instances or AMIs automatically.

AWS Auto Scaling Groups Builder AWS Auto Scaling group is a great way of managing Amazon EC2 instances. AWS Auto Scaling group watches the correspondi

Drain-my-spot - Service draining the k8s worker node in case of spot instances related event occurrence

drain-my-spot Service draining the k8s worker node in case of spot instances rel

Lxmin - Backup and Restore LXC instances from MinIO

lxmin Backup and restore LXC instances from MinIO Usage NAME: lxmin - backup a

🚢  Go package providing lifecycle management for PostgreSQL Docker instances.
🚢 Go package providing lifecycle management for PostgreSQL Docker instances.

🚢 psqldocker powered by ory/dockertest. Go package providing lifecycle management for PostgreSQL Docker instances. Leverage Docker to run unit and in

A simple tool who pulls data from Online.net API and parse them to a Prometheus format

Dedibox backup monitoring A simple tool who reads API from Online.net and parse them into a Prometheus-compatible format. Conceived to be lightweight,

Hello-go - Messing around with Go concepts.

Go Characteristics Naturally non-blocking Compiled directly to machine code M:N Scheduling Model Is a divergence language Goroutine https://medium.com

Comments
  • error `go-layerfs: could not ReadDir because n >= 0 is not supported`

    error `go-layerfs: could not ReadDir because n >= 0 is not supported`

    I tried to use layerfs with 2 embed.FS, but when i was walking the filesystem I saw the message views: go-layerfs: could not ReadDir because n >= 0 is not supported: .

    When I read the docs for fs.ReadDirFile (https://github.com/golang/go/blob/master/src/io/fs/fs.go#L129), there are cases for n > 0 and n <= 0, so i think the check in DirFile.ReadDir should only fail for n > 0 (and not for n == 0) https://github.com/dschmidt/go-layerfs/blob/a0bce2dc294f4c856d88f7668bfab1f13a34ce5f/files.go#L23-L27

    opened by oeaniz 4
  • Reduce stat calls

    Reduce stat calls

    It is valid for any fs.File to implement fs.ReadDirFile even if it's not a directory. We trade a bit of memory (for storing the path in every file now) for less Stat calls which we can do on demand now.

    c.f. https://cs.opensource.google/go/go/+/refs/tags/go1.17.3:src/io/fs/fs.go;l=108

    opened by dschmidt 0
Releases(v0.1.0)
Owner
Dominik Schmidt
Dominik Schmidt
K8S ConfigMap Merging Controller

ConfigMap Merging Controller (cmmc) cmmc is a k8s operator that allows for the merging of ConfigMap resources with data validation. Why? The impetus f

Cash App 15 Oct 2, 2022
A Grafana backend plugin for automatic synchronization of dashboard between multiple Grafana instances.

Grafana Dashboard Synchronization Backend Plugin A Grafana backend plugin for automatic synchronization of dashboard between multiple Grafana instance

Novatec Consulting GmbH 8 Dec 23, 2022
Cloud-Z gathers information and perform benchmarks on cloud instances in multiple cloud providers.

Cloud-Z Cloud-Z gathers information and perform benchmarks on cloud instances in multiple cloud providers. Cloud type, instance id, and type CPU infor

CloudSnorkel 16 Jun 8, 2022
azqlite is a lightweight wrapper around Azure's SDK to interact with the Azure Storage Queue service in a simpler and more idiomatic way.

azqlite azqlite is a lightweight wrapper around github.com/Azure/azure-storage-queue-go to interact with the Azure Storage Queue service in a simpler

Jose Garcia 1 Mar 12, 2022
Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload.

Fleex allows you to create multiple VPS on cloud providers and use them to distribute your workload. Run tools like masscan, puredns, ffuf, httpx or a

null 176 Dec 31, 2022
Count total LoC in a given folder, recursively.

GoLoc A simple CLI tool to calculate the total LoC of a given directory. Usage goloc Flags Flag Description --use-gitignore Use your .gitignore file a

null 3 Apr 27, 2022
A kubernetes plugin which enables dynamically add or remove GPU resources for a running Pod

GPU Mounter GPU Mounter is a kubernetes plugin which enables add or remove GPU resources for running Pods. This Introduction(In Chinese) is recommende

XinYuan 83 Jan 5, 2023
Dynamically provisioning persistent local storage with Kubernetes

Local Path Provisioner Overview Local Path Provisioner provides a way for the Kubernetes users to utilize the local storage in each node. Based on the

Rancher 1.4k Jan 4, 2023
S3pd - CLI utility that downloads multiple s3 objects at a time, with multiple range-requests issued per object

S3 Parallel Downloader CLI utility that downloads multiple s3 objects at a time,

Colin Bookman 2 May 13, 2022
Kubegres is a Kubernetes operator allowing to create a cluster of PostgreSql instances and manage databases replication, failover and backup.

Kubegres is a Kubernetes operator allowing to deploy a cluster of PostgreSql pods with data replication enabled out-of-the box. It brings simplicity w

Reactive Tech Ltd 1.1k Dec 30, 2022