Go filesystem implementations for various URL schemes

Overview

hairyhenderson/go-fsimpl

GoDoc Build

This module contains a collection of Go filesystem implementations that can discovered dynamically by URL scheme. All filesystems are read-only.

These filesystems implement the fs.FS interface introduced in Go 1.16

Most implementations implement the fs.ReadDirFS interface, though the https filesystem does not.

Some extensions are available to help add specific functionality to certain filesystems:

  • WithContextFS - injects a context into a filesystem, for propagating cancellation in filesystems that support it.
  • WithHeaderFS - sets the http.Header for all HTTP requests used by the filesystem. This can be useful for authentication, or for requesting specific content types.
  • WithHTTPClientFS - sets the *http.Client for all HTTP requests to be made with.

This module also provides ContentType, an extension to the fs.FileInfo type to help identify an appropriate MIME content type for a given file. For filesystems that support it, the HTTP Content-Type header is used for this. Otherwise, the type is guessed from the file extension.

History & Project Status

This module is in development, and the API is still subject to change. The filesystems that are supported should operate correctly.

Most of these filesystems are based on code from gomplate, which supports all of these as datasources. This module is intended to eventually be used within gomplate.

Supported Filesystems

Here's the list of planned filesystem support, along with status:

Scheme(s) Description Supported?
aws+sm AWS Secrets Manager
aws+smp AWS Systems Manager Parameter Store
azblob Azure Blob Storage
consul, consul+http, consul+https HashiCorp Consul
file local filesystem
git, git+file, git+http, git+https, git+ssh local/remote git repository
gs Google Cloud Storage
http, https HTTP server
s3 Amazon S3
vault, vault+http, vault+https HashiCorp Vault

See url_schemes.md for more details on each scheme.

Installation

You need Go 1.16 or above to use this module. Use go get to install the latest version of go-fsimpl:

$ go get -u github.com/hairyhenderson/go-fsimpl

Usage

If you know that you want an HTTP filesystem, for example:

import (
	"net/url"

	"github.com/hairyhenderson/go-fsimpl/httpfs"
)

func main() {
	base, _ := url.Parse("https://example.com")
	fsys, _ := httpfs.New(base)

	f, _ := fsys.Open("hello.txt")
	defer f.Close()

	// now do what you like with the file...
}

If you're not sure what filesystem type you need (for example, if you're dealing with a user-provided URL), you can use a filesystem mux:

import (
	"github.com/hairyhenderson/go-fsimpl"
	"github.com/hairyhenderson/go-fsimpl/blobfs"
	"github.com/hairyhenderson/go-fsimpl/filefs"
	"github.com/hairyhenderson/go-fsimpl/gitfs"
	"github.com/hairyhenderson/go-fsimpl/httpfs"
)

func main() {
	mux := fsimpl.NewMux()
	mux.Add(filefs.FS)
	mux.Add(httpfs.FS)
	mux.Add(blobfs.FS)
	mux.Add(gitfs.FS)

	// for example, a URL that points to a subdirectory at a specific tag in a
	// given git repo, hosted on GitHub and authenticated with SSH...
	fsys, err := mux.Lookup("git+ssh://[email protected]/foo/bar.git//baz#refs/tags/v1.0.0")
	if err != nil {
		log.Fatal(err)
	}

	f, _ := fsys.Open("hello.txt")
	defer f.Close()

	// now do what you like with the file...
}

License

The MIT License

Copyright (c) 2021 Dave Henderson

Issues
  • fix: Dependency Requiring CGO

    fix: Dependency Requiring CGO

    I switched from Azure/azure-storage-blob-go/azblob to Azure/azure-sdk-for-go/sdk/storage/azblob for the azblob as azure-storage-blob-go has been deprecated.

    Used the unreleased version gocloud.dev package:

    dependencies 
    opened by bbernays 8
  • Add Vault FS

    Add Vault FS

    Fixes #10

    Initial support. Still to come as follow-ups are #23 (ec2 auth method) and #24 (kv v2 support).

    Signed-off-by: Dave Henderson [email protected]

    documentation filesystem dependencies 
    opened by hairyhenderson 8
  • Reduce number of transitive dependencies loaded

    Reduce number of transitive dependencies loaded

    A downside of this module is the number of transitive dependencies pulled in can get quite high (due to supporting so many different filesystems). This can cause significant bloat.

    I've gotten a few pieces of early feedback from people who might find this useful, but want to avoid all of the transitive dependencies this pulls in. This is absolutely fair - I want to strike a balance between being usable and reasonably lean.

    One way to reduce unwanted bloat could involve splitting each FS implementation into its own sub-package, and then in Go 1.17 this would take advantage of the new lazy module loading work.

    Splitting packages would only be a help to individually-targeted filesystems (like, say, if you knew you wanted a GitFS), because LookupFS would still need to be able to choose the correct implementation for the given scheme.

    One way around this could be to alter the API such that LookupFS only chooses from an explicitly-registered set of implementations, like:

    mux := fsimpl.NewMux()
    mux.Add(blobfs.New)
    mux.Add(gitfs.New)
    
    fs, err := mux.Lookup(url)
    

    (adapted from a conversation in Gophers Slack)

    This feels to me like something that should be solved well before v1, and probably even before the first v0 release, since it completely breaks API.

    help wanted 
    opened by hairyhenderson 6
  • Bump github.com/aws/aws-sdk-go-v2 from 1.16.3 to 1.16.4

    Bump github.com/aws/aws-sdk-go-v2 from 1.16.3 to 1.16.4

    Bumps github.com/aws/aws-sdk-go-v2 from 1.16.3 to 1.16.4.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 1
  • Bump github.com/aws/aws-sdk-go-v2/service/secretsmanager from 1.15.6 to 1.15.7

    Bump github.com/aws/aws-sdk-go-v2/service/secretsmanager from 1.15.6 to 1.15.7

    Bumps github.com/aws/aws-sdk-go-v2/service/secretsmanager from 1.15.6 to 1.15.7.

    Changelog

    Sourced from github.com/aws/aws-sdk-go-v2/service/secretsmanager's changelog.

    Release (2022-05-13)

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/grafana: v1.9.0
      • Feature: This release adds APIs for creating and deleting API keys in an Amazon Managed Grafana workspace.

    Release (2022-05-12)

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/ec2: v1.43.0
      • Feature: This release introduces a target type Gateway Load Balancer Endpoint for mirrored traffic. Customers can now specify GatewayLoadBalancerEndpoint option during the creation of a traffic mirror target.
    • github.com/aws/aws-sdk-go-v2/service/finspacedata: v1.10.5
      • Documentation: We've now deprecated CreateSnapshot permission for creating a data view, instead use CreateDataView permission.
    • github.com/aws/aws-sdk-go-v2/service/iot: v1.25.1
      • Documentation: Documentation update for China region ListMetricValues for IoT
    • github.com/aws/aws-sdk-go-v2/service/ivschat: v1.0.2
      • Documentation: Documentation-only updates for IVS Chat API Reference.
    • github.com/aws/aws-sdk-go-v2/service/kendra: v1.27.0
    • github.com/aws/aws-sdk-go-v2/service/lambda: v1.23.0
      • Feature: Lambda releases NodeJs 16 managed runtime to be available in all commercial regions.
    • github.com/aws/aws-sdk-go-v2/service/lightsail: v1.21.0
      • Feature: This release adds support to include inactive database bundles in the response of the GetRelationalDatabaseBundles request.
    • github.com/aws/aws-sdk-go-v2/service/outposts: v1.19.1
      • Documentation: Documentation updates for AWS Outposts.
    • github.com/aws/aws-sdk-go-v2/service/ssmincidents: v1.14.0
      • Feature: Adding support for dynamic SSM Runbook parameter values. Updating validation pattern for engagements. Adding ConflictException to UpdateReplicationSet API contract.
    • github.com/aws/aws-sdk-go-v2/service/transfer: v1.18.6
      • Documentation: AWS Transfer Family now accepts ECDSA keys for server host keys

    Release (2022-05-11)

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/ec2: v1.42.0
      • Feature: This release updates AWS PrivateLink APIs to support IPv6 for PrivateLink Services and Endpoints of type 'Interface'.
    • github.com/aws/aws-sdk-go-v2/service/secretsmanager: v1.15.7
      • Documentation: Doc only update for Secrets Manager that fixes several customer-reported issues.

    Release (2022-05-10)

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/computeoptimizer: v1.17.5
      • Documentation: Documentation updates for Compute Optimizer
    • github.com/aws/aws-sdk-go-v2/service/ec2: v1.41.0
      • Feature: Added support for using NitroTPM and UEFI Secure Boot on EC2 instances.
    • github.com/aws/aws-sdk-go-v2/service/eks: v1.21.0
      • Feature: Adds BOTTLEROCKET_ARM_64_NVIDIA and BOTTLEROCKET_x86_64_NVIDIA AMI types to EKS managed nodegroups
    • github.com/aws/aws-sdk-go-v2/service/emr: v1.18.0
      • Feature: This release updates the Amazon EMR ModifyInstanceGroups API to support "MERGE" type cluster reconfiguration. Also, added the ability to specify a particular Amazon Linux release for all nodes in a cluster launch request.
    • github.com/aws/aws-sdk-go-v2/service/migrationhubrefactorspaces: v1.5.5

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 1
  • Bump github.com/aws/aws-sdk-go-v2 from 1.16.2 to 1.16.3

    Bump github.com/aws/aws-sdk-go-v2 from 1.16.2 to 1.16.3

    Bumps github.com/aws/aws-sdk-go-v2 from 1.16.2 to 1.16.3.

    Changelog

    Sourced from github.com/aws/aws-sdk-go-v2's changelog.

    Release (2022-04-29)

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/codegurureviewer: v1.16.0
      • Feature: Amazon CodeGuru Reviewer now supports suppressing recommendations from being generated on specific files and directories.
    • github.com/aws/aws-sdk-go-v2/service/mediaconvert: v1.23.0
      • Feature: AWS Elemental MediaConvert SDK nows supports creation of Dolby Vision profile 8.1, the ability to generate black frames of video, and introduces audio-only DASH and CMAF support.
    • github.com/aws/aws-sdk-go-v2/service/rds: v1.21.0
      • Feature: Feature - Adds support for Internet Protocol Version 6 (IPv6) on RDS database instances.
    • github.com/aws/aws-sdk-go-v2/service/ssm: v1.26.0
      • Feature: Update the StartChangeRequestExecution, adding TargetMaps to the Runbook parameter
    • github.com/aws/aws-sdk-go-v2/service/wafv2: v1.20.0
      • Feature: You can now inspect all request headers and all cookies. You can now specify how to handle oversize body contents in your rules that inspect the body.

    Release (2022-04-28)

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/auditmanager: v1.18.5
      • Documentation: This release adds documentation updates for Audit Manager. We provided examples of how to use the Custom_ prefix for the keywordValue attribute. We also provided more details about the DeleteAssessmentReport operation.
    • github.com/aws/aws-sdk-go-v2/service/braket: v1.16.0
      • Feature: This release enables Braket Hybrid Jobs with Embedded Simulators to have multiple instances.
    • github.com/aws/aws-sdk-go-v2/service/connect: v1.24.0
      • Feature: This release introduces an API for changing the current agent status of a user in Connect.
    • github.com/aws/aws-sdk-go-v2/service/ec2: v1.37.0
      • Feature: This release adds support to query the public key and creation date of EC2 Key Pairs. Additionally, the format (pem or ppk) of a key pair can be specified when creating a new key pair.
    • github.com/aws/aws-sdk-go-v2/service/guardduty: v1.13.5
      • Documentation: Documentation update for API description.
    • github.com/aws/aws-sdk-go-v2/service/networkfirewall: v1.17.0
      • Feature: AWS Network Firewall adds support for stateful threat signature AWS managed rule groups.

    Release (2022-04-27)

    General Highlights

    • Dependency Update: Updated to the latest SDK module versions

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/amplify: v1.11.5
      • Documentation: Documentation only update to support the Amplify GitHub App feature launch
    • github.com/aws/aws-sdk-go-v2/service/chimesdkmediapipelines: v1.0.0
      • Release: New AWS service client module
      • Feature: For Amazon Chime SDK meetings, the Amazon Chime Media Pipelines SDK allows builders to capture audio, video, and content share streams. You can also capture meeting events, live transcripts, and data messages. The pipelines save the artifacts to an Amazon S3 bucket that you designate.
    • github.com/aws/aws-sdk-go-v2/service/cloudtrail: v1.16.0
      • Feature: Increases the retention period maximum to 2557 days. Deprecates unused fields of the ListEventDataStores API response. Updates documentation.
    • github.com/aws/aws-sdk-go-v2/service/internal/checksum: v1.1.5
      • Bug Fix: Fixes a bug that could cause the SigV4 payload hash to be incorrectly encoded, leading to signing errors.
    • github.com/aws/aws-sdk-go-v2/service/iotwireless: v1.19.0
      • Feature: Add list support for event configurations, allow to get and update event configurations by resource type, support LoRaWAN events; Make NetworkAnalyzerConfiguration as a resource, add List, Create, Delete API support; Add FCntStart attribute support for ABP WirelessDevice.
    • github.com/aws/aws-sdk-go-v2/service/lookoutequipment: v1.13.0
      • Feature: This release adds the following new features: 1) Introduces an option for automatic schema creation 2) Now allows for Ingestion of data containing most common errors and allows automatic data cleaning 3) Introduces new API ListSensorStatistics that gives further information about the ingested data
    • github.com/aws/aws-sdk-go-v2/service/rekognition: v1.18.0

    ... (truncated)

    Commits
    • 6911981 Release 2022-04-25
    • 1bd13c6 Regenerated Clients
    • 82cbcb0 Update API model
    • b0a3c24 Adds Policy and Duration parameters to stscreds.WebIdentityRoleOptions ...
    • da333d3 internal/sync/singleflight: Update version of singleflight (#1679)
    • 7065f83 Add make command for listing the SDK's complete set of Go dependencies (#1677)
    • d27c530 Release 2022-04-22
    • 67f8605 Regenerated Clients
    • 4e2def0 Update API model
    • aa725de Update smithy-go to smithy 1.21.0 (#1615)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 1
  • Build Fails on Darwin (without CGO enabled)

    Build Fails on Darwin (without CGO enabled)

    A dependency of azure-pipeline-go requires CGO to be enabled in order to build for Darwin

    Issue in Dependency: https://github.com/Azure/azure-pipeline-go/issues/31

    Root issue: https://github.com/mattn/go-ieproxy/issues/34

    opened by bbernays 1
  • Bump github.com/aws/aws-sdk-go-v2 from 1.16.1 to 1.16.2

    Bump github.com/aws/aws-sdk-go-v2 from 1.16.1 to 1.16.2

    Bumps github.com/aws/aws-sdk-go-v2 from 1.16.1 to 1.16.2.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump github.com/aws/aws-sdk-go-v2 from 1.15.0 to 1.16.1

    Bump github.com/aws/aws-sdk-go-v2 from 1.15.0 to 1.16.1

    Bumps github.com/aws/aws-sdk-go-v2 from 1.15.0 to 1.16.1.

    Changelog

    Sourced from github.com/aws/aws-sdk-go-v2's changelog.

    Release (2022-03-25)

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/batch: v1.17.0
      • Feature: Bug Fix: Fixed a bug where shapes were marked as unboxed and were not serialized and sent over the wire, causing an API error from the service.
    • github.com/aws/aws-sdk-go-v2/service/ec2: v1.33.0
      • Feature: This is release adds support for Amazon VPC Reachability Analyzer to analyze path through a Transit Gateway.
    • github.com/aws/aws-sdk-go-v2/service/ssm: v1.24.0
      • Feature: This Patch Manager release supports creating, updating, and deleting Patch Baselines for Rocky Linux OS.

    Release (2022-03-24)

    General Highlights

    • Dependency Update: Updated to the latest SDK module versions

    Module Highlights

    • github.com/aws/aws-sdk-go-v2/service/configservice: v1.20.0
      • Feature: Added new APIs GetCustomRulePolicy and GetOrganizationCustomRulePolicy, and updated existing APIs PutConfigRule, DescribeConfigRule, DescribeConfigRuleEvaluationStatus, PutOrganizationConfigRule, DescribeConfigRule to support a new feature for building AWS Config rules with AWS CloudFormation Guard
    • github.com/aws/aws-sdk-go-v2/service/lambda: v1.21.0
      • Feature: Adds support for increased ephemeral storage (/tmp) up to 10GB for Lambda functions. Customers can now provision up to 10 GB of ephemeral storage per function instance, a 20x increase over the previous limit of 512 MB.
    • github.com/aws/aws-sdk-go-v2/service/transcribe: v1.19.0
      • Feature: This release adds an additional parameter for subtitling with Amazon Transcribe batch jobs: outputStartIndex.

    Release (2022-03-23)

    General Highlights

    • Dependency Update: Updated to the latest SDK module versions

    Module Highlights

    • github.com/aws/aws-sdk-go-v2: v1.16.0
      • Feature: Update CredentialsCache to make use of two new optional CredentialsProvider interfaces to give the cache, per provider, behavior how the cache handles credentials that fail to refresh, and adjusting expires time. See aws.CredentialsCache for more details.
      • Feature: Update ec2rolecreds package's Provider to implememnt support for CredentialsCache new optional caching strategy interfaces, HandleFailRefreshCredentialsCacheStrategy and AdjustExpiresByCredentialsCacheStrategy.
    • github.com/aws/aws-sdk-go-v2/credentials: v1.11.0
      • Feature: Update ec2rolecreds package's Provider to implememnt support for CredentialsCache new optional caching strategy interfaces, HandleFailRefreshCredentialsCacheStrategy and AdjustExpiresByCredentialsCacheStrategy.
    • github.com/aws/aws-sdk-go-v2/service/auditmanager: v1.18.0
      • Feature: This release updates 1 API parameter, the SnsArn attribute. The character length and regex pattern for the SnsArn attribute have been updated, which enables you to deselect an SNS topic when using the UpdateSettings operation.
    • github.com/aws/aws-sdk-go-v2/service/ebs: v1.15.0
      • Feature: Increased the maximum supported value for the Timeout parameter of the StartSnapshot API from 60 minutes to 4320 minutes. Changed the HTTP error code for ConflictException from 503 to 409.
    • github.com/aws/aws-sdk-go-v2/service/elasticache: v1.20.2
      • Documentation: Doc only update for ElastiCache
    • github.com/aws/aws-sdk-go-v2/service/gamesparks: v1.0.0
      • Release: New AWS service client module
      • Feature: Released the preview of Amazon GameSparks, a fully managed AWS service that provides a multi-service backend for game developers.
    • github.com/aws/aws-sdk-go-v2/service/redshift: v1.22.0
      • Feature: This release adds a new [--encrypted | --no-encrypted] field in restore-from-cluster-snapshot API. Customers can now restore an unencrypted snapshot to a cluster encrypted with AWS Managed Key or their own KMS key.
    • github.com/aws/aws-sdk-go-v2/service/ssm: v1.23.0
      • Feature: Update AddTagsToResource, ListTagsForResource, and RemoveTagsFromResource APIs to reflect the support for tagging Automation resources. Includes other minor documentation updates.
    • github.com/aws/aws-sdk-go-v2/service/transfer: v1.18.1
      • Documentation: Documentation updates for AWS Transfer Family to describe how to remove an associated workflow from a server.

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump github.com/aws/aws-sdk-go-v2/service/secretsmanager from 1.15.1 to 1.15.3

    Bump github.com/aws/aws-sdk-go-v2/service/secretsmanager from 1.15.1 to 1.15.3

    Bumps github.com/aws/aws-sdk-go-v2/service/secretsmanager from 1.15.1 to 1.15.3.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump github.com/fsouza/fake-gcs-server from 1.37.0 to 1.37.1

    Bump github.com/fsouza/fake-gcs-server from 1.37.0 to 1.37.1

    Bumps github.com/fsouza/fake-gcs-server from 1.37.0 to 1.37.1.

    Release notes

    Sourced from github.com/fsouza/fake-gcs-server's releases.

    v1.37.1

    Changelog

    • 0d4c0d9 Dockerfile: bump the base image
    • a08c735 Go 1.18
    • ca4486b build(deps): bump docker/build-push-action from 2.9.0 to 2.10.0 (#715)
    • 1d76827 build(deps): bump google-cloud-storage in /examples/python (#714)
    • 4c13abc build(deps): bump google.golang.org/api from 0.71.0 to 0.73.0 (#717)
    • 67452da build(deps): bump google.golang.org/api in /examples/go (#716)
    Commits
    • 0d4c0d9 Dockerfile: bump the base image
    • a08c735 Go 1.18
    • 4c13abc build(deps): bump google.golang.org/api from 0.71.0 to 0.73.0 (#717)
    • 67452da build(deps): bump google.golang.org/api in /examples/go (#716)
    • ca4486b build(deps): bump docker/build-push-action from 2.9.0 to 2.10.0 (#715)
    • 1d76827 build(deps): bump google-cloud-storage in /examples/python (#714)
    • See full diff in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 1
  • Bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 0.3.0 to 0.4.0

    Bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 0.3.0 to 0.4.0

    Bumps github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 0.3.0 to 0.4.0.

    Release notes

    Sourced from github.com/Azure/azure-sdk-for-go/sdk/storage/azblob's releases.

    sdk/keyvault/internal/v0.4.0

    0.4.0 (2022-04-22)

    Breaking Changes

    • Updated ExpiringResource and its dependent types to use generics.

    Other Changes

    • Remove reference to TokenRequestOptions.TenantID as it's been removed and wasn't working anyways.

    sdk/storage/azblob/v0.4.0

    0.4.0 (2022-04-19)

    Breaking Changes

    • Fixed Issue #17150 : Renaming/refactoring high level methods.
    • Fixed Issue #16972 : Constructors should return clients by reference.
    • Renaming the options bags to match the naming convention same as that of response. The behaviour of options bags remains the same.

    Bugs Fixed

    • Fixed Issue #17515 : SetTags options bag missing leaseID.
    • Fixed Issue #17423 : Drop "Type" suffix from GeoReplicationStatusType.
    • Fixed Issue #17335 : Nil pointer exception when passing nil options bag in ListBlobsFlat API call.
    • Fixed Issue #17188 : BlobURLParts not supporting VersionID
    • Fixed Issue #17152 , Issue #17131 , Issue #17061 : UploadStreamToBlockBlob / UploadStreamToBlockBlob methods ignoring the options bag.
    • Fixed Issue #16920 : Fixing error handling example.
    • Fixed Issue #16786 : Refactoring of autorest code generation definition and adding necessary transformations.
    • Fixed Issue #16679 : Response parsing issue in List blobs API.
    Changelog

    Sourced from github.com/Azure/azure-sdk-for-go/sdk/storage/azblob's changelog.

    v60.3.0

    New Packages

    • github.com/Azure/azure-sdk-for-go/services/frontdoor/mgmt/2020-11-01/frontdoor
    • github.com/Azure/azure-sdk-for-go/services/preview/appplatform/mgmt/2022-01-01-preview/appplatform
    • github.com/Azure/azure-sdk-for-go/services/preview/securityinsight/mgmt/2021-09-01-preview/securityinsight

    v60.2.0

    New Packages

    • github.com/Azure/azure-sdk-for-go/services/edgeorder/mgmt/2021-12-01/edgeorder
    • github.com/Azure/azure-sdk-for-go/services/preview/automanage/mgmt/2021-04-30-preview/automanage
    • github.com/Azure/azure-sdk-for-go/services/preview/network/mgmt/2021-03-01-preview/network
    • github.com/Azure/azure-sdk-for-go/services/recoveryservices/mgmt/2021-08-01/recoveryservices

    Updated Packages

    Package Path Changelog
    github.com/Azure/azure-sdk-for-go/services/eventgrid/mgmt/2021-12-01/eventgrid details

    Breaking Changes

    • Rename package github.com/Azure/azure-sdk-for-go/services/preview/windowsesu/2019-09-16-preview/windowsesu/ to github.com/Azure/azure-sdk-for-go/services/preview/windowsesu/mgmt/2019-09-16-preview/windowsesu/

    v60.1.0

    New Packages

    • github.com/Azure/azure-sdk-for-go/services/elastic/mgmt/2020-07-01/elastic
    • github.com/Azure/azure-sdk-for-go/services/virtualmachineimagebuilder/mgmt/2021-10-01/virtualmachineimagebuilder

    v60.0.0

    New Packages

    • github.com/Azure/azure-sdk-for-go/services/kubernetesconfiguration/mgmt/2021-09-01/kubernetesconfiguration
    • github.com/Azure/azure-sdk-for-go/services/netapp/mgmt/2021-08-01/netapp
    • github.com/Azure/azure-sdk-for-go/services/webpubsub/mgmt/2021-10-01/webpubsub

    Breaking Changes

    Package Path Changelog
    github.com/Azure/azure-sdk-for-go/services/datafactory/mgmt/2018-06-01/datafactory details

    v59.4.0

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 2
  • Bump actions/setup-go from 1 to 3

    Bump actions/setup-go from 1 to 3

    Bumps actions/setup-go from 1 to 3.

    Release notes

    Sourced from actions/setup-go's releases.

    v3.0.0

    What's Changed

    Breaking Changes

    With the update to Node 16, all scripts will now be run with Node 16 rather than Node 12.

    This new major release removes the stable input, so there is no need to specify additional input to use pre-release versions. This release also corrects the pre-release versions syntax to satisfy the SemVer notation (1.18.0-beta1 -> 1.18.0-beta.1, 1.18.0-rc1 -> 1.18.0-rc.1).

    steps:
      - uses: actions/[email protected]
      - uses: actions/[email protected]
        with:
          go-version: '1.18.0-rc.1' 
      - run: go version
    

    Add check-latest input

    In scope of this release we add the check-latest input. If check-latest is set to true, the action first checks if the cached version is the latest one. If the locally cached version is not the most up-to-date, a Go version will then be downloaded from go-versions repository. By default check-latest is set to false. Example of usage:

    steps:
      - uses: actions/[email protected]
      - uses: actions/[email protected]
        with:
          go-version: '1.16'
          check-latest: true
      - run: go version
    

    Moreover, we updated @actions/core from 1.2.6 to 1.6.0

    v2.1.5

    In scope of this release we updated matchers.json to improve the problem matcher pattern. For more information please refer to this pull request

    v2.1.4

    What's Changed

    New Contributors

    Full Changelog: https://github.com/actions/setup-go/compare/v2.1.3...v2.1.4

    v2.1.3

    • Updated communication with runner to use environment files rather then workflow commands

    v2.1.2

    This release includes vendored licenses for this action's npm dependencies.

    ... (truncated)

    Commits
    • f6164bd Remove stable input and fix SemVer notation (#195)
    • 2bb2aab update types node (#194)
    • edcbc0c update lockfileVersion (#193)
    • fb9a043 Update default runtime to node16 (#192)
    • 5b0ae0e Bump pathval from 1.1.0 to 1.1.1 (#188)
    • bfdd357 Implement "check-latest" flag to check if pre-cached version is latest one (#...
    • 44e2214 Replace 'ubuntu-16.04' -> 'ubuntu-latest' (#180)
    • 424fc82 Merge pull request #179 from oscard0m/add-cache-to-node-workflows
    • 82388ea ci(workflow): add 'npm' cache for actions/setup-node in .github/workflows
    • fdeec47 Don't require relative paths to start with ./ or ../ (#98)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    build github_actions 
    opened by dependabot[bot] 1
  • vaultfs: Detect K/V Version 2 and support version param

    vaultfs: Detect K/V Version 2 and support version param

    Only the KV Version 1 secret engine will work out of the box (as of #19). KV Version 2 is also necessary. The added complication is the path is a bit different (there are data/ and metadata/ prefixes), and we need to support a version param.

    Note that right now providing params assumes a POST, whereas KVv2 (uniquely?) has a GET API.

    enhancement help wanted 
    opened by hairyhenderson 5
  • vaultfs: Support AWS/EC2 auth method

    vaultfs: Support AWS/EC2 auth method

    To bring this into parity with gomplate, AWS auth needs to be supported. It'd be good to support both the ec2 and iam type, but at least ec2 needs to be present for parity.

    opened by hairyhenderson 0
Owner
Dave Henderson
Just some guy...
Dave Henderson
A FileSystem Abstraction System for Go

A FileSystem Abstraction System for Go Overview Afero is a filesystem framework providing a simple, uniform and universal API interacting with any fil

Steve Francia 4.4k May 17, 2022
A package to allow one to concurrently go through a filesystem with ease

skywalker Skywalker is a package to allow one to concurrently go through a filesystem with ease. Features Concurrency BlackList filtering WhiteList fi

Will Dixon 79 May 6, 2022
An implementation of the FileSystem interface for tar files.

TarFS A wrapper around tar.Reader. Implements the FileSystem interface for tar files. Adds an Open method, that enables reading of file according to i

Eyal Posener 50 May 17, 2022
Takes an input http.FileSystem (likely at go generate time) and generates Go code that statically implements it.

vfsgen Package vfsgen takes an http.FileSystem (likely at go generate time) and generates Go code that statically implements the provided http.FileSys

null 943 Apr 29, 2022
memfs: A simple in-memory io/fs.FS filesystem

memfs: A simple in-memory io/fs.FS filesystem memfs is an in-memory implementation of Go's io/fs.FS interface. The goal is to make it easy and quick t

Peter Sanford 60 May 21, 2022
A Go io/fs filesystem implementation for reading files in a Github gists.

GistFS GistFS is an io/fs implementation that enables to read files stored in a given Gist. Requirements This module depends on io/fs which is only av

Jean Hadrien Chabran 124 May 1, 2022
A Small Virtual Filesystem in Go

This is a virtual filesystem I'm coding to teach myself Go in a fun way. I'm documenting it with a collection of Medium posts that you can find here.

Alyson 31 Apr 18, 2022
CRFS: Container Registry Filesystem

CRFS: Container Registry Filesystem Discussion: https://github.com/golang/go/issues/30829 Overview CRFS is a read-only FUSE filesystem that lets you m

Google 1.2k May 19, 2022
Encrypted overlay filesystem written in Go

An encrypted overlay filesystem written in Go. Official website: https://nuetzlich.net/gocryptfs (markdown source). gocryptfs is built on top the exce

null 2.3k May 20, 2022
A Go filesystem package for working with files and directories

Stowage A Go filesystem package for working with files and directories, it features a simple API with support for the common files and directories ope

null 19 May 28, 2021
filesystem for golang

filesystem filesystem for golang installation go get github.com/go-component/filesystem import import "github.com/go-component/filesystem" Usage sup

null 4 Jul 9, 2021
A set of io/fs filesystem abstractions and utilities for Go

A set of io/fs filesystem abstractions and utilities for Go Please ⭐ this project Overview This package provides io/fs interfaces for: Cloud providers

null 8 Mar 24, 2022
Tarserv serves streaming tar files from filesystem snapshots.

tarserv A collection of tools that allow serving large datasets from local filesystem snapshots. It is meant for serving big amounts of data to shell

Aurora 1 Jan 11, 2022
Grep archive search in any files on the filesystem, in archive and even inner archive.

grep-archive Grep archive search for string in any files on the filesystem, in archive and even inner archive. Supported archive format are : Tar Form

Michel Prunet 0 Jan 26, 2022
Easily create & extract archives, and compress & decompress files of various formats

archiver Introducing Archiver 3.1 - a cross-platform, multi-format archive utility and Go library. A powerful and flexible library meets an elegant CL

Matt Holt 3.6k May 21, 2022
Provide an upload endpoint that stores files on pinata and returns a json response with the uploaded file pinata url

Purpose Build a template repository to get to coding as quickly as possible, by starting from a common template which follows the guidelines here Feat

Dathan Vance Pattishall 0 Dec 30, 2021
Upgit - Upgit helps you simply upload any file to your Github repository and then get a raw URL for it

Upgit - Upgit helps you simply upload any file to your Github repository and then get a raw URL for it

null 256 May 18, 2022
Various implementations of a go-routine safe counter

Shared Counter Implementations Not Safe Counter Mutex Counter Channel Counter Atomic Counter Compare and Swap Counter Compare and Swap Float Counter T

Bruno Calza 26 Jan 16, 2022
Go package to generate and manage color palettes & schemes 🎨

Go package to generate and manage color palettes & schemes

Christian Muehlhaeuser 477 May 11, 2022
Mini-framework for multiple authentication and authorization schemes

Go authorization pattern This repository demonstrates an authorization pattern that allows multiple schemes. Demo To start the demo run the following

Tim van Osch 0 Dec 30, 2021
Singlestore event analytics - Evaluation of sortable ID generation schemes

Singlestore event analytics - Evaluation of sortable ID generation schemes

Carl Sverre 1 Jan 25, 2022
:steam_locomotive: Decodes url.Values into Go value(s) and Encodes Go value(s) into url.Values. Dual Array and Full map support.

Package form Package form Decodes url.Values into Go value(s) and Encodes Go value(s) into url.Values. It has the following features: Supports map of

Go Playgound 524 May 12, 2022
High-Performance Shortlink ( Short URL ) app creator in Golang. For privacy reasons, you may prefer to host your own short URL app and this is the one to use.

About The Project Shortlink App in Golang Multiple Node based Architecture to create and scale at ease Highly performant key-value storage system Cent

null 123 May 8, 2022
Go-based search engine URL collector , support Google, Bing, can be based on Google syntax batch collection URL

Go-based search engine URL collector , support Google, Bing, can be based on Google syntax batch collection URL

Re 45 May 13, 2022
A productivity tools to diagnose list of exported URL status from Google Search Console, Analytics, Sitemap URL...etc.

google-url-checker A productivity tools to diagnose list of exported URL status from Google Search Console, Analytics, Sitemap URL...etc. A quick way

null 0 Dec 31, 2021
Putty-url-scheme - Open PuTTY as a url scheme

PuTTY URL Scheme Helper Open PuTTY as a url scheme Install download release bina

Dark495 4 Apr 25, 2022
Test-app-url-shortner - A sample url shortener app to test Keploy integration capabilities

test-app-url-shortner A sample url shortener app to test Keploy integration capa

null 1 Jan 23, 2022
Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more)

gokv Simple key-value store abstraction and implementations for Go Contents Features Simple interface Implementations Value types Marshal formats Road

Philipp Gillé 432 May 21, 2022