Easily create & extract archives, and compress & decompress files of various formats

Overview

archiver archiver GoDoc

Introducing Archiver 3.1 - a cross-platform, multi-format archive utility and Go library. A powerful and flexible library meets an elegant CLI in this generic replacement for several platform-specific or format-specific archive utilities.

Features

Package archiver makes it trivially easy to make and extract common archive formats such as tarball (and its compressed variants) and zip. Simply name the input and output file(s). The arc command runs the same on all platforms and has no external dependencies (not even libc). It is powered by the Go standard library and several third-party, pure-Go libraries.

Files are put into the root of the archive; directories are recursively added, preserving structure.

  • Make whole archives from a list of files
  • Open whole archives to a folder
  • Extract specific files/folders from archives
  • Stream files in and out of archives without needing actual files on disk
  • Traverse archive contents without loading them
  • Compress files
  • Decompress files
  • Streaming compression and decompression
  • Several archive and compression formats supported

Format-dependent features

  • Gzip is multithreaded
  • Optionally create a top-level folder to avoid littering a directory or archive root with files
  • Toggle overwrite existing files
  • Adjust compression level
  • Zip: store (not compress) already-compressed files
  • Make all necessary directories
  • Open password-protected RAR archives
  • Optionally continue with other files after an error

Supported compression formats

  • brotli (br)
  • bzip2 (bz2)
  • flate (zip)
  • gzip (gz)
  • lz4
  • snappy (sz)
  • xz
  • zstandard (zstd)

Supported archive formats

  • .zip
  • .tar (including any compressed variants like .tar.gz)
  • .rar (read-only)

Tar files can optionally be compressed using any of the above compression formats.

GoDoc

See https://pkg.go.dev/github.com/mholt/archiver/v3

Install

With webi

webi will install webi and arc to ~/.local/bin/ and update your PATH.

Mac, Linux, Raspberry Pi

curl -fsS https://webinstall.dev/arc | bash

Windows 10

curl.exe -fsS -A MS https://webinstall.dev/arc | powershell

With Go

To install the runnable binary to your $GOPATH/bin:

go get github.com/mholt/archiver/cmd/arc

Manually

To install manually

  1. Download the binary for your platform from the Github Releases page.
  2. Move the binary to a location in your path, for example:
    • without sudo:
      chmod a+x ~/Downloads/arc_*
      mkdir -p ~/.local/bin
      mv ~/Downloads/arc_* ~/.local/bin/arc
    • as root:
      chmod a+x ~/Downloads/arc_*
      sudo mkdir -p /usr/local/bin
      sudo mv ~/Downloads/arc_* /usr/local/bin/arc
  3. If needed, update ~/.bashrc or ~/.profile to include add arc in your PATH, for example:
    echo 'PATH="$HOME:/.local/bin:$PATH"' >> ~/.bashrc
    

Build from Source

You can successfully build arc with just the go tooling, or with goreleaser.

With go

go build cmd/arc/*.go

Multi-platform with goreleaser

Builds with goreleaser will also include version info.

goreleaser --snapshot --skip-publish --rm-dist

Command Use

Make new archive

# Syntax: arc archive [archive name] [input files...]

arc archive test.tar.gz file1.txt images/file2.jpg folder/subfolder

(At least one input file is required.)

Extract entire archive

# Syntax: arc unarchive [archive name] [destination]

arc unarchive test.tar.gz

(The destination path is optional; default is current directory.)

The archive name must end with a supported file extension—this is how it knows what kind of archive to make. Run arc help for more help.

List archive contents

# Syntax: arc ls [archive name]

arc ls caddy_dist.tar.gz
drwxr-xr-x  matt    staff   0       2018-09-19 15:47:18 -0600 MDT   dist/
-rw-r--r--  matt    staff   6148    2017-08-07 18:34:22 -0600 MDT   dist/.DS_Store
-rw-r--r--  matt    staff   22481   2018-09-19 15:47:18 -0600 MDT   dist/CHANGES.txt
-rw-r--r--  matt    staff   17189   2018-09-19 15:47:18 -0600 MDT   dist/EULA.txt
-rw-r--r--  matt    staff   25261   2016-03-07 16:32:00 -0700 MST   dist/LICENSES.txt
-rw-r--r--  matt    staff   1017    2018-09-19 15:47:18 -0600 MDT   dist/README.txt
-rw-r--r--  matt    staff   288     2016-03-21 11:52:38 -0600 MDT   dist/gitcookie.sh.enc
...

Extract a specific file or folder from an archive

# Syntax: arc extract [archive name] [path in archive] [destination on disk]

arc extract test.tar.gz foo/hello.txt extracted/hello.txt

Compress a single file

# Syntax: arc compress [input file] [output file]

arc compress test.txt compressed_test.txt.gz
arc compress test.txt gz

For convenience, the output file (second argument) may simply be a compression format (without leading dot), in which case the output filename will be the same as the input filename but with the format extension appended, and the input file will be deleted if successful.

Decompress a single file

# Syntax: arc decompress [input file] [output file]

arc decompress test.txt.gz original_test.txt
arc decompress test.txt.gz

For convenience, the output file (second argument) may be omitted. In that case, the output filename will have the same name as the input filename, but with the compression extension stripped from the end; and the input file will be deleted if successful.

Flags

Flags are specified before the subcommand. Use arc help or arc -h to get usage help and a description of flags with their default values.

Library Use

The archiver package allows you to easily create and open archives, walk their contents, extract specific files, compress and decompress files, and even stream archives in and out using pure io.Reader and io.Writer interfaces, without ever needing to touch the disk.

To use as a dependency in your project:

go get github.com/mholt/archiver/v3
import "github.com/mholt/archiver/v3"

See the package's GoDoc for full API documentation.

For example, creating or unpacking an archive file:

err := archiver.Archive([]string{"testdata", "other/file.txt"}, "test.zip")
// ...
err = archiver.Unarchive("test.tar.gz", "test")

The archive format is determined by file extension. (There are several functions in this package which perform a task by inferring the format from file extension or file header, including Archive(), Unarchive(), CompressFile(), and DecompressFile().)

To configure the archiver used or perform, create an instance of the format's type:

z := archiver.Zip{
	CompressionLevel:       flate.DefaultCompression,
	MkdirAll:               true,
	SelectiveCompression:   true,
	ContinueOnError:        false,
	OverwriteExisting:      false,
	ImplicitTopLevelFolder: false,
}

err := z.Archive([]string{"testdata", "other/file.txt"}, "/Users/matt/Desktop/test.zip")

Inspecting an archive:

err = z.Walk("/Users/matt/Desktop/test.zip", func(f archiver.File) error {
	zfh, ok := f.Header.(zip.FileHeader)
	if ok {
		fmt.Println("Filename:", zfh.Name)
	}
	return nil
})

Streaming files into an archive that is being written to the HTTP response:

err = z.Create(responseWriter)
if err != nil {
	return err
}
defer z.Close()

for _, fname := range filenames {
	info, err := os.Stat(fname)
	if err != nil {
		return err
	}

	// get file's name for the inside of the archive
	internalName, err := archiver.NameInArchive(info, fname, fname)
	if err != nil {
		return err
	}

	// open the file
	file, err := os.Open(f)
	if err != nil {
		return err
	}

	// write it to the archive
	err = z.Write(archiver.File{
		FileInfo: archiver.FileInfo{
			FileInfo:   info,
			CustomName: internalName,
		},
		ReadCloser: file,
	})
	file.Close()
	if err != nil {
		return err
	}
}

The archiver.File type allows you to use actual files with archives, or to mimic files when you only have streams.

There's a lot more that can be done, too. See the GoDoc for full API documentation.

Security note: This package does NOT attempt to mitigate zip-slip attacks. It is extremely difficult to do properly and seemingly impossible to mitigate effectively across platforms. Attempted fixes have broken processing of legitimate files in production, rendering the program unusable. Our recommendation instead is to inspect the contents of an untrusted archive before extracting it (this package provides Walkers) and decide if you want to proceed with extraction.

Project Values

This project has a few principle-based goals that guide its development:

  • Do our thing really well. Our thing is creating, opening, inspecting, compressing, and streaming archive files. It is not meant to be a replacement for specific archive format tools like tar, zip, etc. that have lots of features and customizability. (Some customizability is OK, but not to the extent that it becomes overly complicated or error-prone.)

  • Have good tests. Changes should be covered by tests.

  • Limit dependencies. Keep the package lightweight.

  • Pure Go. This means no cgo or other external/system dependencies. This package should be able to stand on its own and cross-compile easily to any platform -- and that includes its library dependencies.

  • Idiomatic Go. Keep interfaces small, variable names semantic, vet shows no errors, the linter is generally quiet, etc.

  • Be elegant. This package should be elegant to use and its code should be elegant when reading and testing. If it doesn't feel good, fix it up.

  • Well-documented. Use comments prudently; explain why non-obvious code is necessary (and use tests to enforce it). Keep the docs updated, and have examples where helpful.

  • Keep it efficient. This often means keep it simple. Fast code is valuable.

  • Consensus. Contributions should ideally be approved by multiple reviewers before being merged. Generally, avoid merging multi-chunk changes that do not go through at least one or two iterations/reviews. Except for trivial changes, PRs are seldom ready to merge right away.

  • Have fun contributing. Coding is awesome!

We welcome contributions and appreciate your efforts! However, please open issues to discuss any changes before spending the time preparing a pull request. This will save time, reduce frustration, and help coordinate the work. Thank you!

Comments
  • Add support for storing symlinks in tar and zip archives

    Add support for storing symlinks in tar and zip archives

    Also implement extraction of symlinks from zip archives.

    This PR also adds a relative symlink in the testdata directory. It passes all tests on OS X, but has not been tested on Windows.

    This PR is a superset of changes from the following issues and PRs:

    Fixes #21 Fixes #31 Fixes #60 Fixes #74

    opened by jandubois 20
  • v4: Implement FS over an io.ReadSeeker stream

    v4: Implement FS over an io.ReadSeeker stream

    Please include lines https://github.com/mholt/archiver/blob/10c5080fa78f78d10e28abdf11fa6d4abb7f999f/fs.go#L40 - https://github.com/mholt/archiver/blob/10c5080fa78f78d10e28abdf11fa6d4abb7f999f/fs.go#L53 as an example for library use (handle archive as an io.FS) - it took me an hour to find out how to do it, and it is essentially VERY easy, but this information was hard to find.

    feature request 
    opened by tgulacsi 16
  • 404 when serve http

    404 when serve http

    What version of the package or command are you using?

    v4.0.0-alpha.7

    What are you trying to do?

    Serve archiveFS using http.FileServer

    What steps did you take?

    caddy.tar.gz

    func TestServer(t *testing.T) {
    	file := "caddy.tar.gz"
    	fs, _ := archiver.FileSystem(file)
    	server := http.FileServer(http.FS(fs))
    	http.HandleFunc("/", func(writer http.ResponseWriter, request *http.Request) {
    		// disable range support
    		writer.Header().Set("Accept-Ranges", "none")
    		request.Header.Del("Range")
    
    		// disable content-type sniffing
    		ctype := mime.TypeByExtension(filepath.Ext(request.URL.Path))
    		writer.Header()["Content-Type"] = nil
    		if ctype != "" {
    			writer.Header().Set("Content-Type", ctype)
    		}
    		server.ServeHTTP(writer, request)
    	})
    	t.Fatal(http.ListenAndServe("localhost:1234", nil))
    }
    

    Open http://localhost:1234

    What did you expect to happen, and what actually happened instead?

    Browsing archive content on the fly. But every link when opened return 404.

    How do you think this should be fixed?

    When matching files, nomalize NameInArchive to be cross platform compatible.

    Please link to any related issues, pull requests, and/or discussion

    None

    opened by WeidiDeng 14
  • RFE: port to github.com/pierrec/lz4/v4

    RFE: port to github.com/pierrec/lz4/v4

    What would you like to have changed?

    Make it possible to build with github.com/pierrec/lz4/v4.

    Why is this feature a useful, necessary, and/or important addition to this project?

    lz4 was updated to 4.0.2 in Fedora rawhide and archiver no longer builds using distribution-provided Go packages as a result.

    What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

    I'm forced to stop updating archiver until it's ported to lz4 v4 API or someone creates a compatibility package with v3 API.

    Please link to any relevant issues, pull requests, or other discussions.

    To see the issue, patch the source with the following patch:

    diff -up archiver-3.3.2/lz4.go.lz4 archiver-3.3.2/lz4.go
    --- archiver-3.3.2/lz4.go.lz4	2020-09-28 10:43:21.000000000 +0200
    +++ archiver-3.3.2/lz4.go	2020-10-05 13:06:07.879465436 +0200
    @@ -5,7 +5,7 @@ import (
     	"io"
     	"path/filepath"
     
    -	"github.com/pierrec/lz4/v3"
    +	"github.com/pierrec/lz4"
     )
     
     // Lz4 facilitates LZ4 compression.
    diff -up archiver-3.3.2/tarlz4.go.lz4 archiver-3.3.2/tarlz4.go
    --- archiver-3.3.2/tarlz4.go.lz4	2020-09-28 10:43:21.000000000 +0200
    +++ archiver-3.3.2/tarlz4.go	2020-10-05 13:06:17.578418304 +0200
    @@ -5,7 +5,7 @@ import (
     	"io"
     	"strings"
     
    -	"github.com/pierrec/lz4/v3"
    +	"github.com/pierrec/lz4"
     )
     
     // TarLz4 facilitates lz4 compression
    

    and build. Fedora package build process fails with the following errors:

    _build/src/github.com/mholt/archiver/lz4.go:19:3: w.Header undefined (type *lz4.Writer has no field or method Header)
    _build/src/github.com/mholt/archiver/tarlz4.go:87:7: lz4w.Header undefined (type *lz4.Writer has no field or method Header)
    

    It looks like option handling and compression level setting got changed: https://github.com/pierrec/lz4/compare/v3.3.2..v4.0.2

    I have nearly zero Golang knowledge, but this crude patch makes it compile and go test passes:

    diff -up archiver-3.3.2/lz4.go.lz4 archiver-3.3.2/lz4.go
    --- archiver-3.3.2/lz4.go.lz4	2020-09-28 10:43:21.000000000 +0200
    +++ archiver-3.3.2/lz4.go	2020-10-05 13:28:21.581995885 +0200
    @@ -5,7 +5,7 @@ import (
     	"io"
     	"path/filepath"
     
    -	"github.com/pierrec/lz4/v3"
    +	"github.com/pierrec/lz4"
     )
     
     // Lz4 facilitates LZ4 compression.
    @@ -16,7 +16,12 @@ type Lz4 struct {
     // Compress reads in, compresses it, and writes it to out.
     func (lz *Lz4) Compress(in io.Reader, out io.Writer) error {
     	w := lz4.NewWriter(out)
    -	w.Header.CompressionLevel = lz.CompressionLevel
    +	options := []lz4.Option{
    +		lz4.CompressionLevelOption(lz4.CompressionLevel(1 << (8 + lz.CompressionLevel))),
    +	}
    +	if err := w.Apply(options...); err != nil {
    +		return err
    +	}
     	defer w.Close()
     	_, err := io.Copy(w, in)
     	return err
    diff -up archiver-3.3.2/tarlz4.go.lz4 archiver-3.3.2/tarlz4.go
    --- archiver-3.3.2/tarlz4.go.lz4	2020-09-28 10:43:21.000000000 +0200
    +++ archiver-3.3.2/tarlz4.go	2020-10-05 13:28:21.581995885 +0200
    @@ -5,7 +5,7 @@ import (
     	"io"
     	"strings"
     
    -	"github.com/pierrec/lz4/v3"
    +	"github.com/pierrec/lz4"
     )
     
     // TarLz4 facilitates lz4 compression
    @@ -84,7 +84,12 @@ func (tlz4 *TarLz4) wrapWriter() {
     	var lz4w *lz4.Writer
     	tlz4.Tar.writerWrapFn = func(w io.Writer) (io.Writer, error) {
     		lz4w = lz4.NewWriter(w)
    -		lz4w.Header.CompressionLevel = tlz4.CompressionLevel
    +		options := []lz4.Option{
    +			lz4.CompressionLevelOption(lz4.CompressionLevel(1 << (8 + tlz4.CompressionLevel))),
    +		}
    +		if err := lz4w.Apply(options...); err != nil {
    +			return lz4w, err
    +		}
     		return lz4w, nil
     	}
     	tlz4.Tar.cleanupWrapFn = func() {
    
    feature request 
    opened by rathann 13
  • craft zip file for symlink testing

    craft zip file for symlink testing

    We need a special zip file that cannot be created with normal commandline tools. It requires crafting with an API. This should be possible with archive/zip#Writer, for example.

    We want a double entry of a file - the first being a symlink such that the second will be placed in an arbitrary location:

    ./goodfile.txt  "hello world"         (file)
    ./bad/file.txt  => ../../badfile.txt  (symlink)
    ./bad/file.txt  "Mwa-ha-ha"           (file)
    ./morefile.txt  "hello world"         (file)
    

    This should go in testdata/testarchives/evilarchives/ as double-evil.zip and double-evil.tar (if it is allowed).

    See also https://github.com/mholt/archiver/issues/242#issuecomment-703086020

    opened by coolaj86 13
  • fix: prevent extraction of archived files outside target path

    fix: prevent extraction of archived files outside target path

    Why this PR?

    This PR is meant to fix an arbitrary file write vulnerability, that can be achieved using a specially crafted zip archive, that holds path traversal filenames. When the filename gets concatenated to the target extraction directory, the final path ends up outside of the target folder.

    A sample malicious zip file named zip-slip.zip. (see this gist) was used, and when running the code below, resulted in creation of evil.txt file in /tmp folder.

    package main
    
    import "log"
    import "github.com/mholt/archiver"
    
    func main() {
    	err := archiver.Tar.Open("/tmp/evil-tar.tar", "/tmp/safe")
    	if err != nil {
    		log.Fatal(err)
    	}
    }
    

    There are various possible ways to avoid this issue, some include checking for .. (dot dot) characters in the filename, but the best solution in our opinion is to check if the final target filename, starts with the target folder (after both are resolved to their absolute path).

    Stay secure, Snyk Team

    opened by aviadatsnyk 13
  • Fix hard links

    Fix hard links

    Had to refactor how paths/filenames are passed into some functions to accommodate hard-linked files.

    @petemoore Would you please try this out?

    Should fix #152.

    opened by mholt 12
  • Installation Instructions Incorrect / go get fails

    Installation Instructions Incorrect / go get fails

    What version of the package or command are you using?

    Not sure how to check. I just ran go get github.com/mholt/archiver/v3

    What are you trying to do?

    Install the package and use it.

    What steps did you take?

    Installed go on Windows. Ran go get github.com/mholt/archiver/v3

    What did you expect to happen, and what actually happened instead?

    The package should successfully install.

    This happened instead:

    package github.com/mholt/archiver/v3: cannot find package "github.com/mholt/archiver/v3" in any of:
            c:\go\src\github.com\mholt\archiver\v3 (from $GOROOT)
            C:\Users\vroy1\go\src\github.com\mholt\archiver\v3 (from $GOPATH)
    

    Doing go get github.com/mholt/archiver works better and throws the following error instead:

    C:\Users\vroy1\Desktop\server>go get github.com/mholt/archiver/  
    package github.com/pierrec/lz4/v3: cannot find package "github.com/pierrec/lz4/v3" in any of:
            c:\go\src\github.com\pierrec\lz4\v3 (from $GOROOT)
            C:\Users\vroy1\go\src\github.com\pierrec\lz4\v3 (from $GOPATH)
    

    Please link to any related issues, pull requests, and/or discussion

    https://github.com/mholt/archiver/issues/195

    opened by vedantroy 11
  • Update github.com/pierrec/lz4 dependency to module version

    Update github.com/pierrec/lz4 dependency to module version

    Updates dependency to use github.com/pierrec/lz4/v3 at version v3.0.1. This version is functionally equivalent to the previously specified dependency.

    opened by nmiyake 11
  • use filepath.Dir() instead of path.Dir()

    use filepath.Dir() instead of path.Dir()

    Please use filepath.Dir(), not path.Dir().

    In Windows, following code doesn't work with Error peco_windows_amd64\peco_windows_amd64\Changes: creating new file: open peco_windows_amd64\peco_windows_amd64\Changes: The system cannot find the path specified.

    package main
    
    import (
        "fmt"
        "io"
        "net/http"
        "os"
        "path"
    
        "github.com/mholt/archiver"
    )
    
    func main() {
        url := "https://github.com/peco/peco/releases/download/v0.4.0/peco_windows_amd64.zip"
        resp, _ := http.Get(url)
        defer resp.Body.Close()
        fname := path.Base(url)
        f, _ := os.Create(fname)
        io.Copy(f, resp.Body)
    
        err := archiver.Unzip(fname, "peco_windows_amd64")
        if err != nil {
            fmt.Println(err.Error())
        }
    }
    
    

    And following code doesn't extract .tar.gz file including symlink with Error: dest\hoge.txt: creating new file: open dest\hoge.txt: The system cannot find the path specified.

    $ tar -tvf hoge.tar.gz
    -rw-r--r-- username/197121     0 2016-08-20 14:35 hoge.txt
    lrwxrwxrwx username/197121     0 2016-08-20 14:12 link -> hoge.txt
    
    package main
    
    import (
        "fmt"
    
        "github.com/mholt/archiver"
    )
    
    func main() {
        err := archiver.UntarGz("hoge.tar.gz", "dest")
        if err != nil {
            fmt.Println(err.Error())
        }
    
    opened by whatalnk 11
  • m1 related unarchive inconsistency, dropping the root dir

    m1 related unarchive inconsistency, dropping the root dir

    What version of the package or command are you using?

    github.com/mholt/archiver/v4 v4.0.0-alpha.6.0.20220421032531-8a97d87612e9

    What are you trying to do?

    unarchive a tar.gz directory, given the basic wrapper function:

    func Unarchive(input io.Reader, dir string) error {
    	// TODO: consider if should write to a more generic interface
    	// like a writer, or if maybe if the function itself
    	// should take the handler as an input so can be as generic
    	// as you'd like in the handler
    	format, input, err := archiver.Identify("", input)
    	if err != nil {
    		return err
    	}
    	// the list of files we want out of the archive; any
    	// directories will include all their contents unless
    	// we return fs.SkipDir from our handler
    	// (leave this nil to walk ALL files from the archive)
    
    	handler := func(ctx context.Context, f archiver.File) error {
    		newPath := filepath.Join(dir, f.NameInArchive)
    		if f.IsDir() {
    			return os.MkdirAll(newPath, f.Mode())
    		}
    		newFile, err := os.OpenFile(newPath, os.O_CREATE|os.O_WRONLY, f.Mode())
    		if err != nil {
    			return err
    		}
    		defer newFile.Close()
    		// copy file data into tar writer
    		af, err := f.Open()
    		if err != nil {
    			return err
    		}
    		defer af.Close()
    		if _, err := io.Copy(newFile, af); err != nil {
    			return err
    		}
    		return nil
    	}
    	// make sure the format is capable of extracting
    	ex, ok := format.(archiver.Extractor)
    	if !ok {
    		return err
    	}
    	return ex.Extract(context.Background(), input, nil, handler)
    }
    

    What steps did you take?

    On the mac, given a tar archive with a root dir of quarto-0.9.532 and directories

    tar -tvf ~/Downloads/quarto-0.9.532-linux-amd64.tar.gz
    drwxr-xr-x  0 runner docker      0 Jun  6 18:20 quarto-0.9.532/
    drwxr-xr-x  0 runner docker      0 Jun  6 18:20 quarto-0.9.532/bin/
    

    unpacking manually, can likewise see a directory structure:

    .
    └── quarto-0.9.542
       ├── bin
       └── share
    

    however when I add fmt.Println("name in archive: ", f.NameInArchive) I see on the m1 mac

    name in archive:  ./
    name in archive:  ./bin/
    name in archive:  ./share/
    

    On linux, I do see the correct behavior.

    name in archive:  quarto-0.9.542/
    name in archive:  quarto-0.9.542/bin/
    name in archive:  quarto-0.9.542/share/
    

    What did you expect to happen, and what actually happened instead?

    expect to unarchive the directory as present in the archive

    How do you think this should be fixed?

    normalize behavior

    Please link to any related issues, pull requests, and/or discussion

    likely the inverse issue of #336

    Bonus: What do you use archiver for, and do you find it useful?

    unconfirmed 
    opened by dpastoor 10
  • archiver.FileSystem() alternative that takes io.Reader

    archiver.FileSystem() alternative that takes io.Reader

    What would you like to have changed?

    In addition to archiver.FileSystem("filename.ext") there should be a function that takes io.Reader or better io.ReaderAt and size. For example archiver.NewFsReader(r io.ReaderAt, size int64).

    Why is this feature a useful, necessary, and/or important addition to this project?

    This is necessary to use a file thats already open or for files that do not resize on OS filesystem and cannot be opened with os.Open(). For example afero or any other virtual/in memory file system.

    What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

    There are no alternatives, currently I can't use archiver.FileSystem with a file on a synthetic FS because archiver assumes the file can be opened with os.Open().

    Please link to any relevant issues, pull requests, or other discussions.

    Go archive/zip implements NewReader in addition to file Open.

    https://pkg.go.dev/archive/zip#NewReader

    Similarly SevenZip: https://pkg.go.dev/github.com/bodgit/sevenzip?utm_source=godoc#NewReader

    Thank you!

    feature request 
    opened by tenox7 5
  • Expose CompressionLevel for Zstd

    Expose CompressionLevel for Zstd

    What would you like to have changed?

    The TarZstd and Zstd don't express a way of setting the compression level.

    Why is this feature a useful, necessary, and/or important addition to this project?

    Most compressors already expose this.

    What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

    We would have to abandon the TarZstd and Zstd wrappers and handle them directly ourselves.

    Please link to any relevant issues, pull requests, or other discussions.

    #351

    feature request 
    opened by zeripath 1
  • Add common quality/compression interface

    Add common quality/compression interface

    What would you like to have changed?

    Add a common interface to allow setting of compression level using int. e.g.

    type ConfigurableCompressionLevel interface {
      SetCompressionLevel(int) 
    }
    

    (this could be SetCompressionLevel(int) bool to say whether this level is acceptable.)

    Why is this feature a useful, necessary, and/or important addition to this project?

    Many archivers/compressors have a way of setting compression level and this would allow these to be exposed in a common fashion.

    What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

    1. We can continue by casting as individual types of archiver
    2. Could use reflect.ValueOf(...).FieldByName("CompressionLevel").SetInt(...) / "Quality" as appropriate

    Please link to any relevant issues, pull requests, or other discussions.

    N/A at the moment.

    feature request 
    opened by zeripath 3
  • How about providing a dummy archiver?

    How about providing a dummy archiver?

    Dear Matt,

    Thanks to your great lib. What I'm working is optional compressed extensions, so non-compressed ones that they do nothing about compression/decompression are included.

    What would you like to have changed?

    provide a dummy implementation

    Why is this feature a useful, necessary, and/or important addition to this project?

    easy to deal with non-compressed options

    What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

    DIY or writing some special hard codes

    Please link to any relevant issues, pull requests, or other discussions.

    maybe it will look like bellows (in v3)

    type DummyArchiver struct{}
    
    func (DummyArchiver) CheckExt(string) error { return nil }
    func (DummyArchiver) Compress(in io.Reader, out io.Writer) error {
    	_, err := io.Copy(out, in)
    	return err
    }
    func (DummyArchiver) Decompress(in io.Reader, out io.Writer) error {
    	_, err := io.Copy(out, in)
    	return err
    }
    
    feature request 
    opened by orca-zhang 1
  • Add a fs that reads converts achiveFS to a seekable version

    Add a fs that reads converts achiveFS to a seekable version

    What would you like to have changed?

    A similar implementation like tarfs or zipfs is appreciated.

    Why is this feature a useful, necessary, and/or important addition to this project?

    For performance and intuitivity. Current implementation of archiveFs is a glorified extract operation that tries to extract the file from the reader. Also due to the different arcive layout, some file may be at end the of archive. Every time a file is requested the process repeats which is bad for performance.

    an FS.fs can be used in http.Filesystem, however because of archiveFs is implemented, file object can't be seeked, which means when golang http can't figure out content-type from file name or client requested range seek will fail, leading to error.

    What alternatives are there, or what are you doing in the meantime to work around the lack of this feature?

    In cloudreve, registering a mime type is all that's needed.

    Please link to any relevant issues, pull requests, or other discussions.

    Performance issues, seek error, cloudreve ttf can't be recognized without seeking on windows.

    There is a hack for tar, but I doubt you are willing to do this for tar, rar or zip etc.

    help wanted feature request 
    opened by WeidiDeng 2
Releases(v4.0.0-alpha.6)
Owner
Matt Holt
M.S. Computer Science. Author of the Caddy Web Server, CertMagic, Papa Parse, JSON/curl-to-Go, Timeliner, Relica, and more...
Matt Holt
Go bindings for unarr (decompression library for RAR, TAR, ZIP and 7z archives)

go-unarr Golang bindings for the unarr library from sumatrapdf. unarr is a decompression library and CLI for RAR, TAR, ZIP and 7z archives. GoDoc See

Milan Nikolic 215 Nov 25, 2022
Tool to easily rename or move a bunch of files with a text editor of your choice

batch-rename With batch-rename you can utilize your favorite text editor to rename or move a bunch of files at once. It doesn't come with any features

David Vogel 2 Nov 2, 2022
A comphrehensive tool for converting between data table formats

tblconv A simple tool for converting one "table" format into another. Supported Formats CSV source output Excel source output SQL source output CLI Fe

null 0 May 30, 2022
Parse NYT crossword puzzle score screenshots and extract the times.

Parse NYT crossword puzzle score screenshots and extract the times.

Keegan Campbell 3 Mar 11, 2022
Lib to extract information of tag html meta

What is this? Is a lib to extract information to mount preview. For Example: When you insert a url on chat how WhatsApp is mounted an preview of websi

null 5 May 17, 2022
Package buildinfo provides basic building blocks and instructions to easily add build and release information to your app.

Package buildinfo provides basic building blocks and instructions to easily add build and release information to your app. This is done by replacing variables in main during build with ldflags.

null 1 Nov 14, 2021
Gosfdc module - a collection of packages containing the data structures from the various Salesforce APIs and Tools

Gosfdc module - a collection of packages containing the data structures from the various Salesforce APIs and Tools

Rodolphe Blancho 0 Jan 21, 2022
Various Dungeons and Dragons Tools. Written in go as an aid to learning the language.

dnd_tools Various Dungeons and Dragons Tools. Written in go as an aid to learning the language. Some tools are generic, while others will target eithe

SREiously 0 Jan 28, 2022
An simple, easily extensible and concurrent health-check library for Go services

Healthcheck A simple and extensible RESTful Healthcheck API implementation for Go services. Health provides an http.Handlefunc for use as a healthchec

Ether Labs 244 Nov 28, 2022
Alfred 4 workflow to easily search and launch bookmarks from the Brave Browser

Alfred Brave Browser Bookmarks A simple and fast workflow for searching and launching Brave Browser bookmarks. Why this workflow? No python dependency

Josh Newman 9 Nov 28, 2022
Fast conversions across various Go types with a simple API.

Go Package: conv Get: go get -u github.com/cstockton/go-conv Example: // Basic types if got, err := conv.Bool(`TRUE`); err == nil { fmt.Printf("conv.

Chris Stockton 388 Nov 29, 2022
Track health of various dependencies - golang

Background This package helps setup health check based on status of external dependencies. The idea is to add all external dependencies like database,

Dave Amit 1 Dec 17, 2021
This repository provides various utilities to help you build your NFT collection!

Attention! A powerful computer may be required! About This repository provides various utilities to help you build your NFT collection: Generate image

Igor Savenko 6 Nov 4, 2022
This is a demo of various bursavich.dev packages.

Demo This provides a demo of various bursavich.dev packages. See the frontend command for a unified usage example, from which the following sample met

Andy Bursavich 1 Feb 10, 2022
Configmanager - Package used for retrieving application settings from various sources

Config Manager Package used for retrieving application settings from various sou

null 7 Nov 28, 2022
James is your butler and helps you to create, build, debug, test and run your Go projects

go-james James is your butler and helps you to create, build, debug, test and run your Go projects. When you often create new apps using Go, it quickl

Pieter Claerhout 56 Oct 8, 2022
create temporary Firefox profile, install user.js and extensions, launch Firefox

tmpfox tmpfox is a Firefox wrapper that: Creates a temporary Firefox profile Installs user.js configuration file from Arkenfox for increased privacy a

Charalampos Mitsakis 5 Jul 27, 2022
Onboarding exercise to create todo list using golang and postgres

Todo List API Description A RESTful API that allows a user to create, update, view, filter, and delete todos API Documentation Copy the contents of th

null 0 Dec 1, 2021
This example implements a basic example of how to create your own modules, and how to call them from other modules

This example implements a basic example of how to create your own modules, and how to call them from other modules. In addition, an example of how to do unit tests is developed.

null 1 Feb 1, 2022