Go copy directory recursively

Overview

copy

Go Reference Actions Status codecov License: MIT Go Report Card GitHub tag (latest SemVer)

copy copies directories recursively.

Example Usage

err := Copy("your/directory", "your/directory.copy")

Advanced Usage

// Options specifies optional actions on copying.
type Options struct {

	// OnSymlink can specify what to do on symlink
	OnSymlink func(src string) SymlinkAction

	// OnDirExists can specify what to do when there is a directory already existing in destination.
	OnDirExists func(src, dest string) DirExistsAction

	// Skip can specify which files should be skipped
	Skip func(src string) (bool, error)

	// AddPermission to every entities,
	// NO MORE THAN 0777
	AddPermission os.FileMode

	// Sync file after copy.
	// Useful in case when file must be on the disk
	// (in case crash happens, for example),
	// at the expense of some performance penalty
	Sync bool

	// Preserve the atime and the mtime of the entries
	// On linux we can preserve only up to 1 millisecond accuracy
	PreserveTimes bool

}
// For example...
opt := Options{
	Skip: func(src string) {
		return strings.HasSuffix(src, ".git")
	},
}
err := Copy("your/directory", "your/directory.copy", opt)

Issues

Comments
  • Adding Replace option

    Adding Replace option

    Dear @otiai10,

    What do you think of adding copy with replace option (WithReplace, true or false)?

    I noticed that if we copy one directory to another, it preserves old files as the way they are.

    I think, it would be a nice feature to have, if you need to replace the entire destination directory by the source one.

    By the way, nice package!

    opened by mariaefi29 17
  • Added Copy with requested Permissions

    Added Copy with requested Permissions

    In case someone wants to change file and directory permissions upon copying there's a new func Perm(src, dest string, dirPerm, filePerm os.FileMode) error which sets dirPerm for all copied directories and filePerfm for all copied files.

    I came upon the problem that in go modules loaded via go get files and directories have different permissions than in github. So I had to come up with this solution to be able to load from a template folder with the correct dest permissions independent from the src permissions.

    opened by puengel 12
  • Skip by file info too

    Skip by file info too

    Hello! Thank you for package.

    I faced to a problem when I need to copy all files with a specific extension, but then the Skip() function ignores directories.

    Therefore, I propose such a small addition. Thank u!

    P.S. Maybe you will rename a package so that it does not conflict with the standard copy function?

    opened by Antonboom 10
  • Make temporary permission changes of destination directory optional

    Make temporary permission changes of destination directory optional

    As of https://github.com/otiai10/copy/pull/9 the Copy operation makes temporary changes to the permissions on the destination directory to allow for the case when the source directory is not writeable.

    This behaviour is undesirable in the case where the destination directory is not owned by the process running the Copy - in that case the chmod operation will fail causing the copy to abort. This can happen in Kubernetes when mounting a volume into a container; the mount point will usually be owned by root. Using such a directory as the destination for a Copy is currently not possible using this module if the copying process is not also running as root.

    This PR adds a new option, NoTemporaryPermChanges, to skip the temporary modification of permissions on the target directory.

    opened by andrew-farries 9
  • Add license scan report and status

    Add license scan report and status

    Your FOSSA integration was successful! Attached in this PR is a badge and license report to track scan status in your README.

    Below are docs for integrating FOSSA license checks into your CI:

    opened by fossabot 8
  • Does not build on solaris.

    Does not build on solaris.

    Hi.

    I receive the following in goreleaser on solaris:

    ⨯ release failed after 186.87s error=failed to build for solaris_amd64: exit status 2: # github.com/otiai10/copy
    vendor/github.com/otiai10/copy/copy_namedpipes.go:16:9: undefined: syscall.Mkfifo
    vendor/github.com/otiai10/copy/test_setup.go:16:2: undefined: syscall.Mkfifo
    
    opened by gabyx 6
  • Handling owner permissions

    Handling owner permissions

    First of all, thanks for the great lib ;)

    It would be really nice if copy would handle (with an option, maybe) the owner permission of the resulting file - especially when it's recreating the target (symlink, or namedpipe, see also #47)

    Cheers

    opened by mudler 6
  • Make package work with WASM and GopherJS targets

    Make package work with WASM and GopherJS targets

    At present, this package does not work with WASM or GopherJS compile targets. See for example, the output from go test for WASM:

    $ GOOS=js GOARCH=wasm go test ./... -exec=$(go env GOROOT)/misc/wasm/go_js_wasm_exec
    # github.com/otiai10/copy [github.com/otiai10/copy.test]
    ./copy_namedpipes.go:16:9: undefined: syscall.Mkfifo
    ./stat_times.go:17:30: stat.Atim undefined (type *syscall.Stat_t has no field or method Atim)
    ./stat_times.go:18:30: stat.Ctim undefined (type *syscall.Stat_t has no field or method Ctim)
    ./test_setup.go:16:2: undefined: syscall.Mkfifo
    FAIL    github.com/otiai10/copy [build failed]
    FAIL
    

    and GopherJS:

    $ gopherjs test ./...
    test_setup.go:16:10: Mkfifo not declared by package syscall
    stat_times.go:17:31: stat.Atim undefined (type *syscall.Stat_t has no field or method Atim)
    stat_times.go:17:53: stat.Atim undefined (type *syscall.Stat_t has no field or method Atim)
    stat_times.go:18:31: stat.Ctim undefined (type *syscall.Stat_t has no field or method Ctim)
    stat_times.go:18:53: stat.Ctim undefined (type *syscall.Stat_t has no field or method Ctim)
    copy_namedpipes.go:16:17: Mkfifo not declared by package syscall
    

    This PR solves this problem, by using the same fallback used for Windows, which doesn't support Mkfifo, and by using the JS-specific struct field names for Atime/Ctime fields in the syscall package.

    opened by flimzy 5
  • add an option to deep-copy symlinks

    add an option to deep-copy symlinks

    Can we have an option to copy symlinks? I'm imagining something like:

    Opts struct {
      // Called if a symlink is found. If this function returns false, 
      // Copy copies the symlink itself. Else, Copy() follows the link.
      // If FollowSymlink=nil, Copy never follows symlinks.
      FollowSymlink func(path string) bool
    }
    

    func Copy(src, dest string, opt... Opts) error;

    I can create a pull request if it looks good.

    enhancement 
    opened by yasushi-saito 5
  • Support PreserveTimes for symlinks on Unix

    Support PreserveTimes for symlinks on Unix

    This is an addition to https://github.com/otiai10/copy/pull/31 - While symlink timestamps cannot be set universally (neither at time of link creation nor by a simple call to os.Chtimes()) they can be set on Unix making use of unix.Lutimes(). This PR extends the existing PreserveTime option by that, transferring the original link timestamp(s) to the newly created one if the option is set.

    For unsupported architectures, a stub is provided similar to already existing ones (e.g. for ownership preservation).

    opened by fako1024 4
  • copy.Copy hangs with named pipes

    copy.Copy hangs with named pipes

    It seems that copy.Copy is not supporting named pipes, at the moment it tries to copy it, resulting in the process to hang.

    Would you be interested in accepting a PR to reproduce the named pipe in the given destination ?

    opened by mudler 4
  • Fix symlinks resolved relative to current working directory

    Fix symlinks resolved relative to current working directory

    The Deep OnSymlink option now resolves symlinks relative to the directory containing the symlink instead of the current working directory.

    This is a breaking change.

    Fixes #26.

    opened by adombeck 1
  • Time preservation invalid: when the time and year of the file is 2275, the time of the destination cannot be modified

    Time preservation invalid: when the time and year of the file is 2275, the time of the destination cannot be modified

    code

    import (
    	cp "github.com/otiai10/copy"
    )
    opt := cp.Options{PreserveTimes: true}
    	err := cp.Copy("/Volume1/souce", "/Volume1/@usb/dest/",
    		opt,
    	)
    

    File system :exfat32 Create a new file in /Volume1/souce : touch -t 227506062145 top.txt Destination file time image

    Total inconsistency between source and destination times

    opened by Schofi 0
  • Option to copy two or more files simultaneous

    Option to copy two or more files simultaneous

    hi, is it possible to add an option to copy two or more files simultaneous? I think this would be an amazing performance boost on the most systems.

    What do you think?

    enhancement 
    opened by ManuelReschke 2
  • Can we support Update Opt, like --update in `cp`

    Can we support Update Opt, like --update in `cp`

    HiHi, copy is every easy to use, but have a feature to request.

    I have a lot of files need to copy from one folder to anther folder, the source folder have several files changed between dest folder.

    It will cause poor performance when we just want update several files to dest folder.

    So I want copy to support Update Opt like cp in Linux:

    -u, --update
                  copy only when the SOURCE file is newer than the destination file or when the destination file is missing
    
    opened by xiantang 1
  • "no such file or directory" when copying large amounts of data

    I'm using this package in a tool for backing up Docker volumes: https://github.com/offen/docker-volume-backup

    Users that do not want to stop their containers while taking a backup can opt in to copying their data to a temporary location before creating the tar archive so that creating the archive does not fail in case data is being written to a file while it's being backed up. To perform this copy, package copy is used (thanks for making it public, much appreciated).

    This seemed to work well in tests as well as the real world, however recently an issue was raised where copy would fail with the following error when backing up the data volume for a Prometheus container:

    open /backup/prometheus_data/01FSM8TPFEXQ0QC28H11PMQZ0R: no such file or directory
    

    The dataset that is being copied seems to be a. very large and b. pretty volatile which has me thinking this file might actually have been deleted/moved before copy finds the resources to actually copy it. This is the downstream issue: https://github.com/offen/docker-volume-backup/issues/49

    Is this issue somehow known? Is there a way to fix it by configuring copy differently?

    This is the part where I use copy in code and also where the above error is being returned:

    if err := copy.Copy(s.c.BackupSources, backupSources, copy.Options{
    	PreserveTimes: true,
    	PreserveOwner: true,
    }); err != nil {
    	return fmt.Errorf("takeBackup: error creating snapshot: %w", err)
    }
    
    opened by m90 6
Releases(v1.9.0)
Owner
Hiromu OCHIAI
🙋 ❤️ 🍣
Hiromu OCHIAI
copy files for humans

Go-Decent-Copy go-decent-copy provides a copy file for humans Usage package main import "github.com/hugocarreira/go-decent-copy" func main() { e

Hugo Carreira 17 Sep 26, 2022
Go Copy Zip file With Filter

Go Copy Zip file With Filter Unsurprisingly, this was written for Log4shell remediation, to remove the Jndi class from log4j-core.jar's It gives you j

Chris Lane 0 Jan 10, 2022
Recreate embedded filesystems from embed.FS type in current working directory.

rebed Recreate embedded filesystems from embed.FS type in current working directory. Expose the files you've embedded in your binary so users can see

Patricio Whittingslow 23 Sep 27, 2022
FSManager - Tree view Simple util to displays the directory structure of a path or of the disk in a drive graphically.

FSManager - Tree view Simple util to displays the directory structure of a path or of the disk in a drive graphically. If you don't specify a drive or

Kostiantyn Denysov 1 Oct 9, 2021
Dirmap is a tool for generating a directory map.

dirmap ?? dirmap is a tool for generating a directory map. It extracts a part of the document from markdown or source code of each directory and uses

Ken’ichiro Oyama 12 Nov 21, 2022
List all files (recursively) in a directory

typewalk List all files (recursively) in a directory Usage: recursively walk > ./typewalk.exe walk --path "C:\\" merge two files > ./typewalk.exe merg

Drilon Kamberaj 0 Nov 5, 2021
Maintain a lower-bitrate copy of a music library in sync with the main copy.

msync Maintain a lower-bitrate copy of your music library, in sync with the main copy.

Chris Dzombak 18 Mar 6, 2022
This is an example of a keep-it-simple directory layout for Go projects that was created using DDD principles, please copy and share if you like it.

DDD Go Template This project was created to illustrate a great architectural structure I developed together with @fabiorodrigues in the period I was w

Vinícius Garcia 39 Dec 5, 2022
:runner:runs go generate recursively on a specified path or environment variable and can filter by regex

Package generate Package generate runs go generate recursively on a specified path or environment variable like $GOPATH and can filter by regex Why wo

Go Playgound 28 Sep 27, 2022
A small utility command line application that can recursively download Notion pages

notionbackup A small utility command line application that can recursively download Notion pages. I needed something scriptable that could periodicall

Shayan 62 Dec 5, 2022
A API scanner written in GOLANG to scan files recursively and look for API keys and IDs.

GO FIND APIS _____ ____ ______ _____ _ _ _____ _____ _____ _____ / ____|/ __ \ | ____|_ _| \ | | __ \ /\ | __ \_

Sreekanth Sasi 3 Oct 25, 2021
Recursively searches a map[string]interface{} structure for another map[string]interface{} structure

msirecurse Recursively searches a map[string]interface{} structure for existence of a map[string]interface{} structure Motivation I wrote this package

Fred Moyer 1 Mar 3, 2022
Generate a global index for multiple markdown files recursively

markdown index Markdown-index is a library to help you generate a global index for multiple markdown files recursively in a directory, containing a su

Mateus Miranda 18 Sep 25, 2022
Simple wrapper around multiple fs.FS instances, recursively merging them together dynamically.

go-layerfs This is a simple wrapper around multiple fs.FS instances, recursively merging them together dynamically. If you have two directories, of wh

Dominik Schmidt 22 Aug 9, 2022
Command-line tool to organize large directories of media files recursively by date, detecting duplicates.

go-media-organizer Command-line tool written in Go to organise all media files in a directory recursively by date, detecting duplicates.

Allan Avelar 8 Jan 6, 2022
recursively list secrets from Vaults KV2 engine

vkv recursively list secrets from Vaults KV2 engine Installation Find the corresponding binaries, .rpm and .deb packages in the release section. Authe

null 23 Nov 24, 2022
Scans a file or folder recursively for jar files that may be vulnerable to Log4Shell

Velocity A Minecraft server proxy with unparalleled server support, scalability, and flexibility. Velocity is licensed under the GPLv3 license. Goals

PaperMC 1.2k Dec 3, 2022
run/stop goroutines/tasks securely, recursively

grunner - run/stop goroutines/tasks securely, recursively. s1 := grunner.New() s1.Defer(func() { fmt.Println("s1 stopped 2") }) s1.Defer(func() {

vogo 0 Apr 22, 2022
Ztgrep - Recursively search through compressed tarballs and files

ztgrep Search for file names and contents through nested archives. Useful for fi

Stephen Levine 3 Jun 7, 2022
Count total LoC in a given folder, recursively.

GoLoc A simple CLI tool to calculate the total LoC of a given directory. Usage goloc Flags Flag Description --use-gitignore Use your .gitignore file a

null 3 Apr 27, 2022