Encrypted overlay filesystem written in Go

Overview

gocryptfs Build Status MIT License Go Report Card Latest release Homebrew version

An encrypted overlay filesystem written in Go. Official website: https://nuetzlich.net/gocryptfs (markdown source).

Folders side-by-side animation

gocryptfs is built on top the excellent go-fuse FUSE library. This project was inspired by EncFS and strives to fix its security issues while providing good performance (benchmarks). For details on the security of gocryptfs see the Security design document.

All tags from v0.4 onward are signed by the gocryptfs signing key. Please check Signed Releases for details.

Current Status

gocryptfs has reached version 1.0 on July 17, 2016. It has gone through hours and hours of stress (fsstress, extractloop.bash) and correctness testing (xfstests). It is now considered ready for general consumption.

The old principle still applies: Important data should have a backup. Also, keep a copy of your master key (printed on mount) in a safe place. This allows you to access the data even if the gocryptfs.conf config file is damaged or you lose the password.

The security of gocryptfs has been audited in March 3, 2017. The audit is available here (defuse.ca).

Platforms

Linux is gocryptfs' native platform.

Beta-quality Mac OS X support is available, which means most things work fine but you may hit an occasional problem. Check out ticket #15 for the history of Mac OS X support but please create a new ticket if you hit a problem.

For Windows, an independent C++ reimplementation can be found here: cppcryptfs

A standalone Python tool that can decrypt files & file names is here: gocryptfs-inspect

Installation

Precompiled binaries that work on all x86_64 Linux systems are available for download from the github releases page.

On Debian, gocryptfs is available as a deb package:

apt install gocryptfs

On Mac OS X, gocryptfs is available as a Homebrew formula:

brew install gocryptfs

On Fedora, gocryptfs is available as an rpm package:

sudo dnf install gocryptfs

If you use the standalone binary, make sure you install the fuse package from your distributions package repository before running gocryptfs.

See the Quickstart page for more info.

Testing

gocryptfs comes with is own test suite that is constantly expanded as features are added. Run it using ./test.bash. It takes about 1 minute and requires FUSE as it mounts several test filesystems.

The stress_tests directory contains stress tests that run indefinitely.

In addition, I have ported xfstests to FUSE, the result is the fuse-xfstests project. gocryptfs passes the "generic" tests with one exception, results: XFSTESTS.md

A lot of work has gone into this. The testing has found bugs in gocryptfs as well as in the go-fuse library.

Compile

With go 1.11 or higher:

$ git clone https://github.com/rfjakob/gocryptfs.git
$ cd gocryptfs
$ ./build.bash

build.bash needs the OpenSSL headers installed (Debian: apt install libssl-dev, Fedora: dnf install openssl-devel). Alternatively, you can compile without OpenSSL using

$ ./build-without-openssl.bash

Use

$ mkdir cipher plain
$ ./gocryptfs -init cipher
$ ./gocryptfs cipher plain

See the Quickstart page for more info.

The MANPAGE.md describes all available command-line options.

Use: Reverse Mode

$ mkdir cipher plain
$ ./gocryptfs -reverse -init plain
$ ./gocryptfs -reverse plain cipher

Graphical Interface

The SiriKali project supports gocryptfs and runs on Linux and OSX.

cppcryptfs on Windows provides its own GUI.

Stable CLI ABI

If you want to call gocryptfs from your app or script, see CLI_ABI.md for the official stable ABI. This ABI is regression-tested by the test suite.

Storage Overhead

  • Empty files take 0 bytes on disk
  • 18 byte file header for non-empty files (2 bytes version, 16 bytes random file id)
  • 32 bytes of storage overhead per 4kB block (16 byte nonce, 16 bytes auth tag)

file-format.md contains a more detailed description.

Performance

Since version 0.7.2, gocryptfs is as fast as EncFS in the default mode, and significantly faster than EncFS' "paranoia" mode that provides a security level comparable to gocryptfs.

On CPUs without AES-NI, gocryptfs uses OpenSSL through a thin wrapper called stupidgcm. This provides a 4x speedup compared to Go's builtin AES-GCM implementation. See CPU-Benchmarks for details, or run gocryptfs -speed to see the encryption performance of your CPU. Example for a CPU without AES-NI:

$ ./gocryptfs -speed
AES-GCM-256-OpenSSL    165.67 MB/s  (selected in auto mode)
AES-GCM-256-Go          49.62 MB/s
AES-SIV-512-Go          39.98 MB/s

You can run ./benchmark.bash to run gocryptfs' canonical set of benchmarks that include streaming write, extracting a linux kernel tarball, recursively listing and finally deleting it. The output will look like this:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.DwL: gocryptfs v1.6; go-fuse v20170619-45-g95c6370; 2018-08-18 go1.10.3
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.1033 s, 238 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.945291 s, 277 MB/s
UNTAR: 17.768
MD5:   8.459
LS:    1.460
RM:    3.379

Changelog

vNEXT, in progress

  • MANPAGE: Split options into sections acc. to where they apply (#517)
  • -idle: count cwd inside the mount as busy (#533)
  • Make gocryptfs.diriv and gocryptfs.xxx.name files world-readable to make encrypted backups easier when mounting via /etc/fstab (#539)
  • Make it work with MacFUSE v4.x (#524)
  • Disable ACL encryption, it causes a lot of problems (#543, #536)
    • Old encrypted ACLs are reported by gocryptfs -fsck but otherwise ignored
    • This fixes inheritance, but does not yet enforce them correctly

v2.0-beta2, 2020-11-14

v2.0-beta1, 2020-10-15

  • Switch to the improved go-fuse v2 API
    • This is a big change, a lot of code has been reorganized or rewritten to fit the v2 API model.
    • Please test & report bugs
    • No changes to the on-disk format
    • File descriptor caching is not yet implemented, causing a slowdown. Caching will be implemented for v2.0 final.
  • Add support for FIDO2 tokens (-fido2, #505)
  • Add -encrypt-paths / -decrypt-paths functionality to gocryptfs-xray (#416)
  • Accept multiple -passfiles (#288)
  • Make -masterkey=stdin work together with -passwd (#461)
  • Fix Unknown opcode 2016 crash on Google Cloud (go-fuse #276, gocryptfs commit ec74d1d)

v1.8.0, 2020-05-09

  • Enable ACL support (#453)
    • Warning 2021-02-07: This feature is incomplete! Do not use ACLs before gocryptfs v2.0 final! Reading and writing ACLs works, but they are not enforced or inherited (#542)
  • Ignore .nfsXXX temporary files (#367)
  • Handle inode number collisions from multiple devices (#435)
  • Drop -nonempty for fusermount3 (#440)
  • Reverse mode: improve inode number mapping and max=1000000000000000000 limitation (#457)
  • Enable --buildmode=pie (#460)
  • Migrate from dep to Go Modules (commit cad711993)
  • go mod: update dependencies (commit b23f77c)
  • gocryptfs -speed: add XChaCha20-Poly1305-Go (#452)
  • Respect GOMAXPROCS environment variable (commit ff210a06f
  • Completely remove Trezor-related code (commit 1364b44ae356da31e24e5605fe73a307e9d6fb03)
    • Has been disabled since v1.7 due to issues a third-party module.
    • Please use FIDO2 instead (gocryptfs v2.0)

v1.7.1, 2019-10-06

  • Support wild cards in reverse mode via --exclude-wildcard (#367). Thanks @ekalin!
  • Create gocryptfs.diriv files with 0440 permissions to make it easier to share an encrypted folder via a network drive (#387). Note: as a security precaution, the owner must still manually chmod gocryptfs.conf 0440 to allow mounting.
  • Allow the nofail option in /etc/fstab
  • -passwd can now change the -scryptn parameter for existing filesystems (#400)
  • Fix -idle unmounting the filesystem despite recent activity (#421)
  • Fix a race condition related to inode number reuse (#363). It could be triggered by concurrently creating and deleting files and can lead to data loss in the affected file. This bug was found by the automated tests on Travis and was very hard to trigger locally.
  • tests: use /var/tmp instead of /tmp by default (commit 8c4429)

v1.7, 2019-03-17

  • Fix possible symlink race attacks in forward mode when using allow_other + plaintextnames
    • If you use both -allow_other and -plaintextnames, you should upgrade. Malicious users could trick gocryptfs into modifying files outside of CIPHERDIR, or reading files inside CIPHERDIR that they should not have access to.
    • If you do not use -plaintextnames (disabled per default), these attacks do not work as symlinks are encrypted.
    • Forward mode has been reworked to use the "*at" family of system calls everywhere (Openat/Unlinkat/Symlinkat/...).
    • As a result, gocryptfs may run slightly slower, as the caching logic has been replaced and is very simple at the moment.
    • The possibility for such attacks was found during an internal code review.
  • Reverse mode: fix excluded, unaccessible files showing up in directory listings (#285, #286)
  • gocryptfs-xray: add -aessiv flag for correctly parsing AES-SIV format files (#299)
  • Ensure that standard fds 0,1,2 are always initialized (#320). Prevents trouble in the unlikely case that gocryptfs is called with stdin,stdout and/or stderr closed.
  • -extpass now can be specified multiple times to support arguments containing spaces (#289)
  • Drop Fstatat, Mkdirat, Syslinkat, Fchownat, Unlinkat, Renameat, Openat emulation of MacOS and instead use native functions (thanks @slackner !)
  • Use Setreuid to robustly set the owner with allow_other (@slackner, (commit))
  • Pack the rendered man page into the source code archive for user convenience (issue 355)
  • Disable Trezor support again (commit 16fac26c57ba303bf60266d24c17f5243e5ea376)
    • Trezor support has been broken since Sept 2018 due to issues in a third-party module (#261)

v1.6.1, 2018-12-12

  • Fix "Operation not supported" chmod errors on Go 1.11 (#271)

v1.6, 2018-08-18

  • Add -e / -exclude option for reverse mode (#235, commit)
  • Add support for the Trezor One HSM PR#247, thanks @xaionaro!
    • Use ./build.bash -tags enable_trezor to compile with Trezor support
    • Then, use gocryptfs -init -trezor to create a filesystem locked with a physical Trezor device.
    • Note 2021-01-31: Support was removed again in gocryptfs v1.7. Please use -fido2 in gocryptfs v2.0.
  • Only print master key once, on init (#76, commit)
  • Fall back to buffered IO even when passed O_DIRECT (commit)

v1.5, 2018-06-12

  • Support extended attributes (xattr) in forward mode (#217). Older gocryptfs versions will ignore the extended attributes.
  • Add -fsck function (#191)
  • Fix clobbered timestamps on MacOS High Sierra (#229)
  • Add -masterkey=stdin functionality (#218)
  • Accept -dev/-nodev, suid/nosuid, -exec/-noexec, -ro/-rw flags to make mounting via /etc/fstab possible. Thanks @mahkoh! (#233, commit, commit)
  • Fix a logger path issue on SuSE #225
  • Stop printing the help text on a "flag provided but not defined" error (commit)

v1.4.4, 2018-03-18

  • Overwrite secrets in memory with zeros as soon as possible (#211)
  • Fix Getdents problems on i386 and mips64le (#197, #200)
  • Make building with gccgo work (#201)
  • MacOS: fix osxfuse: vnode changed generation / Error code -36 issue in go-fuse (#213, commit)
  • Fix various test issues on MacOS

v1.4.3, 2018-01-21

  • Fix several symlink race attacks in connection with reverse mode and allow_other. Thanks to @slackner for reporting and helping to fix the issues:
    • Fix symlink races in reverse mode (issue #165)
    • Fix symlink races in connection with -allow_other (issue #177)
  • Fix problems with special names when using -plaintextnames (issue #174)
  • Add -devrandom command-line option (commit)
  • Add -sharedstorage command-line option (commit, issue #156)
  • MacOS: let OSXFuse create the mountpoint if it does not exist (issue #194)

v1.4.2, 2017-11-01

  • Add Gopkg.toml file for dep vendoring and reproducible builds (issue #142)
  • MacOS: deal with .DS_Store files inside CIPHERDIR (issue #140)
  • Reverse mode: fix ENOENT error affecting names exactly 176 bytes long (issue #143)
  • Support kernels compiled with > 128 kiB FUSE request size (Synology NAS) (issue #145, commit)
  • Fix a startup hang when $PATH contains the mountpoint (issue #146)

v1.4.1, 2017-08-21

  • Use memory pools for buffer handling ( 3c6fe98, b2a23e9, 12c0101)
    • On my machine, this doubles the streaming read speed (see performance.txt)
  • Implement and use the getdents(2) syscall for a more efficient OpenDir implementation (e50a6a5)
  • Purge masterkey from memory as soon as possible (issue #137)
  • Reverse mode: fix inode number collision between .name and .diriv files (d12aa57)
  • Prevent the logger from holding stdout open (issue #130)
  • MacOS: make testing without openssl work properly (ccf1a84)
  • MacOS: specify a volume name (9f8e19b)
  • Enable writing to write-only files (issue #125)

v1.4, 2017-06-20

  • Switch to static binary releases
    • From gocryptfs v1.4, I will only release statically-built binaries. These support all Linux distributions but cannot use OpenSSL.
    • OpenSSL is still supported - just compile from source!
  • Add -force_owner option to allow files to be presented as owned by a different user or group from the user running gocryptfs. Please see caveats and guidance in the man page before using this functionality.
  • Increase open file limit to 4096 (#82).
  • Implement path decryption via ctlsock (#84). Previously, decryption was only implemented for reverse mode. Now both normal and reverse mode support both decryption and encryption of paths via ctlsock.
  • Add more specific exit codes for the most common failure modes, documented in CLI_ABI.md
  • Reverse mode: make sure hard-linked files always return the same ciphertext (commit 9ecf2d1a)
  • Display a shorter, friendlier help text by default.
  • Parallelize file content encryption by splitting data blocks into two threads (ticket#116)
  • Prefetch random nonces in the background (commit 80516ed)
  • Add -info option to pretty-print infos about a filesystem.

v1.3, 2017-04-29

  • Use HKDF to derive separate keys for GCM and EME
    • New feature flag: HKDF (enabled by default)
    • This is a forwards-compatible change. gocryptfs v1.3 can mount filesystems created by earlier versions but not the other way round.
  • Enable Raw64 filename encoding by default (gets rid of trailing == characters)
    • This is a forwards-compatible change. gocryptfs v1.3 can mount filesystems created by earlier versions but not the other way round.
  • Drop Go 1.4 compatibility. You now need Go 1.5 (released 2015-08-19) or higher to build gocryptfs.
  • Add -serialize_reads command-line option
    • This can greatly improve performance on storage that is very slow for concurrent out-of-order reads. Example: Amazon Cloud Drive (#92)
  • Reject file-header-only files (#90 2.2, commit)
  • Increase max password size to 2048 bytes (#93)
  • Use stable 64-bit inode numbers in reverse mode
    • This may cause problems for very old 32-bit applications that were compiled without Large File Support.
  • Passing "--" now also blocks "-o" parsing

v1.2.1, 2017-02-26

  • Add an integrated speed test, gocryptfs -speed
  • Limit password size to 1000 bytes and reject trailing garbage after the newline
  • Make the test suite work on Mac OS X
  • Handle additional corner cases in -ctlsock path sanitization
  • Use dedicated exit code 12 on "password incorrect"

v1.2, 2016-12-04

  • Add a control socket interface. Allows to encrypt and decrypt filenames. For details see backintime#644.
    • New command-line option: -ctlsock
  • Under certain circumstances, concurrent truncate and read could return an I/O error. This is fixed by introducing a global open file table that stores the file IDs (commit).
  • Coalesce 4kB ciphertext block writes up to the size requested through the write FUSE call (commit with benchmarks)
  • Add -noprealloc command-line option
    • Greatly speeds up writes on Btrfs (#63) at the cost of reduced out-of-space robustness.
    • This is a workaround for Btrfs' slow fallocate(2)
  • Preserve owner for symlinks an device files (fixes bug #64)
  • Include rendered man page gocryptfs.1 in the release tarball

v1.1.1, 2016-10-30

  • Fix a panic on setting file timestamps (go-fuse#131)
  • Work around an issue in tmpfs that caused a panic in xfstests generic/075 (gocryptfs#56)
  • Optimize NFS streaming writes (commit)

v1.1, 2016-10-19

  • Add reverse mode (#19)
    • AES-SIV (RFC5297) encryption to implement deterministic encryption securely. Uses the excellent jacobsa/crypto library. The corresponding feature flag is called AESSIV.
    • New command-line options: -reverse, -aessiv
    • Filesystems using reverse mode can only be mounted with gocryptfs v1.1 and later.
    • The default, forward mode, stays fully compatible with older versions. Forward mode will keep using GCM because it is much faster.
  • Accept -o foo,bar,baz-style options that are passed at the end of the command-line, like mount(1) does. All other options must still precede the passed paths.
    • This allows mounting from /etc/fstab. See #45 for details.
    • Mounting on login using pam_mount works as well. It is described in the wiki.
  • To prevent confusion, the old -o option had to be renamed. It is now called -ko. Arguments to -ko are passed directly to the kernel.
  • New -passfile command-line option. Provides an easier way to read the password from a file. Internally, this is equivalent to -extpass "/bin/cat FILE".
  • Enable changing the password when you only know the master key (#28)

v1.0, 2016-07-17

  • Deprecate very old filesystems, stage 3/3
    • Filesystems created by v0.6 can no longer be mounted
    • Drop command-line options -gcmiv128, -emenames, -diriv. These are now always enabled.
  • Add fallocate(2) support
  • New command-line option -o
    • Allows to pass mount options directly to the kernel
  • Add support for device files and suid binaries
    • Only works when running as root
    • Must be explicitly enabled by passing "-o dev" or "-o suid" or "-o suid,dev"
  • Experimental Mac OS X support. See ticket #15 for details.

v0.12, 2016-06-19

  • Deprecate very old filesystems, stage 2/3
    • Filesystems created by v0.6 and older can only be mounted read-only
    • A message explaining the situation is printed as well
  • New command line option: -ro
    • Mounts the filesystem read-only
  • Accept password from stdin as well (ticket #30)

v0.11, 2016-06-10

  • Deprecate very old filesystems, stage 1/3
    • Filesystems created by v0.6 and older can still be mounted but a warning is printed
    • See ticket #29 for details and join the discussion
  • Add rsync stress test "pingpong-rsync.bash"
    • Fix chown and utimens failures that caused rsync to complain
  • Build release binaries with Go 1.6.2
    • Big speedup for CPUs with AES-NI, see ticket #23

v0.10, 2016-05-30

  • Replace spacemonkeygo/openssl with stupidgcm
    • gocryptfs now has its own thin wrapper to OpenSSL's GCM implementation called stupidgcm.
    • This should fix the compile issues people are seeing with spacemonkeygo/openssl. It also gets us a 20% performance boost for streaming writes.
  • Automatically choose between OpenSSL and Go crypto issue #23
    • Go 1.6 added an optimized GCM implementation in amd64 assembly that uses AES-NI. This is faster than OpenSSL and is used if available. In all other cases OpenSSL is much faster and is used instead.
    • -openssl=auto is the new default
    • Passing -openssl=true/false overrides the autodetection.
  • Warn but continue anyway if fallocate(2) is not supported by the underlying filesystem, see issue #22
    • Enables to use gocryptfs on ZFS and ext3, albeit with reduced out-of-space safety.
  • Fix statfs, by @lxp
  • Fix a fsstress failure in the go-fuse library.

v0.9, 2016-04-10

  • Long file name support
    • gocryptfs now supports file names up to 255 characters.
    • This is a forwards-compatible change. gocryptfs v0.9 can mount filesystems created by earlier versions but not the other way round.
  • Refactor gocryptfs into multiple "internal" packages
  • New command-line options:
    • -longnames: Enable long file name support (default true)
    • -nosyslog: Print messages to stdout and stderr instead of syslog (default false)
    • -wpanic: Make warning messages fatal (used for testing)
    • -d: Alias for -debug
    • -q: Alias for -quiet

v0.8, 2016-01-23

  • Redirect output to syslog when running in the background
  • New command-line option:
    • -memprofile: Write a memory allocation debugging profile the specified file

v0.7.2, 2016-01-19

  • Fix performance issue in small file creation
    • This brings performance on-par with EncFS paranoia mode, with streaming writes significantly faster
    • The actual fix is in the go-fuse library. There are no code changes in gocryptfs.

v0.7.1, 2016-01-09

  • Make the build.bash script compatible with Go 1.3
  • Disable fallocate on OSX (system call not available)
  • Introduce pre-built binaries for Fedora 23 and Debian 8

v0.7, 2015-12-20

  • Extend GCM IV size to 128 bit from Go's default of 96 bit
    • This pushes back the birthday bound to make IV collisions virtually impossible
    • This is a forwards-compatible change. gocryptfs v0.7 can mount filesystems created by earlier versions but not the other way round.
  • New command-line option:
    • -gcmiv128: Use 128-bit GCM IVs (default true)

v0.6, 2015-12-08

  • Wide-block filename encryption using EME + DirIV
    • EME (ECB-Mix-ECB) provides even better security than CBC as it fixes the prefix leak. The used Go EME implementation is https://github.com/rfjakob/eme which is, as far as I know, the first implementation of EME in Go.
    • This is a forwards-compatible change. gocryptfs v0.6 can mount filesystems created by earlier versions but not the other way round.
  • New command-line option:
    • -emenames: Enable EME filename encryption (default true)

v0.5.1, 2015-12-06

  • Fix a rename regression caused by DirIV and add test case
  • Use fallocate to guard against out-of-space errors

v0.5, 2015-12-04

  • Stronger filename encryption: DirIV
    • Each directory gets a random 128 bit file name IV on creation, stored in gocryptfs.diriv
    • This makes it impossible to identify identically-named files across directories
    • A single-entry IV cache brings the performance cost of DirIV close to zero for common operations (see performance.txt)
    • This is a forwards-compatible change. gocryptfs v0.5 can mount filesystems created by earlier versions but not the other way round.
  • New command-line option:
    • -diriv: Use the new per-directory IV file name encryption (default true)
    • -scryptn: allows to set the scrypt cost parameter N. This option can be used for faster mounting at the cost of lower brute-force resistance. It was mainly added to speed up the automated tests.

v0.4, 2015-11-15

  • New command-line options:
    • -plaintextnames: disables filename encryption, added on user request
    • -extpass: calls an external program for prompting for the password
    • -config: allows to specify a custom gocryptfs.conf path
  • Add FeatureFlags gocryptfs.conf parameter
    • This is a config format change, hence the on-disk format is incremented
    • Used for ext4-style filesystem feature flags. This should help avoid future format changes. The first user is -plaintextnames.
  • On-disk format 2

v0.3, 2015-11-01

  • Add a random 128 bit file header to authenticate file->block ownership
    • This is an on-disk-format change
  • On-disk format 1

v0.2, 2015-10-11

  • Replace bash daemonization wrapper with native Go implementation
  • Better user feedback on mount failures

v0.1, 2015-10-07

  • First release
  • On-disk format 0
Issues
  • Mac OS X support

    Mac OS X support

    Go support Mac OS X, as does the FUSE library we use, go-fuse.

    gocrypts may actually work out of the box on OSX, but there probably are small issues that have to be sorted out.

    At the very least, gocryptfs has to be tested on OSX. As I do not have a Mac to test on, this would be an opportunity for somebody from the Mac community to step up. Please comment here if you are interested.

    feature request help wanted 
    opened by rfjakob 121
  • Feature Request: encryption primitives for devices without AES cpu instructions

    Feature Request: encryption primitives for devices without AES cpu instructions

    Hi @rfjakob,

    Thank you for this great application! The reverse mode is what really sets it apart from other options.

    I checked the issues, and it doesn't seem to be discussed yet, but what do you think about adding support for a different collection of encryption primitives that are better suited for more low-end devices?

    I'm running gocryptfs on a few ARMv6/7 based NAS machines, they are nice: low energy, and quite fast. But they lack native AES instructions, my fastest ARM device (Odroid XU4) maxes out at 40MB/s, while for example the raspberry-pi's and friends are quite a bit slower (rpi1 is at 15MB/s).

    Maybe Google Adiantum (also added to linux kernel 5.0 for cryptfs) is a nice fit, Adiantum is based on XChaCha12 and Poly1305 and is roughly 5 quicker than AES-XTS for devices without AES instructions.

    For the reverse mode maybe something based on ChaCha20Poly1305?

    Just for comparison, on my Odroid XU4, ChaCha20Poly1305 runs at 320MB/s, on my RPi1 it gets close to 40MB/s.

    So I'm just wondering what your view is on this topic.

    Cheers, Davy

    feature request maybe some day ⌛ 
    opened by DavyLandman 79
  • any libpam-gocryptfs

    any libpam-gocryptfs

    Is there any pam module to automount after login just like libpam-encfs ?

    bug 
    opened by jsalatiel 59
  • xfstests generic/273 failure

    xfstests generic/273 failure

    Needs further analysis:

    _porter 28 not complete
    cp: cannot create regular file '/var/tmp/check-gocryptfs/scratchdir/sub_28/origin/file_548': No such file or directory
    

    https://github.com/rfjakob/gocryptfs/blob/master/Documentation/XFSTESTS.md#generic273

    bug bug outside gocryptfs 
    opened by rfjakob 36
  • macOS (Error code -36) while copying

    macOS (Error code -36) while copying

    Hey there,

    I am getting this (title) message very often while copying data. In the System Console I see:

    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    

    Although this seems to occur sometimes without the error in Finder. When I copy via cpthen some files do not get copied and cp prints Stale NFS file handle

    The debug output of gocrypt itself does not show any errors. I am using version gocryptfs v1.4.3; go-fuse [vendored]; 2018-02-02 go1.9.3 (Homebrew) and OSX Fuse version 3.7.1.

    Thanks a lot, Magnus

    bug 
    opened by magnusja 36
  • macOS: modification and creation dates reset

    macOS: modification and creation dates reset

    I started using gocryptfs (latest version) on macOS High Sierra and noticed that each time I eject my encrypted volume and remount it, that the creation and modification dates are reset to 1970-01-01. Is this expected behaviour?

    I mount my dirs using

    gocryptfs -ko local /path/to/encDir /Volumes/encDir
    

    Also mounting them without the -ko local option has the same result.

    bug 
    opened by alexanderharm 35
  • Feature Request: Reverse Mode

    Feature Request: Reverse Mode

    Add an option similar to the --reverse option for encfs:

    Normally EncFS provides a plaintext view of data on demand. Normally it stores enciphered data and displays plaintext data. With --reverse it takes as source plaintext data and produces enciphered data on-demand. This can be useful for creating remote encrypted backups, where you do not wish to keep the local files unencrypted.

    For example, the following would create an encrypted view in /tmp/crypt-view.

    encfs --reverse /home/me /tmp/crypt-view You could then copy the /tmp/crypt-view directory in order to have a copy of the encrypted data. You must also keep a copy of the file /home/me/.encfs5 which contains the filesystem information. Together, the two can be used to reproduce the unencrypted data: ENCFS5_CONFIG=/home/me/.encfs5 encfs /tmp/crypt-view /tmp/plain-view Now /tmp/plain-view contains the same data as /home/me Note that --reverse mode only works with limited configuration options, so many settings may be disabled when used.

    (Quote from encfs man page)

    feature request 
    opened by usr42 30
  • Poor read performance with network backend

    Poor read performance with network backend

    Use case

    • /tmp/encrypted is an Amazon Cloud Drive folder, mounted read-only using rclone
    • /tmp/clear is the gocryptfs deciphered version of the above.

    Copying a 300MB file (ciphered) from /tmp/encrypted, I get around 5-6MBps, which is OK, considering my bandwidth is 10MBps. Copying the same file from /tmp/clear yields extremely poor speeds (from 100KBps to 1MBps). Speed is not limited by the CPU. Tracing read requests in rclone shows read sizes from 4KB to 128KB occuring at low frequency (thus low througput)

    Any advice would be greatly appreciated.

    Environment

    • Debian Stretch, x86_64
    • gocryptfs 1.2.1
    • Ciphered volume has the following features:
      • version: 2
      • flags: GCMIV128, DirIV, EMENames, LongNames, AESSIV
    • rclone 1.35, mount max-read-ahead set to 16MB
    opened by j-vizcaino 28
  • gocryptfs for Windows - Cross-platform support

    gocryptfs for Windows - Cross-platform support

    Because it's Go, could it be made to run on Windows as well? (Cross-platform support is an attractive feature.)

    The problem with windows is that it does not have FUSE support. Linux has it built-in and Mac has the OSXFuse project. On windows, the only thing i am aware of is http://encfsmp.sourceforge.net/ . They SOMEHOW managed to get encfs (which also uses FUSE) running on windows, using a closed-source windows kernel driver and lots of black magic.

    See discussion here https://github.com/rfjakob/gocryptfs/issues/2#issuecomment-152955769 and here https://github.com/rfjakob/gocryptfs/issues/2#issuecomment-152962990

    feature request help wanted 
    opened by dakkusingh 27
  • get output from ctags in mountpoint: cannot open path of the current working directory: Permission denied

    get output from ctags in mountpoint: cannot open path of the current working directory: Permission denied

    I used gocryptfs to mount a directory,and then I tried to execute "ctags -R --languages=Verilog rtl", I got "cannot open path of the current working directory: Permission denied" error, but actually I can write or read any files in this directory.

    needs more info 
    opened by lizhirui 6
  • Multiple test failures on MacOS 12 (Monterey) Apple Silicon M1 Max

    Multiple test failures on MacOS 12 (Monterey) Apple Silicon M1 Max

    Opening a new issue since #623 has been already closed. The commit a48d6c3 fixed the tests breakage, so that the tests can run. But unfortunately, quite a few of them still fail. Moreover, the test suite leaves behind several dozens of test directories mounted as gocryptfs file systems. I had to reboot the machine to cleanup the mess. Here are the details.

    Setup

    • MacBook Pro 16" 2021 M1 Max (Apple Silicon)
    • MacOS 12.0.1 Monterey (latest)
    • Xcode-13.1 (latest)
    • go-1.17.3 (latest)
    • macFUSE-4.2.3 (latest)

    Build

    Compiling latest master (hash=a48d6c3...) with

    $ ./build-without-openssl.bash
    

    completes OK and without errors.

    Test

    $ ./test-without-openssl.bash
    

    results in multiple test failures. Here is the output:
    test.log

    opened by slonik-az 9
  • rclone vfs - cache cleanup leads to error

    rclone vfs - cache cleanup leads to error "Device not configured (os error 6)"

    Thanks for the great software!

    I am using rclone to access my files stored in a public cloud and gocryptfs to encrypt everything. To not download all files but have something like a smart selective sync I use rclone mount - https://rclone.org/commands/rclone_mount/

    This works fine especially using VFS file caching of rclone - https://rclone.org/commands/rclone_mount/#vfs-file-caching

    I use the following options to mount the directories:

    rclone --vfs-cache-mode full --allow-other

    gocryptfs -ko modules=iconv,from_code=UTF-8,to_code=UTF-8-MAC

    Nevertheless after some time the mounted directory of gocryptfs is not accessible anymore and gives an error Device not configured (os error 6)

    If I unmount the gocryptfs directory and mount it again with gocryptfs everything is working again. So the rclone mount seems to work fine.

    Not sure what I am doing wrong or where to find more specific logs. The only hint I found was that some files were removed from cache automatically by rclone. Does this have an effect on gocryptfs?

    Environment:

    $ gocryptfs --version                    
    gocryptfs v2.2.1-6-gd530fbd without_openssl; 
    go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 
    2021-11-02 go1.17.2 darwin/arm64
    
    $ rclone --version
    rclone v1.57.0
    - os/version: darwin 12.0.1 (64 bit)
    - os/kernel: 21.1.0 (arm64)
    - os/type: darwin
    - os/arch: arm64
    - go/version: go1.17.2
    - go/linking: dynamic
    - go/tags: cmount
    
    macOS 
    opened by rustikus 2
  • I/O errors and freezes when using encrypted directories residing on SMB3 or WebDAV (davfs2) mounts

    I/O errors and freezes when using encrypted directories residing on SMB3 or WebDAV (davfs2) mounts

    When using a gocryptfs encrypted directory on a remote drive, mounted using either SMB3 or WebDAV (via davfs2), some operations lead to errors (davfs2) or infinite blocking (SMB3).

    To reproduce:

    test.py

    #!/usr/bin/python
    import os
    import shutil
    base_path = '/remote-plain'
    path = os.path.join(base_path, 'rdiff-backup.tmp.0/high_perms_dir')
    os.makedirs(path)
    shutil.rmtree(path)
    shutil.rmtree(base_path)
    
    1. Have some remote storage (i can share credentials to the one i'm using)
    2. Mount it locally using davfs2 or SMB3 (e.g. at /remote-mount)
    3. Mount the gocryptfs encrypted directory locally (e.g. gocryptfs /remote-mount /remote-plain)
    4. Execute test.py with according paths, it will yield an I/O error or infinitely hang

    Executing the script on /remote-mount instead of /remote-plain works just fine.

    More detailed information on when it fails: https://github.com/rdiff-backup/rdiff-backup/issues/641

    opened by nightbluepn 8
  • Discrepancy between speed test and real-world speeds when running locally

    Discrepancy between speed test and real-world speeds when running locally

    I'm using gocryptfs v2.2.1 without_openssl, and the gocryptfs -speed results are:

    gocryptfs v2.2.1 without_openssl; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-10-24 go1.17.2 darwin/amd64
    cpu: unknown; with AES acceleration
    AES-GCM-256-OpenSSL       	    N/A
    AES-GCM-256-Go            	3895.94 MB/s	(selected in auto mode)
    AES-SIV-512-Go            	 325.05 MB/s
    XChaCha20-Poly1305-OpenSSL	    N/A
    XChaCha20-Poly1305-Go     	1856.94 MB/s	(selected in auto mode)
    

    Diskmark speed tests report that my Macbook Pro SSD can perform sequential reads at 3.4GB/s and sequential writes at 2.1GB/s.

    When I do cp random10G.bin random10G.bin.2, that completes at a speed of 1064 MB/s (this is just on the disk directly with gocryptfs not involved at all).

    However, when I create local gocryptfs cipher and plain directories, I only get a write speed of 322 MB/s and a read speed of 312 MB/s to/from the gocryptfs plain directory.

    I checked that AES-GCM-256 is being used:

    gocryptfs -info cipher
    Creator:           gocryptfs v2.2.1
    FeatureFlags:      HKDF GCMIV128 DirIV EMENames LongNames Raw64
    EncryptedKey:      64B
    ScryptObject:      Salt=32B N=65536 R=8 P=1 KeyLen=32
    contentEncryption: AES-GCM-256
    

    Is it normal for gocryptfs to operate at a third of the speed of the underlying disk? Might there be any buffer size changes that could improve performance?

    opened by knaccc 0
  • Performance flags for use over sshfs (Mac)

    Performance flags for use over sshfs (Mac)

    I'm mounting a cipher directory over a 1GbE wired LAN via sshfs, and then mounting that via gocryptfs:

    sudo mkdir /Volumes/cipher
    sudo chown me:staff /Volumes/cipher
    sshfs [email protected]:/cipher /Volumes/cipher -o \
    reconnect,ServerAliveInterval=15,ServerAliveCountMax=3,allow_other,noapplexattr,noappledouble,\
    defer_permissions,volname=displayName,IdentityFile=/Users/me/.ssh/id_ed25519
    sudo mkdir /Volumes/plain
    sudo chown me:staff /Volumes/plain
    gocryptfs /Volumes/cipher /Volumes/plain
    

    If I copy a 10GB file to the /Volumes/cipher directory (i.e. not via gocryptfs), it copies at 76.4 MB/s. If I copy a 10GB file to the /Volumes/plain directory (i.e. via gocryptfs), it copies at 33.8 MB/s.

    gocryptfs -speed reports AES-GCM-256-Go 3963.06 MB/s (using gocryptfs v2.2.1 without_openssl)

    Can anyone recommend any sshfs or other flags that might help increase performance please?

    feature request performance 
    opened by knaccc 3
  • Support for multiple (independently revocable) keys for the same encrypted data

    Support for multiple (independently revocable) keys for the same encrypted data

    Hi, I'm coming here from the feature comparison table on https://wiki.archlinux.org/title/Data-at-rest_encryption#Comparison_table and would like to ask about the "Support for multiple (independently revocable) keys for the same encrypted data".

    Does gocryptfs support it? If so, what are the downsides (if any) of the implementation? If not, would it be possible to implement it?

    feature request question 
    opened by dumblob 2
  • Are inode unique and persistent in reverse mode ?

    Are inode unique and persistent in reverse mode ?

    Hi, I would like to use gocryptfs in reverse mode for cloud backup and I would like to know if the inodes number are unique and persistent in the encrypted view? I would like to use "Smart rsync backup" (https://github.com/wapsi/smart-rsync-backup) which list files moved and renamed since the last time and this tools only works if inodes are unique and persistent. I searched in the documentation but did find the answer. Thank you for your help :)

    documentation 
    opened by hpn789 20
  • macOS, -reverse mode, fs.Mount failed: read-only file system

    macOS, -reverse mode, fs.Mount failed: read-only file system

    Hi, thanks for your work. The normal mode of gocryptfs works fine in my macOS, but the reverse mode doesn't work:

    fs.Mount failed: read-only file system
    Maybe you should run: /Library/Filesystems/osxfuse.fs/Contents/Resources/load_osxfuse
    

    I don't have /Library/Filesystems/osxfuse.fs/Contents/Resources/load_osxfuse, but I tried /Library/Filesystems/macfuse.fs/Contents/Resources/load_macfuse, which had no output and gocryptfs still failed with the same error.

    My macOS is 10.14.5, and I compiled gocryptfs by:

    Install Macfuse
    Install go
    Download / Clone this repository
    Run bash ./build-without-openssl.bash
    

    I tried gocryptfs -d -fusedebug -fg -reverse plain cipher, and the last few output:

    23:57:24.475219 rx 3: ACCESS n1 {u=501 g=20 x}
    openBackingDir "" -> 10 "" <nil>
    23:57:24.475374 tx 3:     OK
    fs.Mount failed: read-only file system
    

    I appreciate your help.

    macOS 
    opened by extr15 2
  • Root directory inode number is missing

    Root directory inode number is missing

    The root directory inode number has gone missing in recent gocryptfs builds. I ran into this while hacking on https://github.com/hanwen/go-fuse/issues/399 -- more details there. I'm running into this myself with a toy filesystem, but still can't tell yet whether this is something we're both doing wrong with the go-fuse v2 fs API, or if it's a bug in go-fuse.

    Here's the latest binary release:

    $ /tmp/gocryptfs_v2.0.1_linux-static_amd64/gocryptfs -version
    gocryptfs v2.0.1 without_openssl; go-fuse v2.1.1-0.20210508151621-62c5aa1919a7; 2021-06-07 go1.13.15 linux/amd64
    
    $ /tmp/gocryptfs_v2.0.1_linux-static_amd64/gocryptfs /tmp/cipher /tmp/plain
    Password: 
    Decrypting master key
    Filesystem mounted and ready.
    
    $ ls -lai /tmp/plain
    total 244
            ? drwxr-xr-x   3 stevegt stevegt   4096 Jun 23 20:04 .
    114556929 drwxrwxrwt 345 root    root    237568 Jul  5 15:17 ..
    115353656 -rw-r--r--   1 stevegt stevegt      0 Jun 23 20:03 asfd
    118631539 drwxr-xr-x   2 stevegt stevegt   4096 Jun 23 20:04 oiewurj
    

    The inode number was there in this older version from an Ubuntu package:

    $ /usr/bin/gocryptfs -version
    gocryptfs 1.7.1; go-fuse 0.0~git20190214.58dcd77; 2019-12-26 go1.13.5 linux/amd64
    
    $ /usr/bin/gocryptfs /tmp/cipher /tmp/plain
    Password: 
    Decrypting master key
    Filesystem mounted and ready.
    
    $ ls -lai /tmp/plain
    total 244
    115353655 drwxr-xr-x   3 stevegt stevegt   4096 Jun 23 20:04 .
    114556929 drwxrwxrwt 345 root    root    237568 Jul  5 15:20 ..
    115353656 -rw-r--r--   1 stevegt stevegt      0 Jun 23 20:03 asfd
    118631539 drwxr-xr-x   2 stevegt stevegt   4096 Jun 23 20:04 oiewurj
    
    bug bug outside gocryptfs 
    opened by stevegt 6
Releases(v2.2.1)
A FileSystem Abstraction System for Go

A FileSystem Abstraction System for Go Overview Afero is a filesystem framework providing a simple, uniform and universal API interacting with any fil

Steve Francia 4.2k Jan 17, 2022
A package to allow one to concurrently go through a filesystem with ease

skywalker Skywalker is a package to allow one to concurrently go through a filesystem with ease. Features Concurrency BlackList filtering WhiteList fi

Will Dixon 68 Jan 16, 2022
An implementation of the FileSystem interface for tar files.

TarFS A wrapper around tar.Reader. Implements the FileSystem interface for tar files. Adds an Open method, that enables reading of file according to i

Eyal Posener 48 Nov 18, 2021
Takes an input http.FileSystem (likely at go generate time) and generates Go code that statically implements it.

vfsgen Package vfsgen takes an http.FileSystem (likely at go generate time) and generates Go code that statically implements the provided http.FileSys

null 936 Jan 12, 2022
memfs: A simple in-memory io/fs.FS filesystem

memfs: A simple in-memory io/fs.FS filesystem memfs is an in-memory implementation of Go's io/fs.FS interface. The goal is to make it easy and quick t

Peter Sanford 51 Jan 13, 2022
A Go io/fs filesystem implementation for reading files in a Github gists.

GistFS GistFS is an io/fs implementation that enables to read files stored in a given Gist. Requirements This module depends on io/fs which is only av

Jean Hadrien Chabran 122 Dec 6, 2021
A Small Virtual Filesystem in Go

This is a virtual filesystem I'm coding to teach myself Go in a fun way. I'm documenting it with a collection of Medium posts that you can find here.

Alyson 28 Jan 12, 2022
CRFS: Container Registry Filesystem

CRFS: Container Registry Filesystem Discussion: https://github.com/golang/go/issues/30829 Overview CRFS is a read-only FUSE filesystem that lets you m

Google 1.2k Jan 4, 2022
Go filesystem implementations for various URL schemes

hairyhenderson/go-fsimpl This module contains a collection of Go filesystem implementations that can discovered dynamically by URL scheme. All filesys

Dave Henderson 219 Jan 14, 2022
A Go filesystem package for working with files and directories

Stowage A Go filesystem package for working with files and directories, it features a simple API with support for the common files and directories ope

null 19 May 28, 2021
filesystem for golang

filesystem filesystem for golang installation go get github.com/go-component/filesystem import import "github.com/go-component/filesystem" Usage sup

null 4 Jul 9, 2021
A set of io/fs filesystem abstractions and utilities for Go

A set of io/fs filesystem abstractions and utilities for Go Please ⭐ this project Overview This package provides io/fs interfaces for: Cloud providers

null 7 Oct 7, 2021
Tarserv serves streaming tar files from filesystem snapshots.

tarserv A collection of tools that allow serving large datasets from local filesystem snapshots. It is meant for serving big amounts of data to shell

Aurora 1 Jan 11, 2022
Encrypted File System in Go

Getting Started: Setup the environment: Install GoLang: $ sudo apt update $ sudo apt upgrade $ sudo apt install libssl-dev gcc pkg-config $ sudo apt

Lucky Verma 0 Nov 26, 2021
A PDF processor written in Go.

pdfcpu: a Go PDF processor pdfcpu is a PDF processing library written in Go supporting encryption. It provides both an API and a CLI. Supported are al

pdfcpu 2.9k Jan 14, 2022
Goful is a CUI file manager written in Go.

Goful Goful is a CUI file manager written in Go. Works on cross-platform such as gnome-terminal and cmd.exe. Displays multiple windows and workspaces.

anmitsu 251 Jan 20, 2022
A simple library for generating PDF written in Go lang

gopdf gopdf is a simple library for generating PDF document written in Go lang. Features Unicode subfont embedding. (Chinese, Japanese, Korean, etc.)

Signin Technology 1.5k Jan 13, 2022
goelftools is library written in Go for parsing ELF file.

goelftools goelftools is library written in Go for parsing ELF file. This library is inspired by pyelftools and rbelftools. Motivation The motivation

null 23 Oct 11, 2021
File uploader with support for multiple hosts and progress reporting written in Go.

go-upload File uploader with support for multiple hosts and progress reporting written in Go. Windows, Linux, macOS and Android binaries Usage Upload

null 7 Jan 3, 2022