Encrypted overlay filesystem written in Go

Overview

gocryptfs Build Status MIT License Go Report Card Latest release Homebrew version

An encrypted overlay filesystem written in Go. Official website: https://nuetzlich.net/gocryptfs (markdown source).

Folders side-by-side animation

gocryptfs is built on top the excellent go-fuse FUSE library. This project was inspired by EncFS and strives to fix its security issues while providing good performance (benchmarks). For details on the security of gocryptfs see the Security design document.

All tags from v0.4 onward are signed by the gocryptfs signing key. Please check Signed Releases for details.

Current Status

gocryptfs has reached version 1.0 on July 17, 2016. It has gone through hours and hours of stress (fsstress, extractloop.bash) and correctness testing (xfstests). It is now considered ready for general consumption.

The old principle still applies: Important data should have a backup. Also, keep a copy of your master key (printed on mount) in a safe place. This allows you to access the data even if the gocryptfs.conf config file is damaged or you lose the password.

The security of gocryptfs has been audited in March 3, 2017. The audit is available here (defuse.ca).

Platforms

Linux is gocryptfs' native platform.

Beta-quality Mac OS X support is available, which means most things work fine but you may hit an occasional problem. Check out ticket #15 for the history of Mac OS X support but please create a new ticket if you hit a problem.

For Windows, an independent C++ reimplementation can be found here: cppcryptfs

A standalone Python tool that can decrypt files & file names is here: gocryptfs-inspect

Installation

Precompiled binaries that work on all x86_64 Linux systems are available for download from the github releases page.

On Debian, gocryptfs is available as a deb package:

apt install gocryptfs

On Mac OS X, gocryptfs is available as a Homebrew formula:

brew install gocryptfs

On Fedora, gocryptfs is available as an rpm package:

sudo dnf install gocryptfs

If you use the standalone binary, make sure you install the fuse package from your distributions package repository before running gocryptfs.

See the Quickstart page for more info.

Testing

gocryptfs comes with is own test suite that is constantly expanded as features are added. Run it using ./test.bash. It takes about 1 minute and requires FUSE as it mounts several test filesystems.

The stress_tests directory contains stress tests that run indefinitely.

In addition, I have ported xfstests to FUSE, the result is the fuse-xfstests project. gocryptfs passes the "generic" tests with one exception, results: XFSTESTS.md

A lot of work has gone into this. The testing has found bugs in gocryptfs as well as in the go-fuse library.

Compile

With go 1.11 or higher:

$ git clone https://github.com/rfjakob/gocryptfs.git
$ cd gocryptfs
$ ./build.bash

build.bash needs the OpenSSL headers installed (Debian: apt install libssl-dev, Fedora: dnf install openssl-devel). Alternatively, you can compile without OpenSSL using

$ ./build-without-openssl.bash

Use

$ mkdir cipher plain
$ ./gocryptfs -init cipher
$ ./gocryptfs cipher plain

See the Quickstart page for more info.

The MANPAGE.md describes all available command-line options.

Use: Reverse Mode

$ mkdir cipher plain
$ ./gocryptfs -reverse -init plain
$ ./gocryptfs -reverse plain cipher

Graphical Interface

The SiriKali project supports gocryptfs and runs on Linux and OSX.

cppcryptfs on Windows provides its own GUI.

Stable CLI ABI

If you want to call gocryptfs from your app or script, see CLI_ABI.md for the official stable ABI. This ABI is regression-tested by the test suite.

Storage Overhead

  • Empty files take 0 bytes on disk
  • 18 byte file header for non-empty files (2 bytes version, 16 bytes random file id)
  • 32 bytes of storage overhead per 4kB block (16 byte nonce, 16 bytes auth tag)

file-format.md contains a more detailed description.

Performance

Since version 0.7.2, gocryptfs is as fast as EncFS in the default mode, and significantly faster than EncFS' "paranoia" mode that provides a security level comparable to gocryptfs.

On CPUs without AES-NI, gocryptfs uses OpenSSL through a thin wrapper called stupidgcm. This provides a 4x speedup compared to Go's builtin AES-GCM implementation. See CPU-Benchmarks for details, or run gocryptfs -speed to see the encryption performance of your CPU. Example for a CPU without AES-NI:

$ ./gocryptfs -speed
AES-GCM-256-OpenSSL    165.67 MB/s  (selected in auto mode)
AES-GCM-256-Go          49.62 MB/s
AES-SIV-512-Go          39.98 MB/s

You can run ./benchmark.bash to run gocryptfs' canonical set of benchmarks that include streaming write, extracting a linux kernel tarball, recursively listing and finally deleting it. The output will look like this:

$ ./benchmark.bash
Testing gocryptfs at /tmp/benchmark.bash.DwL: gocryptfs v1.6; go-fuse v20170619-45-g95c6370; 2018-08-18 go1.10.3
WRITE: 262144000 bytes (262 MB, 250 MiB) copied, 1.1033 s, 238 MB/s
READ:  262144000 bytes (262 MB, 250 MiB) copied, 0.945291 s, 277 MB/s
UNTAR: 17.768
MD5:   8.459
LS:    1.460
RM:    3.379

Changelog

vNEXT, in progress

  • MANPAGE: Split options into sections acc. to where they apply (#517)
  • -idle: count cwd inside the mount as busy (#533)
  • Make gocryptfs.diriv and gocryptfs.xxx.name files world-readable to make encrypted backups easier when mounting via /etc/fstab (#539)
  • Make it work with MacFUSE v4.x (#524)
  • Disable ACL encryption, it causes a lot of problems (#543, #536)
    • Old encrypted ACLs are reported by gocryptfs -fsck but otherwise ignored
    • This fixes inheritance, but does not yet enforce them correctly

v2.0-beta2, 2020-11-14

v2.0-beta1, 2020-10-15

  • Switch to the improved go-fuse v2 API
    • This is a big change, a lot of code has been reorganized or rewritten to fit the v2 API model.
    • Please test & report bugs
    • No changes to the on-disk format
    • File descriptor caching is not yet implemented, causing a slowdown. Caching will be implemented for v2.0 final.
  • Add support for FIDO2 tokens (-fido2, #505)
  • Add -encrypt-paths / -decrypt-paths functionality to gocryptfs-xray (#416)
  • Accept multiple -passfiles (#288)
  • Make -masterkey=stdin work together with -passwd (#461)
  • Fix Unknown opcode 2016 crash on Google Cloud (go-fuse #276, gocryptfs commit ec74d1d)

v1.8.0, 2020-05-09

  • Enable ACL support (#453)
    • Warning 2021-02-07: This feature is incomplete! Do not use ACLs before gocryptfs v2.0 final! Reading and writing ACLs works, but they are not enforced or inherited (#542)
  • Ignore .nfsXXX temporary files (#367)
  • Handle inode number collisions from multiple devices (#435)
  • Drop -nonempty for fusermount3 (#440)
  • Reverse mode: improve inode number mapping and max=1000000000000000000 limitation (#457)
  • Enable --buildmode=pie (#460)
  • Migrate from dep to Go Modules (commit cad711993)
  • go mod: update dependencies (commit b23f77c)
  • gocryptfs -speed: add XChaCha20-Poly1305-Go (#452)
  • Respect GOMAXPROCS environment variable (commit ff210a06f
  • Completely remove Trezor-related code (commit 1364b44ae356da31e24e5605fe73a307e9d6fb03)
    • Has been disabled since v1.7 due to issues a third-party module.
    • Please use FIDO2 instead (gocryptfs v2.0)

v1.7.1, 2019-10-06

  • Support wild cards in reverse mode via --exclude-wildcard (#367). Thanks @ekalin!
  • Create gocryptfs.diriv files with 0440 permissions to make it easier to share an encrypted folder via a network drive (#387). Note: as a security precaution, the owner must still manually chmod gocryptfs.conf 0440 to allow mounting.
  • Allow the nofail option in /etc/fstab
  • -passwd can now change the -scryptn parameter for existing filesystems (#400)
  • Fix -idle unmounting the filesystem despite recent activity (#421)
  • Fix a race condition related to inode number reuse (#363). It could be triggered by concurrently creating and deleting files and can lead to data loss in the affected file. This bug was found by the automated tests on Travis and was very hard to trigger locally.
  • tests: use /var/tmp instead of /tmp by default (commit 8c4429)

v1.7, 2019-03-17

  • Fix possible symlink race attacks in forward mode when using allow_other + plaintextnames
    • If you use both -allow_other and -plaintextnames, you should upgrade. Malicious users could trick gocryptfs into modifying files outside of CIPHERDIR, or reading files inside CIPHERDIR that they should not have access to.
    • If you do not use -plaintextnames (disabled per default), these attacks do not work as symlinks are encrypted.
    • Forward mode has been reworked to use the "*at" family of system calls everywhere (Openat/Unlinkat/Symlinkat/...).
    • As a result, gocryptfs may run slightly slower, as the caching logic has been replaced and is very simple at the moment.
    • The possibility for such attacks was found during an internal code review.
  • Reverse mode: fix excluded, unaccessible files showing up in directory listings (#285, #286)
  • gocryptfs-xray: add -aessiv flag for correctly parsing AES-SIV format files (#299)
  • Ensure that standard fds 0,1,2 are always initialized (#320). Prevents trouble in the unlikely case that gocryptfs is called with stdin,stdout and/or stderr closed.
  • -extpass now can be specified multiple times to support arguments containing spaces (#289)
  • Drop Fstatat, Mkdirat, Syslinkat, Fchownat, Unlinkat, Renameat, Openat emulation of MacOS and instead use native functions (thanks @slackner !)
  • Use Setreuid to robustly set the owner with allow_other (@slackner, (commit))
  • Pack the rendered man page into the source code archive for user convenience (issue 355)
  • Disable Trezor support again (commit 16fac26c57ba303bf60266d24c17f5243e5ea376)
    • Trezor support has been broken since Sept 2018 due to issues in a third-party module (#261)

v1.6.1, 2018-12-12

  • Fix "Operation not supported" chmod errors on Go 1.11 (#271)

v1.6, 2018-08-18

  • Add -e / -exclude option for reverse mode (#235, commit)
  • Add support for the Trezor One HSM PR#247, thanks @xaionaro!
    • Use ./build.bash -tags enable_trezor to compile with Trezor support
    • Then, use gocryptfs -init -trezor to create a filesystem locked with a physical Trezor device.
    • Note 2021-01-31: Support was removed again in gocryptfs v1.7. Please use -fido2 in gocryptfs v2.0.
  • Only print master key once, on init (#76, commit)
  • Fall back to buffered IO even when passed O_DIRECT (commit)

v1.5, 2018-06-12

  • Support extended attributes (xattr) in forward mode (#217). Older gocryptfs versions will ignore the extended attributes.
  • Add -fsck function (#191)
  • Fix clobbered timestamps on MacOS High Sierra (#229)
  • Add -masterkey=stdin functionality (#218)
  • Accept -dev/-nodev, suid/nosuid, -exec/-noexec, -ro/-rw flags to make mounting via /etc/fstab possible. Thanks @mahkoh! (#233, commit, commit)
  • Fix a logger path issue on SuSE #225
  • Stop printing the help text on a "flag provided but not defined" error (commit)

v1.4.4, 2018-03-18

  • Overwrite secrets in memory with zeros as soon as possible (#211)
  • Fix Getdents problems on i386 and mips64le (#197, #200)
  • Make building with gccgo work (#201)
  • MacOS: fix osxfuse: vnode changed generation / Error code -36 issue in go-fuse (#213, commit)
  • Fix various test issues on MacOS

v1.4.3, 2018-01-21

  • Fix several symlink race attacks in connection with reverse mode and allow_other. Thanks to @slackner for reporting and helping to fix the issues:
    • Fix symlink races in reverse mode (issue #165)
    • Fix symlink races in connection with -allow_other (issue #177)
  • Fix problems with special names when using -plaintextnames (issue #174)
  • Add -devrandom command-line option (commit)
  • Add -sharedstorage command-line option (commit, issue #156)
  • MacOS: let OSXFuse create the mountpoint if it does not exist (issue #194)

v1.4.2, 2017-11-01

  • Add Gopkg.toml file for dep vendoring and reproducible builds (issue #142)
  • MacOS: deal with .DS_Store files inside CIPHERDIR (issue #140)
  • Reverse mode: fix ENOENT error affecting names exactly 176 bytes long (issue #143)
  • Support kernels compiled with > 128 kiB FUSE request size (Synology NAS) (issue #145, commit)
  • Fix a startup hang when $PATH contains the mountpoint (issue #146)

v1.4.1, 2017-08-21

  • Use memory pools for buffer handling ( 3c6fe98, b2a23e9, 12c0101)
    • On my machine, this doubles the streaming read speed (see performance.txt)
  • Implement and use the getdents(2) syscall for a more efficient OpenDir implementation (e50a6a5)
  • Purge masterkey from memory as soon as possible (issue #137)
  • Reverse mode: fix inode number collision between .name and .diriv files (d12aa57)
  • Prevent the logger from holding stdout open (issue #130)
  • MacOS: make testing without openssl work properly (ccf1a84)
  • MacOS: specify a volume name (9f8e19b)
  • Enable writing to write-only files (issue #125)

v1.4, 2017-06-20

  • Switch to static binary releases
    • From gocryptfs v1.4, I will only release statically-built binaries. These support all Linux distributions but cannot use OpenSSL.
    • OpenSSL is still supported - just compile from source!
  • Add -force_owner option to allow files to be presented as owned by a different user or group from the user running gocryptfs. Please see caveats and guidance in the man page before using this functionality.
  • Increase open file limit to 4096 (#82).
  • Implement path decryption via ctlsock (#84). Previously, decryption was only implemented for reverse mode. Now both normal and reverse mode support both decryption and encryption of paths via ctlsock.
  • Add more specific exit codes for the most common failure modes, documented in CLI_ABI.md
  • Reverse mode: make sure hard-linked files always return the same ciphertext (commit 9ecf2d1a)
  • Display a shorter, friendlier help text by default.
  • Parallelize file content encryption by splitting data blocks into two threads (ticket#116)
  • Prefetch random nonces in the background (commit 80516ed)
  • Add -info option to pretty-print infos about a filesystem.

v1.3, 2017-04-29

  • Use HKDF to derive separate keys for GCM and EME
    • New feature flag: HKDF (enabled by default)
    • This is a forwards-compatible change. gocryptfs v1.3 can mount filesystems created by earlier versions but not the other way round.
  • Enable Raw64 filename encoding by default (gets rid of trailing == characters)
    • This is a forwards-compatible change. gocryptfs v1.3 can mount filesystems created by earlier versions but not the other way round.
  • Drop Go 1.4 compatibility. You now need Go 1.5 (released 2015-08-19) or higher to build gocryptfs.
  • Add -serialize_reads command-line option
    • This can greatly improve performance on storage that is very slow for concurrent out-of-order reads. Example: Amazon Cloud Drive (#92)
  • Reject file-header-only files (#90 2.2, commit)
  • Increase max password size to 2048 bytes (#93)
  • Use stable 64-bit inode numbers in reverse mode
    • This may cause problems for very old 32-bit applications that were compiled without Large File Support.
  • Passing "--" now also blocks "-o" parsing

v1.2.1, 2017-02-26

  • Add an integrated speed test, gocryptfs -speed
  • Limit password size to 1000 bytes and reject trailing garbage after the newline
  • Make the test suite work on Mac OS X
  • Handle additional corner cases in -ctlsock path sanitization
  • Use dedicated exit code 12 on "password incorrect"

v1.2, 2016-12-04

  • Add a control socket interface. Allows to encrypt and decrypt filenames. For details see backintime#644.
    • New command-line option: -ctlsock
  • Under certain circumstances, concurrent truncate and read could return an I/O error. This is fixed by introducing a global open file table that stores the file IDs (commit).
  • Coalesce 4kB ciphertext block writes up to the size requested through the write FUSE call (commit with benchmarks)
  • Add -noprealloc command-line option
    • Greatly speeds up writes on Btrfs (#63) at the cost of reduced out-of-space robustness.
    • This is a workaround for Btrfs' slow fallocate(2)
  • Preserve owner for symlinks an device files (fixes bug #64)
  • Include rendered man page gocryptfs.1 in the release tarball

v1.1.1, 2016-10-30

  • Fix a panic on setting file timestamps (go-fuse#131)
  • Work around an issue in tmpfs that caused a panic in xfstests generic/075 (gocryptfs#56)
  • Optimize NFS streaming writes (commit)

v1.1, 2016-10-19

  • Add reverse mode (#19)
    • AES-SIV (RFC5297) encryption to implement deterministic encryption securely. Uses the excellent jacobsa/crypto library. The corresponding feature flag is called AESSIV.
    • New command-line options: -reverse, -aessiv
    • Filesystems using reverse mode can only be mounted with gocryptfs v1.1 and later.
    • The default, forward mode, stays fully compatible with older versions. Forward mode will keep using GCM because it is much faster.
  • Accept -o foo,bar,baz-style options that are passed at the end of the command-line, like mount(1) does. All other options must still precede the passed paths.
    • This allows mounting from /etc/fstab. See #45 for details.
    • Mounting on login using pam_mount works as well. It is described in the wiki.
  • To prevent confusion, the old -o option had to be renamed. It is now called -ko. Arguments to -ko are passed directly to the kernel.
  • New -passfile command-line option. Provides an easier way to read the password from a file. Internally, this is equivalent to -extpass "/bin/cat FILE".
  • Enable changing the password when you only know the master key (#28)

v1.0, 2016-07-17

  • Deprecate very old filesystems, stage 3/3
    • Filesystems created by v0.6 can no longer be mounted
    • Drop command-line options -gcmiv128, -emenames, -diriv. These are now always enabled.
  • Add fallocate(2) support
  • New command-line option -o
    • Allows to pass mount options directly to the kernel
  • Add support for device files and suid binaries
    • Only works when running as root
    • Must be explicitly enabled by passing "-o dev" or "-o suid" or "-o suid,dev"
  • Experimental Mac OS X support. See ticket #15 for details.

v0.12, 2016-06-19

  • Deprecate very old filesystems, stage 2/3
    • Filesystems created by v0.6 and older can only be mounted read-only
    • A message explaining the situation is printed as well
  • New command line option: -ro
    • Mounts the filesystem read-only
  • Accept password from stdin as well (ticket #30)

v0.11, 2016-06-10

  • Deprecate very old filesystems, stage 1/3
    • Filesystems created by v0.6 and older can still be mounted but a warning is printed
    • See ticket #29 for details and join the discussion
  • Add rsync stress test "pingpong-rsync.bash"
    • Fix chown and utimens failures that caused rsync to complain
  • Build release binaries with Go 1.6.2
    • Big speedup for CPUs with AES-NI, see ticket #23

v0.10, 2016-05-30

  • Replace spacemonkeygo/openssl with stupidgcm
    • gocryptfs now has its own thin wrapper to OpenSSL's GCM implementation called stupidgcm.
    • This should fix the compile issues people are seeing with spacemonkeygo/openssl. It also gets us a 20% performance boost for streaming writes.
  • Automatically choose between OpenSSL and Go crypto issue #23
    • Go 1.6 added an optimized GCM implementation in amd64 assembly that uses AES-NI. This is faster than OpenSSL and is used if available. In all other cases OpenSSL is much faster and is used instead.
    • -openssl=auto is the new default
    • Passing -openssl=true/false overrides the autodetection.
  • Warn but continue anyway if fallocate(2) is not supported by the underlying filesystem, see issue #22
    • Enables to use gocryptfs on ZFS and ext3, albeit with reduced out-of-space safety.
  • Fix statfs, by @lxp
  • Fix a fsstress failure in the go-fuse library.

v0.9, 2016-04-10

  • Long file name support
    • gocryptfs now supports file names up to 255 characters.
    • This is a forwards-compatible change. gocryptfs v0.9 can mount filesystems created by earlier versions but not the other way round.
  • Refactor gocryptfs into multiple "internal" packages
  • New command-line options:
    • -longnames: Enable long file name support (default true)
    • -nosyslog: Print messages to stdout and stderr instead of syslog (default false)
    • -wpanic: Make warning messages fatal (used for testing)
    • -d: Alias for -debug
    • -q: Alias for -quiet

v0.8, 2016-01-23

  • Redirect output to syslog when running in the background
  • New command-line option:
    • -memprofile: Write a memory allocation debugging profile the specified file

v0.7.2, 2016-01-19

  • Fix performance issue in small file creation
    • This brings performance on-par with EncFS paranoia mode, with streaming writes significantly faster
    • The actual fix is in the go-fuse library. There are no code changes in gocryptfs.

v0.7.1, 2016-01-09

  • Make the build.bash script compatible with Go 1.3
  • Disable fallocate on OSX (system call not available)
  • Introduce pre-built binaries for Fedora 23 and Debian 8

v0.7, 2015-12-20

  • Extend GCM IV size to 128 bit from Go's default of 96 bit
    • This pushes back the birthday bound to make IV collisions virtually impossible
    • This is a forwards-compatible change. gocryptfs v0.7 can mount filesystems created by earlier versions but not the other way round.
  • New command-line option:
    • -gcmiv128: Use 128-bit GCM IVs (default true)

v0.6, 2015-12-08

  • Wide-block filename encryption using EME + DirIV
    • EME (ECB-Mix-ECB) provides even better security than CBC as it fixes the prefix leak. The used Go EME implementation is https://github.com/rfjakob/eme which is, as far as I know, the first implementation of EME in Go.
    • This is a forwards-compatible change. gocryptfs v0.6 can mount filesystems created by earlier versions but not the other way round.
  • New command-line option:
    • -emenames: Enable EME filename encryption (default true)

v0.5.1, 2015-12-06

  • Fix a rename regression caused by DirIV and add test case
  • Use fallocate to guard against out-of-space errors

v0.5, 2015-12-04

  • Stronger filename encryption: DirIV
    • Each directory gets a random 128 bit file name IV on creation, stored in gocryptfs.diriv
    • This makes it impossible to identify identically-named files across directories
    • A single-entry IV cache brings the performance cost of DirIV close to zero for common operations (see performance.txt)
    • This is a forwards-compatible change. gocryptfs v0.5 can mount filesystems created by earlier versions but not the other way round.
  • New command-line option:
    • -diriv: Use the new per-directory IV file name encryption (default true)
    • -scryptn: allows to set the scrypt cost parameter N. This option can be used for faster mounting at the cost of lower brute-force resistance. It was mainly added to speed up the automated tests.

v0.4, 2015-11-15

  • New command-line options:
    • -plaintextnames: disables filename encryption, added on user request
    • -extpass: calls an external program for prompting for the password
    • -config: allows to specify a custom gocryptfs.conf path
  • Add FeatureFlags gocryptfs.conf parameter
    • This is a config format change, hence the on-disk format is incremented
    • Used for ext4-style filesystem feature flags. This should help avoid future format changes. The first user is -plaintextnames.
  • On-disk format 2

v0.3, 2015-11-01

  • Add a random 128 bit file header to authenticate file->block ownership
    • This is an on-disk-format change
  • On-disk format 1

v0.2, 2015-10-11

  • Replace bash daemonization wrapper with native Go implementation
  • Better user feedback on mount failures

v0.1, 2015-10-07

  • First release
  • On-disk format 0
Issues
  • Mac OS X support

    Mac OS X support

    Go support Mac OS X, as does the FUSE library we use, go-fuse.

    gocrypts may actually work out of the box on OSX, but there probably are small issues that have to be sorted out.

    At the very least, gocryptfs has to be tested on OSX. As I do not have a Mac to test on, this would be an opportunity for somebody from the Mac community to step up. Please comment here if you are interested.

    feature request help wanted 
    opened by rfjakob 121
  • Feature Request: encryption primitives for devices without AES cpu instructions

    Feature Request: encryption primitives for devices without AES cpu instructions

    Hi @rfjakob,

    Thank you for this great application! The reverse mode is what really sets it apart from other options.

    I checked the issues, and it doesn't seem to be discussed yet, but what do you think about adding support for a different collection of encryption primitives that are better suited for more low-end devices?

    I'm running gocryptfs on a few ARMv6/7 based NAS machines, they are nice: low energy, and quite fast. But they lack native AES instructions, my fastest ARM device (Odroid XU4) maxes out at 40MB/s, while for example the raspberry-pi's and friends are quite a bit slower (rpi1 is at 15MB/s).

    Maybe Google Adiantum (also added to linux kernel 5.0 for cryptfs) is a nice fit, Adiantum is based on XChaCha12 and Poly1305 and is roughly 5 quicker than AES-XTS for devices without AES instructions.

    For the reverse mode maybe something based on ChaCha20Poly1305?

    Just for comparison, on my Odroid XU4, ChaCha20Poly1305 runs at 320MB/s, on my RPi1 it gets close to 40MB/s.

    So I'm just wondering what your view is on this topic.

    Cheers, Davy

    feature request maybe some day ⌛ 
    opened by DavyLandman 79
  • xfstests generic/273 failure

    xfstests generic/273 failure

    Needs further analysis:

    _porter 28 not complete
    cp: cannot create regular file '/var/tmp/check-gocryptfs/scratchdir/sub_28/origin/file_548': No such file or directory
    

    https://github.com/rfjakob/gocryptfs/blob/master/Documentation/XFSTESTS.md#generic273

    bug bug outside gocryptfs 
    opened by rfjakob 36
  • macOS (Error code -36) while copying

    macOS (Error code -36) while copying

    Hey there,

    I am getting this (title) message very often while copying data. In the System Console I see:

    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:51,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:52,000 kernel[0]: osxfuse: vnode changed generation
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    19/02/18 21:25:53,000 kernel[0]: osxfuse: fuse_vnop_readdir failed (err=22)
    

    Although this seems to occur sometimes without the error in Finder. When I copy via cpthen some files do not get copied and cp prints Stale NFS file handle

    The debug output of gocrypt itself does not show any errors. I am using version gocryptfs v1.4.3; go-fuse [vendored]; 2018-02-02 go1.9.3 (Homebrew) and OSX Fuse version 3.7.1.

    Thanks a lot, Magnus

    bug 
    opened by magnusja 36
  • macOS: modification and creation dates reset

    macOS: modification and creation dates reset

    I started using gocryptfs (latest version) on macOS High Sierra and noticed that each time I eject my encrypted volume and remount it, that the creation and modification dates are reset to 1970-01-01. Is this expected behaviour?

    I mount my dirs using

    gocryptfs -ko local /path/to/encDir /Volumes/encDir
    

    Also mounting them without the -ko local option has the same result.

    bug 
    opened by alexanderharm 35
  • Feature Request: Reverse Mode

    Feature Request: Reverse Mode

    Add an option similar to the --reverse option for encfs:

    Normally EncFS provides a plaintext view of data on demand. Normally it stores enciphered data and displays plaintext data. With --reverse it takes as source plaintext data and produces enciphered data on-demand. This can be useful for creating remote encrypted backups, where you do not wish to keep the local files unencrypted.

    For example, the following would create an encrypted view in /tmp/crypt-view.

    encfs --reverse /home/me /tmp/crypt-view You could then copy the /tmp/crypt-view directory in order to have a copy of the encrypted data. You must also keep a copy of the file /home/me/.encfs5 which contains the filesystem information. Together, the two can be used to reproduce the unencrypted data: ENCFS5_CONFIG=/home/me/.encfs5 encfs /tmp/crypt-view /tmp/plain-view Now /tmp/plain-view contains the same data as /home/me Note that --reverse mode only works with limited configuration options, so many settings may be disabled when used.

    (Quote from encfs man page)

    feature request 
    opened by usr42 30
  • Poor read performance with network backend

    Poor read performance with network backend

    Use case

    • /tmp/encrypted is an Amazon Cloud Drive folder, mounted read-only using rclone
    • /tmp/clear is the gocryptfs deciphered version of the above.

    Copying a 300MB file (ciphered) from /tmp/encrypted, I get around 5-6MBps, which is OK, considering my bandwidth is 10MBps. Copying the same file from /tmp/clear yields extremely poor speeds (from 100KBps to 1MBps). Speed is not limited by the CPU. Tracing read requests in rclone shows read sizes from 4KB to 128KB occuring at low frequency (thus low througput)

    Any advice would be greatly appreciated.

    Environment

    • Debian Stretch, x86_64
    • gocryptfs 1.2.1
    • Ciphered volume has the following features:
      • version: 2
      • flags: GCMIV128, DirIV, EMENames, LongNames, AESSIV
    • rclone 1.35, mount max-read-ahead set to 16MB
    opened by j-vizcaino 28
  • gocryptfs for Windows - Cross-platform support

    gocryptfs for Windows - Cross-platform support

    Because it's Go, could it be made to run on Windows as well? (Cross-platform support is an attractive feature.)

    The problem with windows is that it does not have FUSE support. Linux has it built-in and Mac has the OSXFuse project. On windows, the only thing i am aware of is http://encfsmp.sourceforge.net/ . They SOMEHOW managed to get encfs (which also uses FUSE) running on windows, using a closed-source windows kernel driver and lots of black magic.

    See discussion here https://github.com/rfjakob/gocryptfs/issues/2#issuecomment-152955769 and here https://github.com/rfjakob/gocryptfs/issues/2#issuecomment-152962990

    feature request help wanted 
    opened by dakkusingh 27
  • Dircache should handle fd == 0 as a valid file descriptor

    Dircache should handle fd == 0 as a valid file descriptor

    While rather unlikely in practice, a file descriptor of 0 is completely valid and should not be treated as an error [1]. Both open and Dup typically return the smallest available file descriptor and only negative values are errors. If the user starts gocryptfs with stdin (fd == 0) closed and triggers the dircache code fast enough (not sure if that is actually possible), this could lead to a panic / unexpected behavior (e.g., in the fd validation in Store).

    [1] https://unix.stackexchange.com/questions/100611/aix-open-file-descriptor-is-zero

    opened by slackner 25
  • Cannot initialize gocryptfs due to

    Cannot initialize gocryptfs due to "WriteDirIV: open .../cipher/gocryptfs.diriv: permission denied"

    I'm trying to set up gocryptfs on Ubuntu 14.04.1 LTS. I downloaded the binary release 1.2.1 from here (Debian version) and tried initilaizing a folder. But I get the mentioned error:

    $ cd ~ $ mkdir plain cipher $ ./gocryptfs -init cipher Choose a password for protecting your files. Password: Repeat: WriteDirIV: open /home/<username>/cipher/gocryptfs.diriv: permission denied open /home/<username>/cipher/gocryptfs.diriv: permission denied

    In case this matters: my home folder is an NFS4 mount.

    opened by Natanji 25
  • Far too high RSS memory consumption (memory leak?) with gocryptfs

    Far too high RSS memory consumption (memory leak?) with gocryptfs

    Far too high RSS memory consumption (memory leak?) with gocryptfs

    I'm about to migrate my backup from EncFS to gocryptfs as gocryptfs IMHO is the best choice you can currently make for encrypted filesystems on Unix.

    My version of gocryptfs is the one installed with apt install gocryptfs from my linux distribution Armbian (basically an Ubuntu 20.04 LTS Focal)

    gocryptfs --version
    gocryptfs 1.7.1; go-fuse 0.0~git20190214.58dcd77; 2019-12-26 go1.13.5 linux/arm
    
    cat /etc/issue
    Armbian 21.05.1 Focal \l
    

    My setup uses https://dirvish.org/ to create frequent rotating hard-link-snapshots, which results in approximately these numbers:

    • 500,000 inodes
    • 12,000,000 files (approx 25 hard linked files per inode)
    • Hardware with 2 GB of total RAM (Odroid XU4Q)

    Before migrating from EncFS to gocryptfs there were absolutely no problems with the EncFS RAM (RSS) usage, typically approx 100 kB or even zero (!).

    The RSS memory consumption with EncFS was perfectly low, especially with respect to the fact, that approx. 20 docker containers are running on the system.

    image-20210530070417145

    After switching to gocryptfs, the RAM usage exploded. gocryptfs consumes 32.5% of the total available RSS RAM (645 MB of 2 GB):

    image-20210529071413801

    Even after waiting approx 7 days (without any activity in gocryptfs!), gocryptfs still holds the same amount of RAM.

    Here is a historical comparison with encfs:

    How to read it (see also https://github.com/prometheus/node_exporter ):

    "RAM Used" = RSS-Sum = ${node_memory_MemTotal_bytes} - ${node_memory_MemAvailable_bytes}
    "RAM Free"           = ${node_memory_MemFree_bytes} = ${node_memory_MemTotal_bytes} - ${RAM used}
    "RAM Buffer + Cache" = ${node_memory_MemAvailable_bytes} - ${node_memory_MemFree_bytes}
    "SWAP used"          = ${node_memory_SwapTotal_bytes} - ${node_memory_SwapFree_bytes}
    "RAM + SWAP Used"    = ${RAM used} + ${SWAP used}
    

    image-20210529071714533

    image-20210529092539072

    I found https://github.com/rfjakob/gocryptfs/issues/132 and tried to force the kernel to drop all caches (taken from https://askubuntu.com/questions/609226/freeing-page-cache-using-echo-3-proc-sys-vm-drop-caches-doesnt-work ):

    sync; echo 3 > /proc/sys/vm/drop_caches
    

    Unfortunately this had no effect in the RAM usage of gocryptfs.

    I also tried to allocate 2 GB of RAM with a shell script for 10 Minutes (taken from https://stackoverflow.com/questions/4964799/write-a-bash-shell-script-that-consumes-a-constant-amount-of-ram-for-a-user-defi ):

    #!/bin/bash
    
    echo "Provide sleep time in the form of NUMBER[SUFFIX]"
    echo "   SUFFIX may be 's' for seconds (default), 'm' for minutes,"
    echo "   'h' for hours, or 'd' for days."
    read -p "> " delay
    
    vmstat
    echo "begin allocating memory..."
    for index in $(seq 1000); do
        value=$(seq -w -s '' $index $(($index + 100000)))
        eval array$index=$value
    done
    echo "...end allocating memory"
    vmstat
    echo "sleeping for $delay"
    vmstat
    sleep $delay
    

    This also had no effect on the RAM usage of gocryptfs.

    image-20210529092812055

    image-20210529092900640

    Only remounting of the gocrypt filesystem could reduce the RAM usage:

    image-20210529093127139

    Very nice RAM usage after remount:

    image-20210529093102556

    When doing a ls -R only over the approx. 500,000 "real" files, not over the hardlink snapshots (which basically matches the behaviour of the daily rsync job), the gocryptfs RAM usage again goes up.

    image-20210529094338735

    Doing the same thing a couple of consecutive times, basically all memory usages constantly increase, which could be an indicator for a memory leak (detailed log see attached file rss-mem-log-1.txt ):

    > umount-gocryptfs.sh ; mount-gocryptfs.sh
    > ps -o rss,size,vsize,cmd $(pgrep gocryptfs)
      RSS SIZE   VSZ    CMD
    75124 143104 875808 /usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/
    
    > for i in 1 2 3 4 5 6 7 8 9 ; do
    >  echo "Run $i ----------------------------------"
    >  ls -R private-backup-gocryptfs/current | wc -l
    >  ps -o rss,size,vsize,cmd $(pgrep gocryptfs)
    > done
    

    |Run| RSS| SIZE| VSZ| CMD| | ---- | ---- | ---- | ---- | ---- | |1|142104|285888|950848|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/| |2|193332|365444|952032|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/| |3|205764|365444|952032|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/| |4|224436|365444|952032|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/| |5|224436|365444|952032|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/| |6|223392|365444|952032|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/| |7|225236|373636|960228|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/| |8|224424|373636|960228|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/| |9|230916|373636|960228|/usr/bin/gocryptfs -fg -notifypid=1787 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/|

    Doing the same thing a couple of consecutive times over all files including the hardlink snapshots (approx 12,000,000), basically all memory usages constantly increase, which could be an indicator for a memory leak (detailed log see attached file rss-mem-log-2.txt ):

    umount-gocryptfs.sh ; mount-gocryptfs.sh
    ps -o rss,size,vsize,cmd $(pgrep gocryptfs)
    for i in 1 2 3 4 5 6 7 8 9 ; do 
     	echo "Run $i ----------------------------------"
     	ls -R private-backup-gocryptfs/current private-backup-gocryptfs/snapshots | wc -l 
     	ps -o rss,size,vsize,cmd $(pgrep gocryptfs)
    done
    

    | Run | RSS | SIZE | VSZ | CMD | | ---- | ------ | ------- | ------- | ------------------------------------------------------------ | | 1 | 833348 | 993072 | 1228656 | /usr/bin/gocryptfs -fg -notifypid=7788 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/ | | 2 | 832360 | 1279864 | 1498928 | /usr/bin/gocryptfs -fg -notifypid=7788 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/ | | 3 | 933928 | 1302148 | 1519920 | /usr/bin/gocryptfs -fg -notifypid=7788 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/ | | 4 | 939984 | 1302148 | 1519920 | /usr/bin/gocryptfs -fg -notifypid=7788 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/ | | 5 | 941632 | 1319820 | 1536560 | /usr/bin/gocryptfs -fg -notifypid=7788 -nonempty /home/cb/enc-backup-files/gocryptfs-vol /home/cb/private-backup-gocryptfs/ |

    In case of interest: Here are some details of the gocryptfs process with an RSS of 918 MB: pmap.txt

    With these numbers it's a really hard decision, whether to to do the final switch from EncFS to gocryptfs.

    Does anybody know, whether there have been improvements in this area from the current version distributed by apt (gocryptfs 1.7.1) to the current stable version (gocryptfs 1.8.0) or the current beta version (v2.0-beta4)?

    Any other hints what to use gocryptfs without wasting so much RSS RAM?

    opened by Slartibartfast27 23
  • Build failure against riscv64

    Build failure against riscv64

    github.com/jacobsa/crypto/cmac
    golang.org/x/sys/cpu
    # github.com/jacobsa/crypto/cmac
    vendor/github.com/jacobsa/crypto/cmac/hash.go:97:3: undefined: xorBlock
    

    See: http://autobuild.buildroot.net/results/caa60874781c4077273884eb37281cc9e02ef9ac/build-end.log

    opened by paralin 2
  • Migration path to deterministic file name version

    Migration path to deterministic file name version

    I know I could just create a new fs volume with deterministic file name option enabled and then copy all data from the "old" diriv version to the new diriv-less one.

    but I really, really, really, really (goes on for ages) would appreciate a more elegant migration path.

    It took me years to upload 2 PB of data to google drive. I really dont want to spend another few years copying stuff but I also really want to get rid of the diriv for so many reasons.

    so, who do I have to lobby :-)

    feature request help wanted 
    opened by loungebob 3
  • Logger process gets killed by xfce4-terminal

    Logger process gets killed by xfce4-terminal

    When mounting with the following command line (Arch linux, latest version 2.2.1 from the standard Arch repo) the process list shows a zombie logger (screenshot)

    cat "$KEY_FILE" | gocryptfs -nonempty -masterkey stdin "$VOLUME_PATH" "$ACCESS_DIR"

    image

    The mount seems to be fully operational.

    $ gocryptfs -version
    gocryptfs v2.2.1; go-fuse [vendored]; 2022-04-27 go1.18.1 linux/amd64
    $ uname -a
    Linux host_name 5.17.9-arch1-1 #1 SMP PREEMPT Wed, 18 May 2022 17:30:11 +0000 x86_64 GNU/Linux
    $ go version
    go version go1.18.2 linux/amd64
    

    I do not manually specify neither "-fg" nor "-notifypid" options. BTW the process with "notifypid" PID does not exists.

    bug 
    opened by cherio 3
  • Doesn't exist in latest fedora

    Doesn't exist in latest fedora

    As we can see it's orphaned for more than 6 week and it doesn't in the repo anymore : https://src.fedoraproject.org/rpms/golang-github-rfjakob-gocryptfs

    opened by BirdInFire 1
  • Editing files yields E667: Fsync failed

    Editing files yields E667: Fsync failed

    Description

    I use gocryptfs to encrypt files stored on an SMB volume that I mount via sudo mount -t cifs -o "user=$USERNAME,dir_mode=0755,file_mode=0644,uid=$(id -u $USERNAME),gid=$(id -g $USERNAME)". This worked flawlessly in the past. Recently, I updated from Debian 9 to Debian 10. Since then I am not able to edit files anymore. Copying, moving, renaming or replacing files works without a problem, though. Here is what I see when using vim.

    Trying to save a newly created file via :w yields the following error message but the file is created successfully:

    "test.txt" E667: Fsync failed
    WARNING: Original file may be lost or damaged
    don't quit the editor until the file is successfully written!
    Press ENTER or type command to continue
    

    Trying to save the already opened newly created file via :w yields the following error message (which is unusal) and the file contents is not changed:

    E13: File exists (add ! to override)
    

    Trying to save the already opened newly created file via :w! yields the following error message (which is unusal) and the file contents is not changed:

    WARNING: The file has been changed since reading it!!!
    Do you really want to write to it (y/n)?y
    "test.txt" E667: Fsync failed
    Press ENTER or type command to continue
    

    Reopening the file and trying to save it via :w yields the following error message and the file contents is not changed:

    "test.txt" E667: Fsync failed
    Press ENTER or type command to continue
    

    Expected Behaviour

    I'd expect to be able to edit files.

    Actual Behaviour

    Editing files is not possible. Error messages and the debug log hint to fsync as the potential problem.

    Debug log

    Trying to save a newly created file via :w

    11:32:18.042078 rx 989: LOOKUP n1 ["test.txt"] 9b
    11:32:18.044959 tx 989:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:32:18.045136 rx 990: LOOKUP n1 ["test.txt"] 9b
    11:32:18.047622 tx 990:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:32:18.047992 rx 991: LOOKUP n1 ["test.txt"] 9b
    11:32:18.050846 tx 991:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:32:18.051232 rx 992: LOOKUP n1 ["test.txt"] 9b
    11:32:18.053648 tx 992:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:32:18.053834 rx 993: CREATE n1 {0100644 [CREAT,WRONLY,0x8000] (022)} ["test.txt"] 9b
    11:32:18.060036 tx 993:     OK, {n686 g1 {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940492 A 1651311158.978647 M 1651311158.978647 C 1651311158.978647} &{2 0 0}}
    11:32:18.060530 rx 994: GETXATTR n686 {sz 0} ["security.capability"] 20b
    11:32:18.060617 tx 994:     95=operation not supported
    11:32:18.060745 rx 995: SETATTR n686 {size 0, fh 2} 
    file.GetAttr()
    11:32:18.085449 tx 995:     OK, {tA=0s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940492 A 1651311158.978647 M 1651311138.059458 C 1651311138.059458}}
    11:32:18.085646 rx 996: GETXATTR n686 {sz 0} ["security.capability"] 20b
    11:32:18.085732 tx 996:     95=operation not supported
    11:32:18.085898 rx 997: WRITE n686 {Fh 2 [0 +18)  L 0 WRONLY,0x8000} "Das ist "... 18b
    ino76940492: FUSE Write: offset=0 length=18
    doRead: off=0 len=4096 -> off=18 len=4128 skip=0
    len(oldData)=0 len(blockData)=18
    ino76940492: Writing 18 bytes to block #0
    11:32:18.089115 tx 997:     OK
    11:32:18.089225 rx 998: FSYNC n686 
    11:32:18.106965 tx 998:     13=permission denied
    11:32:18.107042 rx 999: GETATTR n686 {Fh 0} 
    file.GetAttr()
    11:32:18.107093 tx 999:     OK, {tA=1s {M0100644 SZ=18 L=1 1000:1000 B0*16384 i0:76940492 A 1651311159.011647 M 1651311138.059458 C 1651311138.059458}}
    11:32:18.107132 rx 1000: FLUSH n686 {Fh 2} 
    11:32:18.107148 tx 1000:     OK
    11:32:18.107385 rx 1001: RELEASE n686 {Fh 2 WRONLY,0x8000  L0} 
    11:32:18.109893 tx 1001:     OK
    

    Trying to save the already opened newly created file via :w

    11:33:51.127859 rx 1007: LOOKUP n1 ["test.txt"] 9b
    11:33:51.157455 tx 1007:     OK, {n686 g1 tE=1s tA=1s {M0100644 SZ=18 L=1 1000:1000 B16*16384 i0:76940492 A 1651311159.011647 M 1651311159.032983 C 1651311159.032983}}
    11:33:51.157749 rx 1008: ACCESS n686 {u=1000 g=1000 w} 
    11:33:51.157928 tx 1008:     OK
    11:35:13.205990 rx 1009: LOOKUP n1 ["test.txt"] 9b
    11:35:13.227039 tx 1009:     OK, {n686 g1 tE=1s tA=1s {M0100644 SZ=18 L=1 1000:1000 B16*16384 i0:76940492 A 1651311159.011647 M 1651311159.032983 C 1651311159.032983}}
    11:35:13.227283 rx 1010: ACCESS n686 {u=1000 g=1000 w} 
    11:35:13.227389 tx 1010:     OK
    

    Trying to save the already opened newly created file via :w!

    11:36:41.988702 rx 1011: LOOKUP n1 ["test.txt"] 9b
    11:36:42.010506 tx 1011:     OK, {n686 g1 tE=1s tA=1s {M0100644 SZ=18 L=1 1000:1000 B16*16384 i0:76940492 A 1651311159.011647 M 1651311159.032983 C 1651311159.032983}}
    11:36:42.010902 rx 1012: ACCESS n686 {u=1000 g=1000 w} 
    11:36:42.011006 tx 1012:     OK
    11:37:55.858597 rx 1013: LOOKUP n1 ["test.txt"] 9b
    11:37:55.881069 tx 1013:     OK, {n686 g1 tE=1s tA=1s {M0100644 SZ=18 L=1 1000:1000 B16*16384 i0:76940492 A 1651311159.011647 M 1651311159.032983 C 1651311159.032983}}
    11:37:55.881321 rx 1014: GETXATTR n686 {sz 132} ["system.posix_acl_access"] 24b
    11:37:55.881516 tx 1014:     95=operation not supported
    11:37:55.881776 rx 1015: LOOKUP n1 ["4913"] 5b
    11:37:55.885154 tx 1015:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:37:55.885266 rx 1016: LOOKUP n1 ["4913"] 5b
    11:37:55.889081 tx 1016:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:37:55.889310 rx 1017: CREATE n1 {0100644 [EXCL,WRONLY,CREAT,0x28000] (022)} ["4913"] 5b
    11:37:55.902027 tx 1017:     OK, {n691 g1 {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311496.844643 M 1651311496.844643 C 1651311496.844643} &{2 0 0}}
    11:37:55.902139 rx 1018: GETXATTR n691 {sz 0} ["security.capability"] 20b
    11:37:55.902173 tx 1018:     95=operation not supported
    11:37:55.902263 rx 1019: GETATTR n691 {Fh 0} 
    file.GetAttr()
    11:37:55.902358 tx 1019:     OK, {tA=1s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311496.844643 M 1651311496.844643 C 1651311496.844643}}
    11:37:55.902406 rx 1020: SETATTR n691 {uid 1000, gid 1000} 
    11:37:55.902538 tx 1020:     OK, {tA=0s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311496.844643 M 1651311496.844643 C 1651311475.898596}}
    11:37:55.902637 rx 1021: GETATTR n691 {Fh 0} 
    file.GetAttr()
    11:37:55.902668 tx 1021:     OK, {tA=1s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311496.844643 M 1651311496.844643 C 1651311475.898596}}
    11:37:55.902704 rx 1022: FLUSH n691 {Fh 2} 
    11:37:55.902722 tx 1022:     OK
    11:37:55.902758 rx 1023: RELEASE n691 {Fh 2 WRONLY,0x28000  L0} 
    11:37:55.902776 rx 1024: UNLINK n1 ["4913"] 5b
    11:37:55.907382 tx 1023:     OK
    11:37:55.920454 tx 1024:     OK
    11:37:55.920611 rx 1025: LOOKUP n1 ["test.txt~"] 10b
    11:37:55.920628 rx 1026: FORGET n691 {Nlookup=1} 
    11:37:55.927042 tx 1025:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:37:55.927214 rx 1027: LOOKUP n1 ["test.txt~"] 10b
    11:37:55.933214 tx 1027:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:37:55.933362 rx 1028: LOOKUP n1 ["test.txt~"] 10b
    11:37:55.937017 tx 1028:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:37:55.937150 rx 1029: LOOKUP n1 ["test.txt~"] 10b
    11:37:55.941198 tx 1029:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:37:55.941324 rx 1030: RENAME n1 {i1} ["test.txt" "test.txt~"] 19b
    Renameat 7/fSRh8isq8XFiKS2nUsVuLQ -> 10/tSSLRAc4Qxs5lO354T8Afg
    11:37:55.966195 tx 1030:     OK
    11:37:55.966311 rx 1031: FSYNC n685 
    11:37:55.983494 tx 1031:     13=permission denied
    11:37:55.983607 rx 1032: LOOKUP n1 ["test.txt"] 9b
    11:37:55.988671 tx 1032:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:37:55.988778 rx 1033: CREATE n1 {0100644 [CREAT,WRONLY,0x8000] (022)} ["test.txt"] 9b
    11:37:56.004852 tx 1033:     OK, {n692 g1 {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311496.944643 M 1651311496.944643 C 1651311496.944643} &{2 0 0}}
    11:37:56.004983 rx 1034: GETXATTR n692 {sz 0} ["security.capability"] 20b
    11:37:56.005009 tx 1034:     95=operation not supported
    11:37:56.005076 rx 1035: SETATTR n692 {size 0, fh 2} 
    file.GetAttr()
    11:37:56.049764 tx 1035:     OK, {tA=0s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311496.944643 M 1651311476.006594 C 1651311476.006594}}
    11:37:56.049904 rx 1036: GETXATTR n692 {sz 0} ["security.capability"] 20b
    11:37:56.049932 tx 1036:     95=operation not supported
    11:37:56.050068 rx 1037: WRITE n692 {Fh 2 [0 +43)  L 0 WRONLY,0x8000} "Das ist "... 43b
    ino76940493: FUSE Write: offset=0 length=43
    doRead: off=0 len=4096 -> off=18 len=4128 skip=0
    len(oldData)=0 len(blockData)=43
    ino76940493: Writing 43 bytes to block #0
    11:37:56.060580 tx 1037:     OK
    11:37:56.060674 rx 1038: FSYNC n692 
    11:37:56.087105 tx 1038:     13=permission denied
    11:37:56.087280 rx 1039: GETATTR n692 {Fh 0} 
    file.GetAttr()
    11:37:56.087377 tx 1039:     OK, {tA=1s {M0100644 SZ=43 L=1 1000:1000 B0*16384 i0:76940493 A 1651311497.005643 M 1651311476.006594 C 1651311476.006594}}
    11:37:56.087448 rx 1040: SETATTR n692 {mode 0100644} 
    11:37:56.087603 tx 1040:     OK, {tA=0s {M0100644 SZ=43 L=1 1000:1000 B0*16384 i0:76940493 A 1651311497.005643 M 1651311476.006594 C 1651311476.082593}}
    11:37:56.087723 rx 1041: FLUSH n692 {Fh 2} 
    11:37:56.087755 tx 1041:     OK
    11:37:56.087822 rx 1042: RELEASE n692 {Fh 2 WRONLY,0x8000  L0} 
    11:37:56.087853 rx 1043: GETATTR n686 {Fh 0} 
    11:37:56.088054 tx 1043:     OK, {tA=1s {M0100644 SZ=18 L=1 1000:1000 B16*16384 i0:76940492 A 1651311159.011647 M 1651311159.032983 C 1651311159.032983}}
    11:37:56.088127 rx 1044: GETATTR n692 {Fh 0} 
    11:37:56.088208 tx 1044:     OK, {tA=1s {M0100644 SZ=43 L=1 1000:1000 B0*16384 i0:76940493 A 1651311497.005643 M 1651311476.006594 C 1651311476.082593}}
    11:37:56.088294 rx 1045: UNLINK n1 ["test.txt"] 9b
    11:37:56.093713 tx 1042:     OK
    11:37:56.112744 tx 1045:     OK
    11:37:56.112959 rx 1046: FORGET n692 {Nlookup=1} 
    11:37:56.112988 rx 1047: LOOKUP n1 ["test.txt"] 9b
    11:37:56.116812 tx 1047:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:37:56.116941 rx 1048: RENAME n1 {i1} ["test.txt~" "test.txt"] 19b
    Renameat 7/tSSLRAc4Qxs5lO354T8Afg -> 10/fSRh8isq8XFiKS2nUsVuLQ
    11:37:56.133482 tx 1048:     OK
    

    Reopening the file and trying to save it via :w

    11:41:50.750046 rx 1380: LOOKUP n1 ["test.txt"] 9b
    11:41:50.770383 tx 1380:     OK, {n686 g1 tE=1s tA=1s {M0100644 SZ=18 L=1 1000:1000 B16*16384 i0:76940492 A 1651311665.673640 M 1651311159.032983 C 1651311159.032983}}
    11:41:50.770602 rx 1381: ACCESS n686 {u=1000 g=1000 w} 
    11:41:50.770718 tx 1381:     OK
    11:41:50.771328 rx 1382: ACCESS n686 {u=1000 g=1000 w} 
    11:41:50.771459 tx 1382:     OK
    11:41:50.771720 rx 1383: GETXATTR n686 {sz 132} ["system.posix_acl_access"] 24b
    11:41:50.771854 tx 1383:     95=operation not supported
    11:41:50.772005 rx 1384: LOOKUP n1 ["4913"] 5b
    11:41:50.776124 tx 1384:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:41:50.776347 rx 1385: LOOKUP n1 ["4913"] 5b
    11:41:50.779300 tx 1385:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:41:50.779493 rx 1386: CREATE n1 {0100644 [CREAT,EXCL,WRONLY,0x28000] (022)} ["4913"] 5b
    11:41:50.786311 tx 1386:     OK, {n826 g1 {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.754640 M 1651311731.754640 C 1651311731.754640} &{1 0 0}}
    11:41:50.786451 rx 1387: GETXATTR n826 {sz 0} ["security.capability"] 20b
    11:41:50.786482 tx 1387:     95=operation not supported
    11:41:50.786519 rx 1388: GETATTR n826 {Fh 0} 
    file.GetAttr()
    11:41:50.786576 tx 1388:     OK, {tA=1s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.754640 M 1651311731.754640 C 1651311731.754640}}
    11:41:50.786614 rx 1389: SETATTR n826 {uid 1000, gid 1000} 
    11:41:50.786694 tx 1389:     OK, {tA=0s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.754640 M 1651311731.754640 C 1651311710.783225}}
    11:41:50.786875 rx 1390: GETATTR n826 {Fh 0} 
    file.GetAttr()
    11:41:50.786958 tx 1390:     OK, {tA=1s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.754640 M 1651311731.754640 C 1651311710.783225}}
    11:41:50.787037 rx 1391: FLUSH n826 {Fh 1} 
    11:41:50.787066 tx 1391:     OK
    11:41:50.787145 rx 1393: UNLINK n1 ["4913"] 5b
    11:41:50.787149 rx 1392: RELEASE n826 {Fh 1 WRONLY,0x28000  L0} 
    11:41:50.794817 tx 1392:     OK
    11:41:50.802501 tx 1393:     OK
    11:41:50.802610 rx 1394: LOOKUP n1 ["test.txt~"] 10b
    11:41:50.802712 rx 1395: FORGET n826 {Nlookup=1} 
    11:41:50.807643 tx 1394:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:41:50.807726 rx 1396: LOOKUP n1 ["test.txt~"] 10b
    11:41:50.810499 tx 1396:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:41:50.810586 rx 1397: LOOKUP n1 ["test.txt~"] 10b
    11:41:50.813354 tx 1397:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:41:50.813452 rx 1398: LOOKUP n1 ["test.txt~"] 10b
    11:41:50.816179 tx 1398:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:41:50.816326 rx 1399: RENAME n1 {i1} ["test.txt" "test.txt~"] 19b
    Renameat 7/fSRh8isq8XFiKS2nUsVuLQ -> 8/tSSLRAc4Qxs5lO354T8Afg
    11:41:50.827086 tx 1399:     OK
    11:41:50.827227 rx 1400: FSYNC n824 
    11:41:50.840178 tx 1400:     13=permission denied
    11:41:50.840327 rx 1401: LOOKUP n1 ["test.txt"] 9b
    11:41:50.846534 tx 1401:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:41:50.846736 rx 1402: CREATE n1 {0100644 [CREAT,WRONLY,0x8000] (022)} ["test.txt"] 9b
    11:41:50.852853 tx 1402:     OK, {n827 g1 {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.821640 M 1651311731.821640 C 1651311731.821640} &{1 0 0}}
    11:41:50.853259 rx 1403: GETXATTR n827 {sz 0} ["security.capability"] 20b
    11:41:50.853350 tx 1403:     95=operation not supported
    11:41:50.853537 rx 1404: SETATTR n827 {size 0, fh 1} 
    file.GetAttr()
    11:41:50.875710 tx 1404:     OK, {tA=0s {M0100644 SZ=0 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.821640 M 1651311710.851224 C 1651311710.851224}}
    11:41:50.875884 rx 1405: GETXATTR n827 {sz 0} ["security.capability"] 20b
    11:41:50.875970 tx 1405:     95=operation not supported
    11:41:50.876070 rx 1406: WRITE n827 {Fh 1 [0 +45)  L 0 WRONLY,0x8000} "Das ist "... 45b
    ino76940493: FUSE Write: offset=0 length=45
    doRead: off=0 len=4096 -> off=18 len=4128 skip=0
    len(oldData)=0 len(blockData)=45
    ino76940493: Writing 45 bytes to block #0
    11:41:50.878446 tx 1406:     OK
    11:41:50.878526 rx 1407: FSYNC n827 
    11:41:50.893228 tx 1407:     13=permission denied
    11:41:50.893393 rx 1408: GETATTR n827 {Fh 0} 
    file.GetAttr()
    11:41:50.893634 tx 1408:     OK, {tA=1s {M0100644 SZ=45 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.851640 M 1651311710.851224 C 1651311710.851224}}
    11:41:50.893748 rx 1409: SETATTR n827 {mode 0100644} 
    11:41:50.894014 tx 1409:     OK, {tA=0s {M0100644 SZ=45 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.851640 M 1651311710.851224 C 1651311710.891224}}
    11:41:50.894115 rx 1410: FLUSH n827 {Fh 1} 
    11:41:50.894182 tx 1410:     OK
    11:41:50.894277 rx 1412: GETATTR n686 {Fh 0} 
    11:41:50.894289 rx 1411: RELEASE n827 {Fh 1 WRONLY,0x8000  L0} 
    11:41:50.894377 tx 1412:     OK, {tA=1s {M0100644 SZ=18 L=1 1000:1000 B16*16384 i0:76940492 A 1651311665.673640 M 1651311159.032983 C 1651311159.032983}}
    11:41:50.894464 rx 1413: GETATTR n827 {Fh 0} 
    11:41:50.894556 tx 1413:     OK, {tA=1s {M0100644 SZ=45 L=1 1000:1000 B0*16384 i0:76940493 A 1651311731.851640 M 1651311710.851224 C 1651311710.891224}}
    11:41:50.894642 rx 1414: UNLINK n1 ["test.txt"] 9b
    11:41:50.898276 tx 1411:     OK
    11:41:50.905820 tx 1414:     OK
    11:41:50.906078 rx 1415: FORGET n827 {Nlookup=1} 
    11:41:50.906180 rx 1416: LOOKUP n1 ["test.txt"] 9b
    11:41:50.912295 tx 1416:     2=no such file or directory, {n0 g0 tE=1s tA=0s {M00 SZ=0 L=0 0:0 B0*0 i0:0 A 0.000000 M 0.000000 C 0.000000}}
    11:41:50.912449 rx 1417: RENAME n1 {i1} ["test.txt~" "test.txt"] 19b
    Renameat 7/tSSLRAc4Qxs5lO354T8Afg -> 8/fSRh8isq8XFiKS2nUsVuLQ
    11:41:50.926613 tx 1417:     OK
    

    Version

    fusermount

    fusermount version: 2.9.9
    

    gocryptfs

    gocryptfs v2.2.1 without_openssl; go-fuse v2.1.1-0.20210825171523-3ab5d95a30ae; 2021-10-20 go1.17.2 linux/amd64
    

    vim

    VIM - Vi IMproved 8.1 (2018 May 18, compiled Dec 25 2021 15:48:51)
    Included patches: 1-875, 878, 881, 883-884, 936, 948, 1046, 1365-1368, 1382, 1401
    Extra patches: 8.2.3402, 8.2.3403, 8.2.3409, 8.2.3428
    
    needs more info 
    opened by yahesh 1
Releases(v2.2.1)
A FileSystem Abstraction System for Go

A FileSystem Abstraction System for Go Overview Afero is a filesystem framework providing a simple, uniform and universal API interacting with any fil

Steve Francia 4.5k Jun 27, 2022
A package to allow one to concurrently go through a filesystem with ease

skywalker Skywalker is a package to allow one to concurrently go through a filesystem with ease. Features Concurrency BlackList filtering WhiteList fi

Will Dixon 80 Jun 18, 2022
An implementation of the FileSystem interface for tar files.

TarFS A wrapper around tar.Reader. Implements the FileSystem interface for tar files. Adds an Open method, that enables reading of file according to i

Eyal Posener 50 May 17, 2022
Takes an input http.FileSystem (likely at go generate time) and generates Go code that statically implements it.

vfsgen Package vfsgen takes an http.FileSystem (likely at go generate time) and generates Go code that statically implements the provided http.FileSys

null 945 May 31, 2022
memfs: A simple in-memory io/fs.FS filesystem

memfs: A simple in-memory io/fs.FS filesystem memfs is an in-memory implementation of Go's io/fs.FS interface. The goal is to make it easy and quick t

Peter Sanford 64 Jun 21, 2022
A Go io/fs filesystem implementation for reading files in a Github gists.

GistFS GistFS is an io/fs implementation that enables to read files stored in a given Gist. Requirements This module depends on io/fs which is only av

Jean Hadrien Chabran 124 May 1, 2022
A Small Virtual Filesystem in Go

This is a virtual filesystem I'm coding to teach myself Go in a fun way. I'm documenting it with a collection of Medium posts that you can find here.

Alyson 31 Apr 18, 2022
CRFS: Container Registry Filesystem

CRFS: Container Registry Filesystem Discussion: https://github.com/golang/go/issues/30829 Overview CRFS is a read-only FUSE filesystem that lets you m

Google 1.2k Jul 1, 2022
Go filesystem implementations for various URL schemes

hairyhenderson/go-fsimpl This module contains a collection of Go filesystem implementations that can discovered dynamically by URL scheme. All filesys

Dave Henderson 225 Jun 27, 2022
A Go filesystem package for working with files and directories

Stowage A Go filesystem package for working with files and directories, it features a simple API with support for the common files and directories ope

null 19 May 28, 2021
filesystem for golang

filesystem filesystem for golang installation go get github.com/go-component/filesystem import import "github.com/go-component/filesystem" Usage sup

null 4 Jul 9, 2021
A set of io/fs filesystem abstractions and utilities for Go

A set of io/fs filesystem abstractions and utilities for Go Please ⭐ this project Overview This package provides io/fs interfaces for: Cloud providers

null 8 Mar 24, 2022
Tarserv serves streaming tar files from filesystem snapshots.

tarserv A collection of tools that allow serving large datasets from local filesystem snapshots. It is meant for serving big amounts of data to shell

Aurora 1 Jan 11, 2022
Grep archive search in any files on the filesystem, in archive and even inner archive.

grep-archive Grep archive search for string in any files on the filesystem, in archive and even inner archive. Supported archive format are : Tar Form

Michel Prunet 0 Jan 26, 2022
Warp across your filesystem in ~5 ms

WarpDrive: the Go version. What does this do? Instead of having a huge cd routine to get where you want, with WarpDrive you use short keywords to warp

Ishan Goel 17 Jun 13, 2022
Encrypted File System in Go

Getting Started: Setup the environment: Install GoLang: $ sudo apt update $ sudo apt upgrade $ sudo apt install libssl-dev gcc pkg-config $ sudo apt

Lucky Verma 0 Apr 30, 2022
A PDF processor written in Go.

pdfcpu: a Go PDF processor pdfcpu is a PDF processing library written in Go supporting encryption. It provides both an API and a CLI. Supported are al

pdfcpu 3.2k Jun 24, 2022
Goful is a CUI file manager written in Go.

Goful Goful is a CUI file manager written in Go. Works on cross-platform such as gnome-terminal and cmd.exe. Displays multiple windows and workspaces.

anmitsu 280 Jun 28, 2022
A simple library for generating PDF written in Go lang

gopdf gopdf is a simple library for generating PDF document written in Go lang. Features Unicode subfont embedding. (Chinese, Japanese, Korean, etc.)

Signin Technology 1.6k Jun 26, 2022