Syncthing is a continuous file synchronization program.

Overview

Syncthing


Latest Linux & Cross Build Latest Windows Build Latest Mac Build MPLv2 License CII Best Practices Go Report Card

Goals

Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals below. The goals are listed in order of importance, the most important one being the first. This is the summary version of the goal list - for more commentary, see the full Goals document.

Syncthing should be:

  1. Safe From Data Loss

    Protecting the user's data is paramount. We take every reasonable precaution to avoid corrupting the user's files.

  2. Secure Against Attackers

    Again, protecting the user's data is paramount. Regardless of our other goals we must never allow the user's data to be susceptible to eavesdropping or modification by unauthorized parties.

  3. Easy to Use

    Syncthing should be approachable, understandable and inclusive.

  4. Automatic

    User interaction should be required only when absolutely necessary.

  5. Universally Available

    Syncthing should run on every common computer. We are mindful that the latest technology is not always available to any given individual.

  6. For Individuals

    Syncthing is primarily about empowering the individual user with safe, secure and easy to use file synchronization.

  7. Everything Else

    There are many things we care about that don't make it on to the list. It is fine to optimize for these values, as long as they are not in conflict with the stated goals above.

Getting Started

Take a look at the getting started guide.

There are a few examples for keeping Syncthing running in the background on your system in the etc directory. There are also several GUI implementations for Windows, Mac and Linux.

Docker

To run Syncthing in Docker, see the Docker README.

Vote on features/bugs

We'd like to encourage you to vote on issues that matter to you. This helps the team understand what are the biggest pain points for our users, and could potentially influence what is being worked on next.

Getting in Touch

The first and best point of contact is the Forum. There is also an IRC channel, #syncthing on freenode (with a web client), for talking directly to developers and users. If you've found something that is clearly a bug, feel free to report it in the GitHub issue tracker.

Building

Building Syncthing from source is easy, and there's a guide that describes it for both Unix and Windows systems.

Signed Releases

As of v0.10.15 and onwards release binaries are GPG signed with the key D26E6ED000654A3E, available from https://syncthing.net/security.html and most key servers.

There is also a built in automatic upgrade mechanism (disabled in some distribution channels) which uses a compiled in ECDSA signature. macOS binaries are also properly code signed.

Documentation

Please see the Syncthing documentation site.

All code is licensed under the MPLv2 License.

Comments
  • Filesystem Notification - Continuation

    Filesystem Notification - Continuation

    This is the continuation of plouj's PR #2807 on a branch in the syncthing repo for ease of maintenance. Refer to that PR for general information what this is about and commits previous to https://github.com/syncthing/syncthing/commit/1d424d84609a40d2a11ac8e06cb87aac6b130629.

    frozen-due-to-age pr-merged 
    opened by imsodin 184
  • Support for file encryption (e.g. non-trusted servers)

    Support for file encryption (e.g. non-trusted servers)

    So I have had a look at BitTorrent sync, syncthing and alternatives and what I always wondered about was the possibility to not only sync between resources I own and trust, but also external resources/servers which I do NOT trust with my data, up to a certain extent.

    One way to do this is using ecryptfs or encfs, but this has many obvious downsides: it is not an interoperable solution (only works on Linux), the files are actually stored in encrypted form on the disk (even if the resource is trusted and this is not necessary, for instance because of the file system being encrypted already), etc.

    What I propose is somehow configuring nodes which are only sent the files in an encrypted format, with all file contents (and potentially file/directory names as well; or even permissions) being encrypted. This way, if I want to store my private files on a fast server in a datacenter to access them from anywhere, I could do this with syncthing without essentially giving up ownership of those files. I could also prevent that particular sync node from being allowed/able to make any changes to the files without me noticing.

    I realize that this requires a LOT of additional effort, but it would be a killer feature that seems to not be available in any other "private cloud" solution so far. What are your thoughts on this feature?

    EDIT: BitTorrent sync mentions a feature like this in their API docs: "Encryption secret API users can generate folder secrets with encrypted peer support. Encryption secrets are read-only. They make Sync data encrypted on the receiver’s side. Recipients can sync files, but they can’t see file content, and they can’t modify the files. Encryption secrets come in handy if you need to sync to an untrusted location." (from http://www.bittorrent.com/intl/de/sync/developers/api)

    enhancement 
    opened by Natanji 143
  • all: Store pending devices and folders in database (fixes #7178)

    all: Store pending devices and folders in database (fixes #7178)

    Purpose

    As discussed in #5758 and mentioned on the forum, storing the pending (offered from remote but not yet added locally) devices and folders in the XML configuration is not a nice and scalable design. Instead, the information should live in the database, properly structured, and made available over dedicated API endpoints.

    This is also ground work and practice to finally approach an acceptable implementation of the prototype in #5758, extending the concept to other devices we have heard about in ClusterConfig messages. That is deliberately saved for another PR though.

    Testing

    All code paths have undergone extensive manual testing within an existing setup of four instances (one compiled from this PR). Especially regarding the clean-up of pending entries upon config changes, both online through the POST /rest/system/config REST API and offline by modifying the XML while Syncthing is not running. Notifications on the GUI look as before, API endpoints have been verified with curl.

    ~~Unit tests would be rather complicated and were discussed as not strictly needed considering how low the importance of the handled information is.~~ Unit tests included, with a few basic test cases.

    Documentation

    The envisioned API for this stuff is described in https://github.com/syncthing/docs/pull/498.

    The commit messages contain much of the rationale for each change, so I'd be happy to keep them intact instead of squash-merging in a single commit. If desired, I can rebase to clean out obvious fixup commits and end up with a nice patch series of self-contained changes.

    Breaking Changes For The User

    Existing <pendingDevice> / <pendingFolder> entries in the XML config are not carried over to the database. The corresponding notifications will disappear until the next connection attempt with the offering device. It has been discussed that the low importance of this information does not warrant a separate logic for config --> database info migrations. One possibility to minimize disruption would be to just keep the XML entries intact for some releases and hold back on commit 8ae24612397fea6182cfb3d6dcce592c78c6df5d until then.

    frozen-due-to-age 
    opened by acolomb 109
  • Option to not sync mtime

    Option to not sync mtime

    Trying to sync my android device and my NAS(armv5) both using syncthing v0.10.0:

    On the ntfs mount of the NAS the file /my/directory/.syncthing.example.test is created then I get:

    puller: final: chtimes /my/directory/.syncthing.example.test: invalid argument

    File /my/directory/example.test is not created. Syncthing continues with the next file but the same error happens for all the files in the directory.

    enhancement frozen-due-to-age 
    opened by otrag 108
  • GUI display of global changes

    GUI display of global changes

    Purpose

    This PR adds a button to the device section of the GUI that the user can click on to see all filesystem changes that originated/occurred on that machine. That is to say, not changed copied from the network. This should help people discover more easily what computer made a certain change that was propagated in larger P2P swarms.

    Screenshots

    image

    frozen-due-to-age 
    opened by nrm21 107
  • Folder isn't making progress

    Folder isn't making progress

    All nodes are on v0.10.2 GUI says

    9:35:58: Folder "default" isn't making progress - check logs for possible root cause. Pausing puller for 1m0s.
    9:36:41: Folder "PIA" isn't making progress - check logs for possible root cause. Pausing puller for 1m0s.
    

    In the log I see the "hash mismatch" info but the file is not changed during pull. It was probably the whole night in that state but i don't have more logs, I should probably increase the size... logs from the pulling node: http://alex-graf.de/public/st/no-progress.tar.gz

    Restarting also does not fix it

    opened by alex2108 104
  • gui: add advance config port mapping to gui

    gui: add advance config port mapping to gui

    Purpose

    https://github.com/syncthing/syncthing/issues/4824

    Gives the user the opportunity to display a link to web gui remote machines. The user can choose the port and set if the link will display.

    Testing

    I just check to see if the link was shown depending on the setting.

    Screenshots

    Screen Shot 2020-09-29 at 11 13 53 PM Screen Shot 2020-09-29 at 11 34 37 PM

    Documentation

    https://github.com/syncthing/docs/pull/573

    Authorship

    Your name and email will be added automatically to the AUTHORS file based on the commit metadata.

    frozen-due-to-age 
    opened by rjpruitt16 91
  • lib/connections: Add KCP support (fixes #804)

    lib/connections: Add KCP support (fixes #804)

    This is mostly for benchmarks and testing.

    Seems to connect, and not crash immediately, which is good news.

    Known issues:

    1. No timeouts as part of the protocol, so we usually end up with a dead connection for 5-7.5 minutes until it times out. This could be addressed by always using the new connection that we get, as long as the priority is right
    2. Has some weirdness around closing connections. See https://github.com/xtaci/kcp-go/issues/18

    The diff is massive due to deps, so I suggest you pull the PR locally and diff inside the lib and cmd/syncthing dirs.

    frozen-due-to-age pr-merged 
    opened by AudriusButkevicius 91
  • Memory and CPU usage is prohibitively high

    Memory and CPU usage is prohibitively high

    I've been testing syncthing across 3 machines, a laptop with 8GB of RAM and two NAS-style servers with 1GB and 2GB of RAM. My repositories have the following sizes:

    • 19 items, 536 KiB
    • 83471 items, 16.2 GiB
    • 181482 items, 387 GiB

    To sync these three repositories syncthing 0.9.0 uses a bit over 700MB of RAM and while syncing continuously pegs the CPU at 150% on all nodes.

    While the CPU usage I could manage during the initial sync the memory usage is simply too high. A typical NAS server like the two I have holds >4TB of storage. At the current level of usage that would require ~8GB of memory just for syncthing.

    Without looking at the code I assume an index is being kept in memory for all the repository contents. 700MB is 2.6kb per item on disk, which seems way too high. The index should only really need to store filename, parent, permissions, size and timestamp. On average (depends on filename sizes) that should only be 50-60 bytes per item which would only be 13MB. Moving that index to disk would also make a lot more sense.

    I assume the CPU usage is hashing lots of files. There I assume using librsync might be a better option.

    enhancement frozen-due-to-age 
    opened by pedrocr 86
  • /rest/system/status returns empty response during startup

    /rest/system/status returns empty response during startup

    Syncthing version 0.12.20 (although it's been around since at least the beginning of February).

    During startup, the /rest/system/status endpoint can return an empty response (seems to happen more on some devices than others).

    The headers returned are:

      Access-Control-Allow-Origin: *
      Pragma: no-cache
      X-Syncthing-Id: IDIDIID-IDIDID-IDIDID-IDIDID-IDIDID-IDIDID-IDIDID-IDIDID
      X-Syncthing-Version: v0.12.20
      Cache-Control: no-store, no-cache, max-age=0
      Date: Sat, 19 Mar 2016 22:40:58 GMT
      Content-Length: 0
      Content-Type: application/json; charset=utf-8
      Expires: Sat, 19 Mar 2016 22:40:58 GMT
    

    Of particular interest is the Content-Length: 0. I had a quick look through the code but couldn't see why it might be doing this. I fetch the /rest/system/version resource milliseconds afterwards and that returns the correct data.

    Thanks!

    frozen-due-to-age 
    opened by canton7 82
  • Use protocol buffers instead of custom XDR

    Use protocol buffers instead of custom XDR

    Not done, not for merging, just for review and discussion.

    This changes our serialization code to use protocol buffers instead of our custom code. Todo/done:

    • [X] Protocol structs should be protobuf
    • [X] Database structs should be protobuf
    • [x] Local discovery structs should be protobuf by humans?
    • [x] Clean up bitfields vs booleans (as per Audrius stuff)
    • [x] Remove currently unused fields as these can be added painlessly in the future

    Changes I think we should not do:

    • The global discovery protocol. No need, might be nice to have human readable, no real reason to cause the disturbance.
    • The relay protocol, as far as it concerns speaking to a relay. No reason to break it currently and leaving it unchanged lets us reap the benefits of the deployed infrastructure.

    Some changes that were necessary but unexpected or non-obvious:

    • ~~The protobuf serializer uses a Size() method which conflicted with our Size fields on BlockInfo. I've renamed it to Length.~~
    • I've added a Size field instead of CachedSize to FileInfo and let this be serialized everywhere. It means we get it for free in the TruncatedFileInfo instead of having to extract it from the block list.
    • ~~The TruncatedFileInfo type is "officialized" and moved into the protocol package, but it's still mostly used by the database. I'm not sure of this, maybe it can be undone and moved back to db as we have some protobufs stuff there as well (but didn't when I first made this change)...~~
    • The version vector type Vector []Counter definition doesn't work with protobuf so I had to make it a struct which resulted in a lot of tedious changes everywhere, but it's conceptually the same.
    • I added FileName() and FileLength() methods to the interface that covers both FileInfo and FileInfoTruncated so we can get the name or size of the interface without having to type assert to the underlying type.
    • I threw out the conversion from 0.12 db:s. We'll need full rescan for this change, but on the other hand it ought to be the last time ever.
    frozen-due-to-age 
    opened by calmh 79
  • Unknown Device after suspend

    Unknown Device after suspend

    After waking up my laptop from suspension, the web UI will fail to identify this device (XPS) and show it as "unknown device," but also show the correct information of this device in the Remote Devices tab. There are no errors in the JS console.

    image

    This issue can always be fixed using a systemctl restart [email protected] reboot, but always show up after waking from suspension.

    image

    Include required information

    Please be sure to include at least:

    Syncthing Version: syncthing v1.22.1 "Fermium Flea" (go1.19.3 linux-amd64) [email protected] 2022-11-03 07:35:59 UTC [noupgrade] Browser Version: Google Chrome 108.0.5359.124 OS: image

    bug needs-triage 
    opened by hykilpikonna 1
  • Container start-time volume check

    Container start-time volume check

    Purpose

    Opt-in to check if mapped volume contains configuration file. If mapped volume is mounted from remote host, it may be temporarily unavailable during boot time. Which the optional NOCREATE environment variable entrypoint script checks if config.xml is available; if not - container exits for Docker to handle restarting periodically until configuration appears (volume mounts).

    Testing

    Tested on local setup: container keeps restarted by Docker until mounted volume with data (config/config.xml) becomes available.

    Documentation

    Related paragraph added to README.

    opened by yattengate 3
  • build(deps): bump github.com/shirou/gopsutil/v3 from 3.22.11 to 3.22.12

    build(deps): bump github.com/shirou/gopsutil/v3 from 3.22.11 to 3.22.12

    Bumps github.com/shirou/gopsutil/v3 from 3.22.11 to 3.22.12.

    Release notes

    Sourced from github.com/shirou/gopsutil/v3's releases.

    v3.22.12

    Important notice

    v3.22.11 was reverted because some issue by #1384, and retracted by #1386.

    What's Changed

    cpu

    disk

    host

    load

    Other Changes

    New Contributors

    Full Changelog: https://github.com/shirou/gopsutil/compare/v3.22.10...v3.22.12

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies go 
    opened by dependabot[bot] 0
  • Incorrect local state when using `!` negated ignore patterns combined with parent folder ignored

    Incorrect local state when using `!` negated ignore patterns combined with parent folder ignored

    I believe this is likely responsible for the long-term issue I've been experiencing since basically forever which leads to non-matching local and remote folder states despite using the exact same ignore patterns on both sides.

    1. Share an empty folder between Device 1 (D1) and Device 2 (D2).
    2. Set the folder to "Send & Receive" on both sides.
    3. Add the following ignore patterns to the folder on both sides.
      !*.txt
      /
      
    4. The folders on both sides should now look as follows. image image
    5. Create the following hierarchy in the folder on D1.
      /folder
      /folder/file.txt
      
    6. Scan the folder on D1. Wait for the changes to sync to D2.
    7. The final state looks as follows. image image
    8. As you can see, the folder state on D1 is incorrect. Because file.txt is present inside /folder, the folder should be included in the local state as well, however in reality only the file is. Once this happens, neither rescanning the folder nor restarting Syncthing repairs the broken local state.

    There is a caveat though. The steps above aren't complete. I've just managed to reproduce this in my test environment and I'm attaching logs with model,db enabled below, however in my testing I was repeatedly creating and removing folders and files in order to find the culprit. While doing so, I eventually managed to reach the state as seen above, but I can't reproduce it at will directly yet.

    The logs should contain the relevant information though. They aren't very large either, as the whole testing took less than 10 minutes or so.

    syncthing1.log syncthing2.log

    For the record, I think the actual synchronisation still functions fine despite the incorrect local states. However, I've encountered problems with Receive Only folders being stuck in a permanent "Out of Sync" state which I believe is likely related to this very bug. I still haven't been able to reproduce that one in my test environment though.

    bug needs-triage 
    opened by tomasz1986 6
  • gui: add xattr filter editor (fixes #8660)

    gui: add xattr filter editor (fixes #8660)

    Purpose

    Added a GUI element in the advanced folder configuration panel to be able to manipulate the extended attribute filter settings.

    I intended to keep it as simple as possible, no automatic additions or other magic behind the screens. Mostly because I expect that the users who will use this will know what they are doing. But also because I'm not a fan myself of automatically added rules/lines when manipulating settings, unnecessary complexity. However, some information will still be shown when appropriate;

    • Default behaviour if a wild-card/any is not present as last element
    • A reminder in case that only deny-rules exist while the default in that case is deny - suggesting the addition of a permit-any rule in the end.

    Otherwise empty rules are being cleaned up on saving the configuration and new entries are being pushed before a wild-card/any rule, if that one is present as last element.

    Any further logic/validation is not present, on purpose.

    Testing

    • Manipulate a folder's xattr filter using the new part in the GUI -> check result in the config file.
    • Manipulate the default folder configuration -> add a new folder -> verify that the settings are correct for the newly added folder (and default settings).

    Screenshots

    Screenshot 2022-12-28 at 11 51 46 Screenshot 2022-12-28 at 11 51 54 Screenshot 2022-12-28 at 11 52 03

    Documentation

    Supporting documentation is prepared as well, https://github.com/syncthing/docs/pull/779

    opened by er-pa 3
  • Ignore patterns combining `/**/` and `{a,b}` alternatives not matching as expected

    Ignore patterns combining `/**/` and `{a,b}` alternatives not matching as expected

    For reference: https://forum.syncthing.net/t/confusing-behaviour-when-trying-to-ignore-direct-and-indirect-child-files-involving-curly-bracketed-alternatives/19536

    1. Create the following folder structure
    Documents/document.txt
    Documents/Photos/document.txt
    
    1. Add Documents/**/document.txt to ignore patterns. Both files have been ignored.
    2. Replace the patterns with Documents/**/{document.txt}. Only Documents/Photos/document.txt has been ignored.
    3. Replace the patterns with Documents{/**/}{document.txt}. Now both files have been ignored again.

    The behaviour doesn't seem consistent because a) both files should be ignored in 3), and b) there should be no difference between 3) and 4) while in reality the two patterns behave differently.

    For the record, tested under Windows 10 and Syncthing v1.22.2.

    bug needs-triage 
    opened by tomasz1986 0
Releases(v1.23.0)
Owner
The Syncthing Project
The Syncthing Project
oDrop, a fast efficient cross-platform file transfer software for server and home environments

oDrop is a cross-platform LAN file transfer software to efficiently transfer files between computers, oDrop is useful in environments where GUI is not available.

Flew Software 16 Jun 4, 2022
JuiceFS is a distributed POSIX file system built on top of Redis and S3.

JuiceFS is an open-source POSIX file system built on top of Redis and object storage

Juicedata, Inc 7.2k Jan 5, 2023
fsync - a file sync server

fsync - a file sync server

null 5 Aug 25, 2022
Yet another netcat for fast file transfer

nyan Yet another netcat for fast file transfer When I need to transfer a file in safe environment (e.g. LAN / VMs), I just want to use a simple comman

null 6 Apr 30, 2022
A web based drag and drop file transfer tool for sending files across the internet.

DnD A file transfer tool. Demo Usage Get go get github.com/0xcaff/dnd or download the latest release (you don't need go to run it) Run dnd Now navig

Martin Charles 20 Dec 16, 2022
transfer.sh - Easy and fast file sharing from the command-line.

Easy and fast file sharing from the command-line. This code contains the server with everything you need to create your own instance.

Dutchcoders 13.6k Jan 2, 2023
Easy and fast file sharing from the command-line.

Easy and fast file sharing from the command-line. This code contains the server with everything you need to create your own instance.

Dutchcoders 13.6k Jan 2, 2023
Golang PoC software for reliable file transfers over a data diode. DIY gigabit data diode hardware instructions

DIY Data Diode Simple DIY gigabit data diode (hardware and software). Presented at SEC-T 2021. Hardware By doing a simple hardware mod to a fiber conv

Klockcykel 21 Dec 1, 2022
aqua is a simple file uploading and sharing server for personal use.

aqua is a simple file uploading and sharing server for personal use. It is built to be easy to set up and host on your own server, for example to use it in combination with uploading tools like ShareX.

Tobias B. 14 Jul 7, 2022
Delta : File Sharing system for golang

delta is File Sharing system its good for Local networks or small teams Cross-platform delta runs anywhere Go can compile for: Windows, Mac, Linux, AR

null 0 Nov 29, 2021
Built in user interface, LAN file transfer, such as mobile phone, computer, tablet, different operating system

Modao Built in user interface, LAN file transfer, such as mobile phone, computer, tablet, different operating systems, etc., as well as text transfer

null 0 May 7, 2022
Subspace - File sharing application for golang

subspace File sharing application. Supported Platforms OS 386 amd64 arm6 arm64 L

Jim Male 1 Jan 29, 2022
Rsync - rsync (File syncing) in golang

Go rsync Minimal file syncing based on the rsync algorithm completely written

Emmanuel Garcia 1 Jun 27, 2022
FileTransferGo - File Transfer With Golang

FileTransferGo Packages used ?? Go: Gin (http server) ?? Cobra (CLI command fram

PiterDev 3 Jun 24, 2022
mini file transfer tool, use it just curl o wget

miniTransfer mini file transfer tool, use it just curl o wget How to use upload file curl -T localFileName 127.0.0.1:1234 # default save as localFileN

chenlianghong 0 Jan 12, 2022
peer-to-peer file sharing

what i want is a tool to use to send files my many virtual machines. I want to do this myself, and i want to make it work as expected. So maybe a daem

Joseph Attah 0 Jun 13, 2022
wholeaked is a file-sharing tool that allows you to find the responsible person in case of a leakage

wholeaked is a file-sharing tool that allows you to find the responsible person in case of a leakage

Utku Sen 749 Dec 25, 2022
Simple temporary file upload and transfer web application coding with Go language.

Temp File Transfer Web Application Simple temporary file upload and transfer web application coding with Go language. Explore the Golang » Live Demo T

Alper Reha 0 Dec 2, 2022
Open Source Continuous File Synchronization

Goals Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals belo

The Syncthing Project 48.4k Dec 31, 2022
Open Source Continuous File Synchronization

Goals Syncthing is a continuous file synchronization program. It synchronizes files between two or more computers. We strive to fulfill the goals belo

The Syncthing Project 48.6k Jan 9, 2023