📦 Command line peer-to-peer data transfer tool based on libp2p.

Related tags

Network pcp
Overview

pcp - Peer Copy

standard-readme compliant Go Report Card Maintainability Test Coverage

Command line peer-to-peer data transfer tool based on libp2p.

Demo animation

Table of Contents

Motivation

There already exists a long list of file transfer tools (see Related Efforts), so why bother building another one? The problem I had with the existing tools is that they rely on a limited set of servers to orchestrate peer matching and data relaying which poses a centralisation concern. Many of the usual centralisation vs. decentralisation arguments apply here, e.g. the servers are single points of failures, the service operator has the power over whom to serve and whom not, etc. Further, as this recent issue in croc shows, this is a real risk for sustainable operation of the provided service. A benevolent big player jumped in to sponsor the service.

Project Status

The tool is in a very early stage, and I'm aware of performance, usability and security issues. Don't use it for anything serious. Although I criticised tools like magic-wormhole or croc above, they are amazing and way more mature.

There are also drawbacks with this approach: It's slower than established centralised methods if you want to transmit data across network boundaries. A DHT query to find your peer can easily take several minutes. Further, the bandwidth and geographic location of a potential relaying peer is not guaranteed which can lead to long transmission times.

How does it work?

When running pcp send a new peer identity is generated. The first bytes of the public key are encoded in four words from the Bitcoin improvement proposal BIP39, so each word is basically random among the 2048 possible words. The first word is interpreted as a channel ID in the range from 0 to 2047. pcp advertises in its local network via mDNS and in the DHT of IPFS the identifier /pcp/{unix-timestamp}/channel-id. The unix timestamp is the current time truncated to 5 minutes and the prefix /pcp is the protocol prefix.

Your peer enters pcp receive four-words-from-above and pcp uses the first word together with the current time truncated to 5 minutes to find the sending peer in the DHT and in your local network via mDNS. When the peer is found, its public key is checked against the three remaining words (as the words were derived from that key), and a password authenticated key exchange happens to authenticate each other. The key exchange is not used for encryption as the connection uses TLS per default. After the peer is authenticated the receiver must confirm the file transfer, and the file gets transmitted.

Usage

The sending peer runs:

$ pcp send my_file
Code is:  bubble-enemy-result-increase
On the other machine run:
	pcp receive bubble-enemy-result-increase

The receiving peer runs:

$ pcp receive bubble-enemy-result-increase
Looking for peer bubble-enemy-result-increase...

If you're on different networks the lookup can take quite long (~ 2-3 minutes). Currently, there is no output while both parties are working on peer discovery, so just be very patient.

Install

Release download

Head over to the releases and download the latest binary for your platform.

From source

To compile it yourself clone the repository:

git clone https://github.com/dennis-tra/pcp.git

Navigate into the pcp folder and run:

go install cmd/pcp/pcp.go # Go 1.13 or higher is required

Make sure the $GOPATH/bin is in your PATH variable to access the installed pcp executable.

Package managers

It's on the roadmap to distribute pcp via apt, yum, brew, scoop and more ...

Development

Protobuf definitions

First install the protoc compiler:

make tools # downloads gofumpt and protoc
make proto # generates protobuf

The current proto definitions were generated with libprotoc 3.14.0.

Feature Roadmap

Shamelessly copied from croc:

  • allows any two computers to transfer data (using a relay)
  • provides end-to-end encryption (using PAKE)
    • actually PAKE is only used for authentication TLS for end-to-end encryption
  • enables easy cross-platform transfers (Windows, Linux, Mac)
    • 🤔 only tested Linux <-> Mac
  • allows multiple file transfers
    • not yet
  • allows resuming transfers that are interrupted
    • not yet
  • local server or port-forwarding not needed
  • ipv6-first with ipv4 fallback
    • 🤔 I think that's the case, but I'm not sure about the libp2p internals
  • can use proxy, like tor
    • not yet

Related Efforts

  • croc - Easily and securely send things from one computer to another
  • magic-wormhole - get things from one computer to another, safely
  • dcp - Remote file copy, powered by the Dat protocol.
  • iwant - CLI based decentralized peer to peer file sharing
  • p2pcopy - Small command line application to do p2p file copy behind firewalls without a central server.
  • zget - Filename based peer to peer file transfer
  • sharedrop - Easy P2P file transfer powered by WebRTC - inspired by Apple AirDrop
  • filepizza - Peer-to-peer file transfers in your browser
  • toss - Dead simple LAN file transfers from the command line
  • Forgot yours? Open an issue or submit a PR :)

Maintainers

@dennis-tra.

Acknowledgment

  • go-libp2p - The Go implementation of the libp2p Networking Stack.
  • pake/v2 - PAKE library for generating a strong secret between parties over an insecure channel
  • progressbar - A really basic thread-safe progress bar for Golang applications

Contributing

Feel free to dive in! Open an issue or submit PRs.

Other Projects

You may be interested in one of my other projects:

  • image-stego - A novel way to image manipulation detection. Steganography-based image integrity - Merkle tree nodes embedded into image chunks so that each chunk's integrity can be verified on its own.

License

Apache License Version 2.0 © Dennis Trautwein

Issues
  • Enhance hole punching

    Enhance hole punching

    We can try to establish direct connection before transfer (while hole-punching is going on background). I also enable libp2p "p2p-holepunching" log when -debug flag set, so we can see why hole punching fail (in my case it will mostly timeout with the default dial timeout value).

    opened by optman 6
  • brew package conflict with PCP - Perfomance CoPilot

    brew package conflict with PCP - Perfomance CoPilot

    the package entry in brew conflicts with Performance CoPilot.

    See https://github.com/performancecopilot/pcp/blob/main/INSTALL.md "section 2. Mac OS X Installation" for more information.

    opened by ciphernaut 10
  • [Fedora 33] Installation via .rpm written over during system upgrade

    [Fedora 33] Installation via .rpm written over during system upgrade

    When upgrading the system using the command sudo dnf upgrade, the pcp package is written over by this package

    Last metadata expiration check: 5:27:05 ago on Fri 12 Mar 2021 07:59:14 PM EST.
    Available Packages
    Name         : pcp
    Version      : 5.2.5
    Release      : 2.fc33
    Architecture : x86_64
    Size         : 1.3 M
    Source       : pcp-5.2.5-2.fc33.src.rpm
    Repository   : updates
    Summary      : System-level performance monitoring and performance management
    URL          : https://pcp.io
    License      : GPLv2+ and LGPLv2+ and CC-BY
    Description  : Performance Co-Pilot (PCP) provides a framework and services to
                 : support system-level performance monitoring and performance
                 : management.
                 : 
                 : The PCP open source release provides a unifying abstraction for
                 : all of the interesting performance data in a system, and allows
                 : client applications to easily retrieve and process any subset of
                 : that data.
    
    

    which renders this pcp no longer accessible or installed (not quite sure what happens to the package). Current workaround is to uninstall that pcp using sudo dnf remove pcp and then re-install this pcp with standard software installation from either command line of Gnome software.

    Perhaps changing the distribution name of this package to something along the lines of pcp-filetransfer (?) would fix this issue, at least on Fedora.

    opened by AmberCronin 3
  • Simplify language

    Simplify language

    Dear @dennis-tra, consider providing alternative for receive (or is it recieve, or recive?) command. Simple get seems to work much better for those folks like me whose native language is not English, or even omit this step (as croc does) with even simpler pcp <code>.

    opened by sergeevabc 1
Releases(v0.4.0)
Owner
Dennis Trautwein
BSc./MSc. in extraterrestrial/solid state physics | iOS/WebDev WellingtonNZ/MelbourneAU | [email protected] HamburgGER | [email protected] CH
Dennis Trautwein
An imageboard, but images are stored in a peer-to-peer network

Interplanetary File Dumpster An imageboard, but images are stored in a peer-to-peer network Features: Easy file sharing without registration and SMS.

George 93 Jun 25, 2022
DeepValueNetwork is a peer-to-peer database network managed and hosted by its community.

DeepValueNetwork To understand what DeepValueNetwork will be, I suggest you read this document. In progress This software is currently being developed

DeepValue Network 337 May 29, 2022
Group peer to peer video calls for everyone written in Go and TypeScript

Peer Calls v4 WebRTC peer to peer calls for everyone. See it live in action at peercalls.com. The server has been completely rewriten in Go and all th

Peer Calls 1.1k Jun 26, 2022
gopunch is a go implementation of a peer-to-peer chat service built using UDP hole punching.

Gopunch gopunch is a go implementation of a peer-to-peer chat service built using UDP hole punching. This is a toy implementation that I put together

Peyton Walters 10 May 24, 2022
Peer-to-peer hypermedia protocol

IPFS powers the Distributed Web A peer-to-peer hypermedia protocol to make the web faster, safer, and more open. TL;DR Get help and talk about ideas i

IPFS 20.9k Jun 28, 2022
Steve - A peer-to-peer (p2p) decentralized network

Steve Steve is a peer-to-peer (p2p) decentralized network that enables people to

Steve Care Software Inc 4 Feb 5, 2022
Zero Trust Network Communication Sentinel provides peer-to-peer, multi-protocol, automatic networking, cross-CDN and other features for network communication.

Thank you for your interest in ZASentinel ZASentinel helps organizations improve information security by providing a better and simpler way to protect

ZTALAB 5 May 25, 2022
P2P Forwarder - a tool for farwarding tcp/udp ports. Made using libp2p.

P2P Forwarder A tool for farwarding ports. Made using libp2p. How it works A: opens desired ports ports inside P2P Forwarder A: shares it's id from P2

null 23 Jan 14, 2022
HTTP based Tree-shaped Peer2Peer blob transfer proxy, distributing images or blob data.

DadiP2P DadiP2P is an accelerator that uses P2P protocol to speed up HTTP file download, usually use for docker image layer download. The key features

null 8 May 1, 2022
A Lightweight VPN Built on top of Libp2p for Truly Distributed Networks.

Hyprspace A Lightweight VPN Built on top of Libp2p for Truly Distributed Networks. demo.mp4 Table of Contents A Bit of Backstory Use Cases A Digital N

Hyprspace 282 Jun 29, 2022
🌌 A libp2p DHT crawler that gathers information about running nodes in the network.

A libp2p DHT crawler that gathers information about running nodes in the network. The crawler runs every 30 minutes by connecting to the standard DHT bootstrap nodes and then recursively following all entries in the k-buckets until all peers have been visited.

Dennis Trautwein 83 Jul 2, 2022
RPC over libp2p pubsub with error handling

go-libp2p-pubsub-rpc RPC over libp2p pubsub with error handling Table of Contents Background Install Usage Contributing Changelog License Background g

textile.io 7 Apr 17, 2022
libp2p implementation in Go

The Go implementation of the libp2p Networking Stack. Table of Contents Background Usage API Examples Development Using the go-libp2p Workspace About

libp2p 4.6k Jun 29, 2022
Transport to allow go-libp2p applications to natively use i2p for communication

I2P Transport for go-libp2p This library can be used to build go-libp2p applications using the i2p network. Look at transport_test.go for example usag

null 2 Jun 15, 2022
A simple toy example for running Graphsync + libp2p.

graphsync-example Here we outline a simple toy example (main.go) where two local peers transfer Interplanetary Linked Data (IPLD) graphs using Graphsy

Myel 1 Dec 8, 2021
Libp2p chat with discovery and pubsub

Dicovery - pubsub chat with libp2p How to test Run boostrap node $ go run main/main.go --port 35005 --nick boot --pk XDLjuaVJ2yKQ2zHMmsee5PGHtDHmkkvFA

Igor Crevar 0 Jan 14, 2022
A simple port forward tools build on libp2p with holepunch support.

p2p-tun A simple port forward and tun2socks tools build on libp2p with holepunch support. Usage NAME: p2p-tun - port forward and tun2socks through

null 6 May 3, 2022
Client-Server tcp-based file transfer application in GoLang

Клиент-серверный файловый сервис на базе протокола TCP Клиент client.go шифрует свои файлы алгоритмом AES с режимом CBC и помещает их на сервер server

null 1 Apr 1, 2022
Command-line tool and library for Windows remote command execution in Go

WinRM for Go Note: if you're looking for the winrm command-line tool, this has been splitted from this project and is available at winrm-cli This is a

Brice Figureau 365 Jun 19, 2022