We can now run the program and specify log level via arg or env :

• ENV LOG=debug go run . start
• go run . -log=debug start

3 levels available : trace, debug and info

• trace & debug for dev purpose with slow but clean logs
• info for prod with fast json encoded logs

Also, this PR will try to replace every fmt.print based logs with the zerolog lib, and will log all the silenced errors, at the cost that during the tests for exemple, too many lines might be displayed.

• Confirm that a Myel node can be used as a bootstrap node (I think the DHT server mode is enabled by default but I might be wrong so you can double check by connecting 2 nodes to a 3rd Myel node and see if they discover each other.
• Use a connection gater to prevent Myel nodes from connecting to IPFS nodes. Two ways to go about it would be either checking the user agent header or the protocols a peer supports.

How could Myel providers who run a Pop on their home devices easily generate a SSL certificate so clients can retrieve over WSS

Benefits: Each swarm could use its own SSL certificates to ensure :

1. A possible communication of peers using web browsers with the Providers
2. A safe communication channel

Problem:

1. Deal with ACME DNS challenge
2. Might be too SPOF

Hints :

• DNS provider modules in Caddy 2
• Mitigate the SPOF by using

Currently there is very little logs and most implemented with fmt.Print* we need a performant logging solution that can also save logs to disk when running on remote machines. Also support different levels of logging. https://github.com/ipfs/go-log might be convenient otherwise https://github.com/go-kit/kit/tree/master/log is nice as well. We can discuss.

When it is possible to detect file type with the extension name, we should select an appropriate chunking strategy. This improves deduplication, data transfer speed and makes the network overall more efficient. Here's some general guidelines:

• Audio and video content should have trickle layout and chunk sizes of 1MB.
• Images, compressed archives (.zip etc), size splitter with 1MB chunks, balanced layout.
• Text, JSON etc. Buzhash chunker with balanced layout and 16kb chunks for best deduplication. We can probably experiment with different params but this seems to be reasonable efforts.

We need nodes with "testing" roles in that they are capable of autonomously:

• Sending files to be cached by other peers.
• Retrieving files from peers at random time intervals.
• Logging and displaying stats from these actions.

The nodes should be capable of running multiple test scenarios.

eg. nodes send File A 99% of the time to emulate the coverage of a "popular"/ oft requested file. They send File B 1% of time and then gather stats that compare coverage and retrieval performance for A and B.

eg. nodes have a library of files in order of increasing size. They collect stats on pushing / retrieval times relative to file size.

This seems like a better way for waiting for a transfer without having to do gymnastics to check it is the correct transfer. @gallexis maybe try this way on the replication dispatch (replication.go:141)

This adds a new cli tool for running benchmarks against a JS client running in a headless chrome browser. The cli offers 2 different test modes:

• e2e: starts a pop node as provider, adds all the content in the given directory and retrieves it all in parallel
• daemon: starts a service worker client in a headless browser and a cli command server listening for get commands. Each get command navigates to the given URL loading the content from the service worker client during a shared browser session.

@alexander-camuto you will need to set the replication factor to the exact number of nodes you'd like to dispatch too. Make sure they're connected otherwise it might hang for a while.

Now if a github secret is provided, the server will listen for webhook requests and trigger the upgrade handler. This means you can send requests to <name.myel.zone>/upgrade and if activate, the node should auto upgrade. TODO:

• [x] We're still missing a function to call the process to restart itself. @alexander-camuto I've seen some libraries do that if you wanna look into it.

• allows for the deployment of a global CDN
• a core number of nodes are run on regular EC2 on-demand instance -- these form the backbone of the CDN
• scaling the CDN is then done using volatile aws spot / excess compute
• master node autoscales CDN as requests increase i.e deploys more or less worker nodes
• master node provides a monitoring dashboard to track usage / performance
• worker nodes all share same FIL private key

• Add the ability to make requests for CIDs from arbitrary IPFS gateways eg. bcli get ipfs.com cid
• Log TTFB and transfer time as when fetching from pop nodes

• We currently use ngrok as a NAT hole punching solution. Although easy to use this introduces a myriad of setbacks:
  • ngrok servers are not distributed geographically, so we really take a hit in performance
  • ngrok code is proprietary so its hard to figure out what exactly is going on behind the scenes
  • ngrok relays / servers can't act as providers on our network -- which is a missed opportunity

In light of the PL project flare developments we should consider rolling out their relay-circuit v2 implementation for the public nodes on our networks (i.e those not behind a NAT).

Some brief notes on how to implement this:

• Use autonat to determine if a node is behind a NAT or not -- if not, automatically promote a node to a relay.
• Multi-address can contain information as to whether a peer requires a connection via a relay or not -- and through which relay.
• Relays can see requests for content and would determine if they themselves should cache the content to boost performance and avoid the messaging roundtrips hole-punching requires -- this would be a first step in introducing a performance boosting hierarchy to the network.
• Baking in support for WebRTC would remove the need for the DNS records we have to maintain atm -- which would make it easier to onboard new providers (as WebRTC is currently the only protocol that can perform hole-punching browser side)

• Currently providers can claim they already have a CID locally when receiving a caching request. This provides a vector of attack whereby malicious providers could lie about the content they have locally to intentionally reduce the replication factor / redundancy of specific pieces of content.

• A simple solution to this is for the provider to back up this claim using a new hash of the content DAG (eg. keccak / sha-3) that the CID alone wouldn't provide -- serving as a simple proof that the provider does have the content.

  • the problem is that the provider can hold content, hash it, and subsequently delete it, but store the hash to respond to subsequent requests.
  • a potential solution is to use a keyed hash function, whereby a node sending caching requests also includes a randomly generated key as a payload. The nodes responding then have to hash the DAG using that key, providing a proof that, at least at that given point in time, they did actually hold the content.

Failed in CI with output:

{"level":"error","error":"No state for /1626167599029: datastore: key not found","time":"2021-07-13T09:13:20Z","message":"attempting to configure data store"}
2021-07-13T09:13:20.035Z	ERROR	fsm	fsm/fsm.go:92	Executing event planner failed: Invalid transition in queue, state `0`, event `4`:
    github.com/filecoin-project/go-statemachine/fsm.eventProcessor.Apply
        /home/runner/go/pkg/mod/github.com/filecoin-project/[email protected]/fsm/eventprocessor.go:137
2021-07-13T09:13:20.035Z	ERROR	fsm	fsm/fsm.go:92	Executing event planner failed: Invalid transition in queue, state `0`, event `4`:
    github.com/filecoin-project/go-statemachine/fsm.eventProcessor.Apply
        /home/runner/go/pkg/mod/github.com/filecoin-project/[email protected]/fsm/eventprocessor.go:137
{"level":"error","error":"No state for /1626167599030: datastore: key not found","time":"2021-07-13T09:13:20Z","message":"attempting to configure data store"}
2021-07-13T09:13:20.051Z	ERROR	fsm	fsm/fsm.go:92	Executing event planner failed: Invalid transition in queue, state `0`, event `4`:
    github.com/filecoin-project/go-statemachine/fsm.eventProcessor.Apply
        /home/runner/go/pkg/mod/github.com/filecoin-project/[email protected]/fsm/eventprocessor.go:137
2021-07-13T09:13:20.051Z	ERROR	fsm	fsm/fsm.go:92	Executing event planner failed: Invalid transition in queue, state `0`, event `4`:
    github.com/filecoin-project/go-statemachine/fsm.eventProcessor.Apply
        /home/runner/go/pkg/mod/github.com/filecoin-project/[email protected]/fsm/eventprocessor.go:137
2021-07-13T09:13:20.053Z	ERROR	fsm	fsm/fsm.go:92	Executing event planner failed: Invalid transition in queue, state `6`, event `27`:
    github.com/filecoin-project/go-statemachine/fsm.eventProcessor.Apply
        /home/runner/go/pkg/mod/github.com/filecoin-project/[email protected]/fsm/eventprocessor.go:137
    tx_test.go:415: could not finish gtx2

• Context: In the case of a simple retrieval, the provider can redeem the received vouchers and settle the channel once the transfer is finished.
• Problem: if the client wishes to reuse a payment channel a while longer say for progressively retrieving parts of a DAG, it becomes more complex for the provider to know when is a good time to settle a payment channel.
• Naive solution: wait for the client to call settle so the provider knows the client no longer needs the channel and can redeem all the vouchers as one. This is nice because it means the provider needn't pay for the settle gas costs.
• Caveat: What if the client disappears for whatever reasons without calling settle, the provider must then have a way to collect their earnings. It also means the provider must subscribe to chain events.
• Enhanced solution: The client must set a MinSettleHeight param on the vouchers which guarantees no one can call settle before then. The provider reads the value and can decide to update the payment channel using the voucher or just wait knowing more transfers might be coming. If the client doesn't call settle by the chain height, the provider can just redeem and settle the channel.
• Security consideration: Providers shouldn't accept vouchers with a TimelockMin value 12h over the MinSettleHeight as it would mean the client can call settle and collect back their funds before the provider can redeem the vouchers.
• Additional improvements: Subscribing to chain epochs from a lotus node RPC puts too much strain and dependency on 3rd party infrastructure. Nodes should connect to a few lotus peers directly and subscribe to the gossip topic announcing new blocks. This could be a good start for enabling pushing blocks directly in the future.