Implementation of the Filecoin protocol, written in Go

Related tags

Cryptography lotus
Overview

Project Lotus Logo

Project Lotus - 莲


Lotus is an implementation of the Filecoin Distributed Storage Network. For more details about Filecoin, check out the Filecoin Spec.

Building & Documentation

For instructions on how to build, install and setup lotus, please visit https://docs.filecoin.io/get-started/lotus.

Reporting a Vulnerability

Please send an email to [email protected]. See our security policy for more details.

Related packages

These repos are independent and reusable modules, but are tightly integrated into Lotus to make up a fully featured Filecoin implementation:

Contribute

Lotus is a universally open project and welcomes contributions of all kinds: code, docs, and more. However, before making a contribution, we ask you to heed these recommendations:

  1. If the proposal entails a protocol change, please first submit a Filecoin Improvement Proposal.
  2. If the change is complex and requires prior discussion, open an issue or a discussion to request feedback before you start working on a pull request. This is to avoid disappointment and sunk costs, in case the change is not actually needed or accepted.
  3. Please refrain from submitting PRs to adapt existing code to subjective preferences. The changeset should contain functional or technical improvements/enhancements, bug fixes, new features, or some other clear material contribution. Simple stylistic changes are likely to be rejected in order to reduce code churn.

When implementing a change:

  1. Adhere to the standard Go formatting guidelines, e.g. Effective Go. Run go fmt.
  2. Stick to the idioms and patterns used in the codebase. Familiar-looking code has a higher chance of being accepted than eerie code. Pay attention to commonly used variable and parameter names, avoidance of naked returns, error handling patterns, etc.
  3. Comments: follow the advice on the Commentary section of Effective Go.
  4. Minimize code churn. Modify only what is strictly necessary. Well-encapsulated changesets will get a quicker response from maintainers.
  5. Lint your code with golangci-lint (CI will reject your PR if unlinted).
  6. Add tests.
  7. Title the PR in a meaningful way and describe the rationale and the thought process in the PR description.
  8. Write clean, thoughtful, and detailed commit messages. This is even more important than the PR description, because commit messages are stored inside the Git history. One good rule is: if you are happy posting the commit message as the PR description, then it's a good commit message.

License

Dual-licensed under MIT + Apache 2.0

Comments
  • Arm64 Raspberry Pi Build Failure

    Arm64 Raspberry Pi Build Failure

    Describe the bug When trying to build the lotus project from source files I run into this error:

        error[E0433]: failed to resolve: use of undeclared type or module `cc`
         --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/fil-sapling-crypto-0.6.0/build.rs:6:9
          |
        6 |         cc::Build::new()
          |         ^^ use of undeclared type or module `cc`
        
        error: aborting due to previous error
    

    To Reproduce Steps to reproduce the behavior:

    1. followed the steps in this issue: https://github.com/filecoin-project/lotus/issues/1779
    export RUSTFLAGS="-C target-cpu=native -g"
    export FFI_BUILD_FROM_SOURCE=1
    make clean deps bench
    

    Expected behavior The lotus project to build without errors

    Screenshots Someone already reported the issue on an upstream repo. https://github.com/zcash-hackworks/sapling-crypto/issues/104

    Version (run lotus --version): unable to compile the latest version

    Additional context Add any other context about the problem here. Arm64 Architecture

    help wanted 
    opened by deefactorial 60
  •  ChainGetTipSetByHeight method synchronization lacks message

    ChainGetTipSetByHeight method synchronization lacks message

    image

    The ChainGetTipSetByHeight method cannot synchronize the messages under all miners and there are references bug i used the ChainGetTipSetByHeight rpc Can't sync to that transaction

    the method is follow img: image

    opened by yuzhongrong 47
  • updating to new datastore/blockstore code with contexts

    updating to new datastore/blockstore code with contexts

    Status

    The following deps need to be tagged, as we have go mod versions pointing at unstable things:

    • github.com/drand/drand

    The following deps also exist in the lotus-soup go.mod:

    • github.com/drand/drand
    opened by whyrusleeping 37
  • Mined blocks became orphan blocks

    Mined blocks became orphan blocks

    Describe the bug

    We mined some blocks,but don't know why these blocks are not exists on chain.

    2020-12-06T01:23:36.133Z        INFO    storageminer    storage/miner.go:251    Computing WinningPoSt ;[{SealProof:8 SectorNumber:25272 SealedCID:bagboea4b5abcbe2cyns3zmw3mhhlftwx3im76ytulqdpfkivvmh7tf5bub46qzrk}]; [194 38 156 168 46 37 44 32 46 145 33 78 103 176 212 16 46 155 183 43 24 72 86 19 54 56 37 183 139 26 184 11]
    2020-12-06T01:23:41.759Z        INFO    storageminer    storage/miner.go:258    GenerateWinningPoSt took 5.626304116s
    2020-12-06T01:23:42.157Z        INFO    miner   miner/miner.go:448      mined new block {"cid": "bafy2bzaceaj6mkgqeh6qrnl4jrqwxptuanijcpgg6eo2dzspnxewj3bzbhp7k", "height": "297048", "miner": "f03339", "parents": ["f07830","f02520","f030347","f02777","f020331","f020522"], "took": 6.035145888}
    
    2020-12-05T23:20:06.100Z        INFO    storageminer    storage/miner.go:251    Computing WinningPoSt ;[{SealProof:3 SectorNumber:16435 SealedCID:bagboea4b5abcb4t4fybuumey4xfiefynlhqqrdlmzfoyg7uqwym7vrx75d6uwzc5}]; [182 6 130 38 137 107 109 218 22 88 224 234 22 86 97 212 80 201 160 117 242 136 51 129 201 109 113 41 181 201 28 175]
    2020-12-05T23:20:07.577Z        WARN    sectors storage-sealing/states_sealing.go:355   revert in interactive commit sector step
    2020-12-05T23:20:11.758Z        INFO    storageminer    storage/miner.go:258    GenerateWinningPoSt took 5.657667875s
    2020-12-05T23:20:12.203Z        INFO    miner   miner/miner.go:448      mined new block {"cid": "bafy2bzaceceuzscc5kxhkc4eytak4wlbmemdd4m3e7dqlmmsrb37hva4p7zri", "height": "296801", "miner": "f03339", "parents": ["f034548","f023530","f022374","f010253"], "took": 6.113739295}
    
    P2 need/analysis area/mining 
    opened by liangping 34
  • 1.18.0, the miner process crashes during execution of the AddPiece phase

    1.18.0, the miner process crashes during execution of the AddPiece phase

    Checklist

    • [X] This is not a security-related bug/issue. If it is, please follow please follow the security policy.
    • [X] This is not a question or a support request. If you have any lotus related questions, please ask in the lotus forum.
    • [X] This is not a new feature request. If it is, please file a feature request instead.
    • [X] This is not an enhancement request. If it is, please file a improvement suggestion instead.
    • [X] I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
    • [X] I am running the Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
    • [X] I did not make any code changes to lotus.

    Lotus component

    • [ ] lotus daemon - chain sync
    • [ ] lotus miner - mining and block production
    • [X] lotus miner/worker - sealing
    • [ ] lotus miner - proving(WindowPoSt)
    • [X] lotus miner/market - storage deal
    • [ ] lotus miner/market - retrieval deal
    • [ ] lotus miner/market - data transfer
    • [ ] lotus client
    • [ ] lotus JSON-RPC API
    • [ ] lotus message management (mpool)
    • [ ] Other

    Lotus Version

    lotus --version
    lotus version 1.18.0+mainnet+git.bd10bdf99
    lotus-miner --version
    lotus-miner version 1.18.0+mainnet+git.bd10bdf99
    

    Describe the Bug

    From time to time, the miner process crashes during (or when trying to) execute the AddPiece phase. In the time that has passed since the network upgrade, this has already happened twice.

    Logging Information

    Last lines of miner.log:
    
    2022-12-02T03:26:40.285+0300    INFO    miner   miner/miner.go:475      completed mineOne       {"tookMilliseconds": 6, "forRound": 2387814, "baseEpoch": 2387813, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 2483659, "lookbackEpochs": 900, "networkPowerAtLookback": "21373846934587277312", "minerPowerAtLookback": "3248215606362112", "isEligible": true, "isWinner": false, "error": null}
    2022-12-02T03:27:10.272+0300    INFO    miner   miner/miner.go:475      completed mineOne       {"tookMilliseconds": 8, "forRound": 2387815, "baseEpoch": 2387814, "baseDeltaSeconds": 10, "nullRounds": 0, "lateStart": false, "beaconEpoch": 2483660, "lookbackEpochs": 900, "networkPowerAtLookback": "21374157681872076800", "minerPowerAtLookback": "3248215606362112", "isEligible": true, "isWinner": false, "error": null}
    2022-12-02T03:27:30.315+0300    INFO    sectors pipeline/input.go:286   Adding piece for deal 17660229 (publish msg: bafy2bzacedj5ceycbivgvvx3k22lpntnjnmyiirowgft6izgr2amarwyrdnys)
    2022-12-02T03:27:30.316+0300    INFO    sectors pipeline/input.go:286   Adding piece for deal 17660230 (publish msg: bafy2bzacedj5ceycbivgvvx3k22lpntnjnmyiirowgft6izgr2amarwyrdnys)
    
    
    Last lines of miner_rust.log:
    
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x1ab85be]
    
    goroutine 269380609 [running]:
    github.com/filecoin-project/lotus/storage/pipeline.(*Sealing).updateInput.func1(0x3a28500?)
            /home/admfc/lotus/storage/pipeline/input.go:449 +0xbe
    github.com/filecoin-project/lotus/storage/pipeline.(*Sealing).updateInput(0xc00055c3c0, {0x46836a8, 0xc04558b710}, 0x8?)
            /home/admfc/lotus/storage/pipeline/input.go:491 +0x1035
    github.com/filecoin-project/lotus/storage/pipeline.(*Sealing).addPendingPiece.func2()
            /home/admfc/lotus/storage/pipeline/input.go:401 +0x7b
    created by github.com/filecoin-project/lotus/storage/pipeline.(*Sealing).addPendingPiece
            /home/admfc/lotus/storage/pipeline/input.go:399 +0x267
    

    Repo Steps

    Not sure if the problem is reproducible. There are two servers on the same site that are almost identical in hardware stuffing, both on the same software version. The 32GiB miner with the Threadripper PRO 3945WX processor is experiencing a problem, the 64GiB miner with the 3955WX processor has not experienced it yet. But in general -

    1. Install release 1.18.0
    2. Accept deals.
    3. Observe the problem.
    need/community-input kind/bug need/analysis area/sealing 
    opened by Shekelme 28
  • GPU stuck at P2

    GPU stuck at P2

    Describe the bug

    This problem is an occasional problem, When the worker is started for the first time, there will be a high probability of problems, Then the probability of encountering this problem after restarting will decrease

    Our GPU: Geforce RTX 2080 Ti

    worker log:

    2020-11-24T01:18:01.561 INFO filcrypto::proofs::api > seal_pre_commit_phase2: start
    2020-11-24T01:18:01.575 INFO filecoin_proofs::api > validate_cache_for_precommit_phase2:start
    2020-11-24T01:18:01.603 INFO filecoin_proofs::api > validate_cache_for_precommit_phase2:finish
    2020-11-24T01:18:01.637 INFO filecoin_proofs::api::seal > seal_pre_commit_phase2:start
    2020-11-24T01:18:01.708 INFO storage_proofs_porep::stacked::vanilla::proof > replicate_phase2
    2020-11-24T01:18:01.708 INFO storage_proofs_porep::stacked::vanilla::proof > generating tree c using the GPU
    2020-11-24T01:18:01.708 INFO storage_proofs_porep::stacked::vanilla::proof > Building column hashes
    2020-11-24T01:18:01.736 INFO neptune::cl > getting context for ~Index(0)
    2020-11-24T01:18:01.793 WARN neptune::cl > Cannot get device list for platform: Clover!
    2020-11-24T01:18:01.793 WARN neptune::cl > Cannot get device list for platform: Clover!
    2020-11-24T01:19:55.835 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 1/8 of length 153391689
    2020-11-24T01:21:31.248 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 2/8 of length 153391689
    2020-11-24T01:23:04.417 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 3/8 of length 153391689
    2020-11-24T01:24:38.587 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 4/8 of length 153391689
    2020-11-24T01:26:13.404 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 5/8 of length 153391689
    2020-11-24T01:27:46.782 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 6/8 of length 153391689
    2020-11-24T01:29:27.555 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 7/8 of length 153391689
    2020-11-24T01:31:13.127 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 8/8 of length 153391689
    2020-11-24T01:31:18.598 INFO storage_proofs_porep::stacked::vanilla::proof > tree_c done
    2020-11-24T01:31:18.598 INFO storage_proofs_porep::stacked::vanilla::proof > building tree_r_last
    2020-11-24T01:31:18.598 INFO storage_proofs_porep::stacked::vanilla::proof > generating tree r last using the GPU
    2020-11-24T01:31:19.224 INFO neptune::cl > getting context for ~Index(0)
    2020-11-24T01:31:19.369 WARN neptune::cl > Cannot get device list for platform: Clover!
    2020-11-24T10:43:56.641+0800    ^[[33mWARN^[[0m main    lotus-seal-worker/main.go:421   Shutting down...
    2020-11-24T10:43:56.655+0800    ^[[33mWARN^[[0m main    lotus-seal-worker/main.go:98    http: Server closed
    

    dmesg log:

    [Tue Nov 24 01:18:26 2020] NVRM: GPU at PCI:0000:81:00: GPU-77feb6df-eb6f-ae6d-8f89-5f84fb7c3e40
    [Tue Nov 24 01:18:26 2020] NVRM: GPU Board Serial Number:
    [Tue Nov 24 01:18:26 2020] NVRM: Xid (PCI:0000:81:00): 13, pid=30010, Graphics SM Warp Exception on (GPC 2, TPC 0, SM 0): Out Of Range Address
    [Tue Nov 24 01:18:26 2020] NVRM: Xid (PCI:0000:81:00): 13, pid=30010, Graphics Exception: ESR 0x514730=0xc01000e 0x514734=0x20 0x514728=0x4c1eb72 0x51472c=0x174
    [Tue Nov 24 01:18:26 2020] NVRM: Xid (PCI:0000:81:00): 43, pid=30398, Ch 00000008
    

    I compared the logs of the worker after the first startup and restart, I found one place is different:

    first startup:

    2020-11-24T01:18:01.561 INFO filcrypto::proofs::api > seal_pre_commit_phase2: start
    2020-11-24T01:18:01.575 INFO filecoin_proofs::api > validate_cache_for_precommit_phase2:start
    2020-11-24T01:18:01.603 INFO filecoin_proofs::api > validate_cache_for_precommit_phase2:finish
    2020-11-24T01:18:01.637 INFO filecoin_proofs::api::seal > seal_pre_commit_phase2:start
    2020-11-24T01:18:01.708 INFO storage_proofs_porep::stacked::vanilla::proof > replicate_phase2
    2020-11-24T01:18:01.708 INFO storage_proofs_porep::stacked::vanilla::proof > generating tree c using the GPU
    2020-11-24T01:18:01.708 INFO storage_proofs_porep::stacked::vanilla::proof > Building column hashes
    2020-11-24T01:18:01.736 INFO neptune::cl > getting context for ~Index(0)
    
    ----------------------------------------------------------------------------------------------------------------
    
    2020-11-24T01:18:01.793 WARN neptune::cl > Cannot get device list for platform: Clover!
    2020-11-24T01:18:01.793 WARN neptune::cl > Cannot get device list for platform: Clover!
    
    ## Appeared here twice
    ----------------------------------------------------------------------------------------------------------------
    
    2020-11-24T01:19:55.835 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 1/8 of length 153391689
    2020-11-24T01:21:31.248 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 2/8 of length 153391689
    2020-11-24T01:23:04.417 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 3/8 of length 153391689
    2020-11-24T01:24:38.587 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 4/8 of length 153391689
    2020-11-24T01:26:13.404 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 5/8 of length 153391689
    2020-11-24T01:27:46.782 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 6/8 of length 153391689
    2020-11-24T01:29:27.555 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 7/8 of length 153391689
    2020-11-24T01:31:13.127 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 8/8 of length 153391689
    2020-11-24T01:31:18.598 INFO storage_proofs_porep::stacked::vanilla::proof > tree_c done
    2020-11-24T01:31:18.598 INFO storage_proofs_porep::stacked::vanilla::proof > building tree_r_last
    2020-11-24T01:31:18.598 INFO storage_proofs_porep::stacked::vanilla::proof > generating tree r last using the GPU
    2020-11-24T01:31:19.224 INFO neptune::cl > getting context for ~Index(0)
    2020-11-24T01:31:19.369 WARN neptune::cl > Cannot get device list for platform: Clover!
    
    

    restart:

    2020-11-24T15:42:33.077 INFO filecoin_proofs::api::seal > seal_pre_commit_phase2:start
    2020-11-24T15:42:33.079 INFO storage_proofs_porep::stacked::vanilla::proof > replicate_phase2
    2020-11-24T15:42:33.079 INFO storage_proofs_porep::stacked::vanilla::proof > generating tree c using the GPU
    2020-11-24T15:42:33.079 INFO storage_proofs_porep::stacked::vanilla::proof > Building column hashes
    2020-11-24T15:42:33.079 INFO neptune::cl > getting context for ~Index(0)
    
    ----------------------------------------------------------------------------------------------------------------
    
    2020-11-24T15:42:33.084 WARN neptune::cl > Cannot get device list for platform: Clover!
    
    ## Appeared here once
    ----------------------------------------------------------------------------------------------------------------
    
    2020-11-24T15:44:44.044 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 1/8 of length 153391689
    2020-11-24T15:47:00.733 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 2/8 of length 153391689
    2020-11-24T15:49:14.234 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 3/8 of length 153391689
    2020-11-24T15:51:29.922 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 4/8 of length 153391689
    2020-11-24T15:53:44.894 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 5/8 of length 153391689
    2020-11-24T15:55:59.994 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 6/8 of length 153391689
    2020-11-24T15:58:15.629 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 7/8 of length 153391689
    2020-11-24T16:00:31.432 INFO storage_proofs_porep::stacked::vanilla::proof > persisting base tree_c 8/8 of length 153391689
    2020-11-24T16:00:36.050 INFO storage_proofs_porep::stacked::vanilla::proof > tree_c done
    2020-11-24T16:00:36.050 INFO storage_proofs_porep::stacked::vanilla::proof > building tree_r_last
    2020-11-24T16:00:36.050 INFO storage_proofs_porep::stacked::vanilla::proof > generating tree r last using the GPU
    2020-11-24T16:00:36.632 INFO neptune::cl > getting context for ~Index(0)
    2020-11-24T16:00:36.632 WARN neptune::cl > Cannot get device list for platform: Clover!
    2020-11-24T16:00:45.490 INFO storage_proofs_porep::stacked::vanilla::proof > building base tree_r_last with GPU 1/8
     ......... snip ..............
    
    support kind/stale 
    opened by Xib1uvXi 27
  • Lotus-bench results thread (v20 params)

    Lotus-bench results thread (v20 params)

    This issue is a place to put lotus-bench results for v20 params.

    To best help us, run four tests:

    Start by installing build dependencies from https://docs.lotu.sh/en+getting-started

    git clone https://github.com/filecoin-project/lotus.git
    cd lotus
    make build bench
    
    ./bench --sector-size=1073741824
    ./bench --sector-size=1073741824 --no-gpu
    
    # Only run these with > 64GiB of ram, recommended 128G
    ./bench --sector-size=34359738368
    ./bench --sector-size=34359738368 --no-gpu
    

    Additionally, please tell us what CPU, GPU, and memory (including speed) you have in your setup.

    Previous (v19) thread - https://github.com/filecoin-project/lotus/issues/694

    opened by magik6k 27
  • Cannot perform any retrieval for 32GB files

    Cannot perform any retrieval for 32GB files

    Describe the bug I tried several retrievals to several different miners for the deals I made. None of them went through. I do not believe they all offline. I suspect it has something to do with the deal size. These are all 32GB offline deals, made with --fast-retrieval=false.

    Example 1 - stuck after DealStatusOngoing Provider is doing unsealing but it takes hours without going further

    lotus client retrieve --miner f024008 bafykbzaced6noeziyglm2frycaknydeijbimvhpk72eimhfupoxd72dehp2k4 abc

    Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) Recv: 0 B, Paid 0 FIL, ClientEventDealProposed (DealStatusWaitForAcceptance) Recv: 0 B, Paid 0 FIL, ClientEventDealAccepted (DealStatusAccepted) Recv: 0 B, Paid 0 FIL, ClientEventPaymentChannelCreateInitiated (DealStatusPaymentChannelCreating) Recv: 0 B, Paid 0 FIL, ClientEventPaymentChannelReady (DealStatusPaymentChannelAllocatingLane) Recv: 0 B, Paid 0 FIL, ClientEventLaneAllocated (DealStatusOngoing)

    Example 2 - unmarshalling error The provider told me his miner crashed upon receiving my retrieval request

    lotus client retrieve --miner f083550 bafykbzacecu4qt4tlr5vqojgtlt553rbskqremq67z37x5cufaonjqfh5y7no mysql-2016-04-19.tar.gz.partbl 2021-01-25T04:59:20.145Z WARN rpc [email protected]/client.go:541 unmarshaling failed {"message": "{"Err":"exhausted 5 attempts but failed to open stream, err: failed to dial 12D3KooWFJ6iPAiW82pR7REB8pJfKAsoKoEAhgWKYduoQT1734a9: all dials failed\n * [/ip4/127.0.0.1/tcp/45989] dial tcp4 127.0.0.1:45989: connect: connection refused\n * [/ip6/::1/tcp/41337] dial tcp6 [::1]:41337: connect: connection refused\n * [/ip4/59.12.56.212/tcp/45989] dial tcp4 59.12.56.212:45989: connect: connection refused\n * [/ip4/59.12.56.215/tcp/45989] dial tcp4 0.0.0.0:33463-\u003e59.12.56.215:45989: i/o timeout","Root":null,"Piece":null,"Size":0,"MinPrice":"\u003cnil\u003e","UnsealPrice":"\u003cnil\u003e","PaymentInterval":0,"PaymentIntervalIncrease":0,"Miner":"f083550","MinerPeer":{"Address":"f083550","ID":"12D3KooWFJ6iPAiW82pR7REB8pJfKAsoKoEAhgWKYduoQT1734a9","PieceCID":null}}"} ERROR: RPC client error: unmarshaling result: failed to parse big string: '"\u003cnil\u003e"'

    Example 3 - incomplete response One of the provider told me his miner is set as allowUnseal=false but his worker is set as allowUnseal=true, yet the retrieval failed before the worker picked up the unseal.

    lotus client retrieve --miner f064218 baga6ea4seaqgsbrsupr6az5zf2yjqai5t4xtgn7lf2zd62wnxxq2mgyhlvbxyda publicdomainmovies.tar.05

    Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) Recv: 0 B, Paid 0 FIL, ClientEventDealProposed (DealStatusWaitForAcceptance) Recv: 0 B, Paid 0 FIL, ClientEventUnsealPaymentRequested (DealStatusAccepted) Recv: 0 B, Paid 0 FIL, ClientEventPaymentChannelAddingFunds (DealStatusPaymentChannelAllocatingLane) Recv: 0 B, Paid 0 FIL, ClientEventLaneAllocated (DealStatusOngoing) Recv: 0 B, Paid 0 FIL, ClientEventPaymentRequested (DealStatusFundsNeeded) Recv: 0 B, Paid 0 FIL, ClientEventSendFunds (DealStatusSendFunds) Recv: 0 B, Paid 0.1 FIL, ClientEventPaymentSent (DealStatusOngoing) Recv: 0 B, Paid 0.1 FIL, ClientEventDataTransferError (DealStatusErrored) Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) ERROR: retrieval failed: Retrieve: Retrieval Error: error generated by data transfer: deal data transfer failed: incomplete response

    lotus client retrieve --miner f01278 bafykbzaceaj7ube4k2vhniqgdb6vq7ggcoebf4c6punse7ike2almme6esdxi mysql-2018-11-01.tar.gz.partcy

    Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) Recv: 0 B, Paid 0 FIL, ClientEventDealProposed (DealStatusWaitForAcceptance) Recv: 0 B, Paid 0 FIL, ClientEventUnsealPaymentRequested (DealStatusAccepted) Recv: 0 B, Paid 0 FIL, ClientEventPaymentChannelCreateInitiated (DealStatusPaymentChannelCreating) Recv: 0 B, Paid 0 FIL, ClientEventPaymentChannelReady (DealStatusPaymentChannelAllocatingLane) Recv: 0 B, Paid 0 FIL, ClientEventLaneAllocated (DealStatusOngoing) Recv: 0 B, Paid 0 FIL, ClientEventPaymentRequested (DealStatusFundsNeeded) Recv: 0 B, Paid 0 FIL, ClientEventSendFunds (DealStatusSendFunds) Recv: 0 B, Paid 0.000000000000000002 FIL, ClientEventPaymentSent (DealStatusOngoing) Recv: 0 B, Paid 0.000000000000000002 FIL, ClientEventDataTransferError (DealStatusErrored) Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) ERROR: retrieval failed: Retrieve: Retrieval Error: error generated by data transfer: deal data transfer failed: incomplete response

    Example 4 - miner is not accepting online retrieval deals lotus client retrieve --miner f047419 baga6ea4seaqgsbrsupr6az5zf2yjqai5t4xtgn7lf2zd62wnxxq2mgyhlvbxyda publicdomainmovies.tar.05

    Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) Recv: 0 B, Paid 0 FIL, ClientEventDealProposed (DealStatusWaitForAcceptance) Recv: 0 B, Paid 0 FIL, ClientEventDealRejected (DealStatusRetryLegacy) Recv: 0 B, Paid 0 FIL, ClientEventDealProposed (DealStatusWaitForAcceptanceLegacy) Recv: 0 B, Paid 0 FIL, ClientEventDealRejected (DealStatusRejected) Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) ERROR: retrieval failed: Retrieve: Retrieval Proposal Rejected: deal rejected: miner is not accepting online retrieval deals

    Example 5 - normal shutdown of state machine This provider is able to serve my retrieval for a 8GB file, but not 32GB ones.

    lotus client retrieve --miner f022352 baga6ea4seaqoyyc52ajdq2v7723d3uixeem5ulwvbtv7h45776vrrg53nnf2gjq trusted_setup_phase2.tar.47

    Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) Recv: 0 B, Paid 0 FIL, ClientEventDealProposed (DealStatusWaitForAcceptance) Recv: 0 B, Paid 0 FIL, ClientEventDealAccepted (DealStatusAccepted) Recv: 0 B, Paid 0 FIL, ClientEventPaymentChannelAddingFunds (DealStatusPaymentChannelAllocatingLane) Recv: 0 B, Paid 0 FIL, ClientEventLaneAllocated (DealStatusOngoing) Recv: 0 B, Paid 0 FIL, ClientEventProviderCancelled (DealStatusCancelling) Recv: 0 B, Paid 0 FIL, ClientEventDataTransferError (DealStatusErrored) Recv: 0 B, Paid 0 FIL, ClientEventOpen (DealStatusNew) ERROR: retrieval failed: Retrieve: Retrieval Error: error generated by data transfer: unable to send cancel to channel FSM: normal shutdown of state machine

    lotus version 1.4.1+git.d6c06881e

    area/markets/client area/markets/retrieval team/ignite 
    opened by kernelogic 25
  • Importing chain ends with chain validation failed

    Importing chain ends with chain validation failed

    Running lotus daemon --import-chain lotus daemon --import-chain minimal_finality_stateroots_336340_2020-12-19_17-00-00.car

    v1.4.0 ends with:

     15.54 GiB / 15.58 GiB [======================================================================================================================================]  99.74% 115.68 MiB/s2020-12-21T07:06:20.325Z        INFO    badgerbs        [email protected]/levels.go:962      LOG Compact 1->2, del 3 tables, add 3 tables, took 658.043122ms
    
    2020-12-21T07:06:20.325Z        INFO    badgerbs        [email protected]/levels.go:1010     [Compactor: 0] Compaction for level: 1 DONE
    2020-12-21T07:06:20.325Z        INFO    badgerbs        [email protected]/levels.go:1000     [Compactor: 0] Running compaction: {level:1 score:1.5613754019141197 dropPrefixes:[]} for level: 1
    
     15.58 GiB / 15.58 GiB [================================================================================================================================] 100.00% 115.67 MiB/s 2m17s
    2020-12-21T07:06:20.609Z        INFO    chainstore      store/store.go:527      clearing block validation cache...
    2020-12-21T07:06:20.609Z        INFO    chainstore      store/store.go:566      0 block validation entries cleared.
    2020-12-21T07:06:22.436Z        INFO    badgerbs        [email protected]/levels.go:962      LOG Compact 1->2, del 8 tables, add 8 tables, took 2.110439345s
    
    2020-12-21T07:06:22.436Z        INFO    badgerbs        [email protected]/levels.go:1010     [Compactor: 0] Compaction for level: 1 DONE
    2020-12-21T07:06:22.436Z        INFO    badgerbs        [email protected]/levels.go:1000     [Compactor: 0] Running compaction: {level:1 score:1.301148410886526 dropPrefixes:[]} for level: 1
    
    2020-12-21T07:06:23.041Z        INFO    badgerbs        [email protected]/levels.go:962      LOG Compact 1->2, del 2 tables, add 2 tables, took 604.994671ms
    
    2020-12-21T07:06:23.041Z        INFO    badgerbs        [email protected]/levels.go:1010     [Compactor: 0] Compaction for level: 1 DONE
    2020-12-21T07:06:23.041Z        INFO    badgerbs        [email protected]/levels.go:1000     [Compactor: 0] Running compaction: {level:1 score:1.0409185104072094 dropPrefixes:[]} for level: 1
    
    2020-12-21T07:06:24.137Z        INFO    badgerbs        [email protected]/levels.go:962      LOG Compact 1->2, del 4 tables, add 4 tables, took 1.09613443s
    
    2020-12-21T07:06:24.137Z        INFO    badgerbs        [email protected]/levels.go:1010     [Compactor: 0] Compaction for level: 1 DONE
    2020-12-21T07:06:42.565Z        WARN    chainstore      store/store.go:508      no heaviest tipset found, using [bafy2bzacecnamqgqmifpluoeldx7zzglxcljo6oja4vrmtj7432rphldpdmm2]
    2020-12-21T07:06:42.565Z        INFO    chainstore      store/store.go:513      New heaviest tipset! [bafy2bzacecnamqgqmifpluoeldx7zzglxcljo6oja4vrmtj7432rphldpdmm2] (height=0)
    2020-12-21T07:06:42.566Z        INFO    main    lotus/daemon.go:470     validating imported chain...
    2020-12-21T07:07:04.375Z        INFO    statemgr        stmgr/stmgr.go:878      computing state (height: 0, ts=[bafy2bzacecnamqgqmifpluoeldx7zzglxcljo6oja4vrmtj7432rphldpdmm2])
    2020-12-21T07:07:04.375Z        INFO    statemgr        stmgr/stmgr.go:878      computing state (height: 1, ts=[bafy2bzacechdx6xd62lcyy7rnyc4uxcxhuwqslcxfvj77fxlwafij3nhzchpy])
    2020-12-21T07:07:04.375Z        WARN    chainstore      store/store.go:485      reorgWorker quit
    2020-12-21T07:07:04.421Z        INFO    badgerbs        [email protected]/db.go:1030 Storing value log head: {Fid:21 Len:33 Offset:513553269}
    
    2020-12-21T07:07:04.574Z        INFO    badgerbs        [email protected]/levels.go:1000     [Compactor: 173] Running compaction: {level:0 score:1.73 dropPrefixes:[]} for level: 0
    
    2020-12-21T07:07:05.444Z        INFO    badgerbs        [email protected]/levels.go:962      LOG Compact 0->1, del 4 tables, add 4 tables, took 870.449062ms
    
    2020-12-21T07:07:05.445Z        INFO    badgerbs        [email protected]/levels.go:1010     [Compactor: 173] Compaction for level: 0 DONE
    2020-12-21T07:07:05.445Z        INFO    badgerbs        [email protected]/db.go:553  Force compaction on level 0 done
    ERROR: chain validation failed: getting block messages for tipset: failed to get messages for block: failed to load msgmeta (bafy2bzacecmwp4imjqhdg2zvc7j2s4xxahnn5jnudtrt335re24i4zim7ccfi): blockstore: block not found
    
    opened by astudnev 25
  • [Thread] Documentation Requests

    [Thread] Documentation Requests

    If you have a request for documentation on lotus, lotus-miner, or any related ecosystem tooling, please leave a comment here with what you want documented.

    An example of a good request would be:

    I would like documentation on how to operate multiple lotus-workers on different machines on my local network, with different machines for different jobs

    or

    I would like some documentation on how to examine exactly why my message failed on chain.

    Please read through existing requests, and thumbs up any requests you also want (think of it like voting) instead of posting a duplicate (though if you want to add something to an existing request, please link to it in your comment, and let us know what you want to add)

    impact/docs 
    opened by whyrusleeping 25
  • Available sectors get upgraded to UpdateReplica sectors with no deals, when deals are ready to be added to sectors

    Available sectors get upgraded to UpdateReplica sectors with no deals, when deals are ready to be added to sectors

    Checklist

    • [X] This is not a security-related bug/issue. If it is, please follow please follow the security policy.
    • [X] This is not a question or a support request. If you have any lotus related questions, please ask in the lotus forum.
    • [X] This is not a new feature request. If it is, please file a feature request instead.
    • [X] This is not an enhancement request. If it is, please file a improvement suggestion instead.
    • [X] I have searched on the issue tracker and the lotus forum, and there is no existing related issue or discussion.
    • [X] I am running the Latest release, or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
    • [ ] I did not make any code changes to lotus.

    Lotus component

    • [ ] lotus daemon - chain sync
    • [ ] lotus miner - mining and block production
    • [X] lotus miner/worker - sealing
    • [ ] lotus miner - proving(WindowPoSt)
    • [ ] lotus miner/market - storage deal
    • [ ] lotus miner/market - retrieval deal
    • [ ] lotus miner/market - data transfer
    • [ ] lotus client
    • [ ] lotus JSON-RPC API
    • [ ] lotus message management (mpool)
    • [ ] Other

    Lotus Version

    lotus v1.15.2
    

    Describe the Bug

    Our sofiaminer has been configured with:

      MaxWaitDealsSectors = 2
      MaxSealingSectors = 2
      MaxSealingSectorsForDeals = 2
    

    Today we noticed that 4 deals (each ~14GiB) were blocked in Adding to sector state, and sealing pipeline was blocked with 2 sectors:

    957   UpdateReplica    YES      YES     3374050 (in 1 year 24 weeks)     CC
    1007  UpdateReplica    YES      YES     3374170 (in 1 year 24 weeks)     CC
    

    Those two sectors got created from Available sectors and their expiration epochs are after the 4 deals end epochs (i.e. the deals fit in these 2 sectors).

    It is not clear why the sectors got into UpdateReplica states without the deals inside them.


    After aborting the upgrade on these 2 sectors, 2 new sectors (from Available state) got upgraded, and successfully added the 4 deals to them and sealed properly into UpdateActivating state.

    Logging Information

    Broken UpdateReplica sectors with no deals:

    2022-05-19T06:06:30.646+0300    INFO    sectors storage-sealing/input.go:278    Adding piece for deal 6328251 (publish msg: bafy2bzaceaajmdwdoik4u3gjmtfqhdoqpo57e5el6ac7akt4bupxclskcxkzs)
    2022-05-19T06:06:30.650+0300    ERROR   sectors storage-sealing/input.go:514    we are trying to create a new sector with open sectors map[]
    2022-05-19T06:06:30.651+0300    INFO    sectors storage-sealing/input.go:674    new deal sector decision        {"sealing": 0, "maxSeal": 2, "maxUpgrade": 2, "preferNew": false, "canCreate": true, "canUpgrade": true, "shouldUpgrade": true}
    2022-05-19T06:06:30.665+0300    INFO    sectors storage-sealing/input.go:278    Adding piece for deal 6328250 (publish msg: bafy2bzaceaajmdwdoik4u3gjmtfqhdoqpo57e5el6ac7akt4bupxclskcxkzs)
    2022-05-19T06:06:31.185+0300    INFO    sectors storage-sealing/input.go:620    Upgrading sector        {"number": "1007", "type": "deal", "proofType": 8, "expiration": "3374170", "pledge": "0.149331217034391605 FIL"}
    2022-05-19T06:06:31.185+0300    ERROR   sectors storage-sealing/input.go:514    we are trying to create a new sector with open sectors map[]
    2022-05-19T06:06:31.191+0300    INFO    sectors storage-sealing/input.go:144    starting to seal deal sector    {"sector": "1007", "trigger": "wait-timeout"}
    2022-05-19T06:06:31.197+0300    INFO    sectors storage-sealing/states_sealing.go:48    performing filling up rest of the sector...     {"sector": "1007"}
    2022-05-19T06:06:31.198+0300    WARN    sectors storage-sealing/states_sealing.go:72    Creating 1 filler pieces for sector 1007
    2022-05-19T06:06:31.198+0300    INFO    sectors storage-sealing/states_sealing.go:88    Pledge {{127896 1007} 8}, contains []
    2022-05-19T06:06:31.198+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 1 queued; 4 open windows
    2022-05-19T06:06:31.200+0300    DEBUG   advmgr  sector-storage/sched.go:457     SCHED windows: [{allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]}]
    2022-05-19T06:06:31.200+0300    DEBUG   advmgr  sector-storage/sched.go:458     SCHED Acceptable win: [[0 1]]
    2022-05-19T06:06:31.200+0300    DEBUG   advmgr  sector-storage/sched.go:480     SCHED try assign sqi:0 sector 1007 to window 0 (awi:0)
    2022-05-19T06:06:31.200+0300    DEBUG   advmgr  sector-storage/sched.go:480     SCHED try assign sqi:0 sector 1007 to window 1 (awi:1)
    2022-05-19T06:06:31.200+0300    DEBUG   advmgr  sector-storage/sched.go:521     SCHED ASSIGNED  {"sqi": 0, "sector": "1007", "task": "seal/v0/addpiece", "window": 0, "worker": "80b730ae-31c9-4830-b176-b828a0df9dbe", "utilization": 0.2852374219200975}
    2022-05-19T06:06:31.200+0300    DEBUG   advmgr  sector-storage/sched_worker.go:369      assign worker sector 1007
    2022-05-19T06:06:31.200+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 0 queued; 4 open windows
    2022-05-19T06:06:31.201+0300    DEBUG   advmgr  sector-storage/sched_worker.go:274      task done       {"workerid": "80b730ae-31c9-4830-b176-b828a0df9dbe"}
    2022-05-19T06:06:31.201+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 0 queued; 4 open windows
    2022-05-19T06:07:34.664+0300    ERROR   sectors storage-sealing/fsm.go:34       unhandled sector error (1007): checkPieces sanity check error:
        github.com/filecoin-project/lotus/extern/storage-sealing.handleErrors
            /root/lotus/extern/storage-sealing/states_replica_update.go:295
      - sector 1007 must have deals, but does not
    
    2022-05-19T08:28:59.970+0300    INFO    sectors storage-sealing/input.go:278    Adding piece for deal 6333497 (publish msg: bafy2bzaceawz6tlxavlwdjiynfhsp2lrdnfeo66phthy42jaorhc5tbctalog)
    2022-05-19T08:28:59.972+0300    ERROR   sectors storage-sealing/input.go:514    we are trying to create a new sector with open sectors map[]
    2022-05-19T08:28:59.973+0300    INFO    sectors storage-sealing/input.go:674    new deal sector decision        {"sealing": 1, "maxSeal": 2, "maxUpgrade": 2, "preferNew": false, "canCreate": true, "canUpgrade": true, "shouldUpgrade": true}
    2022-05-19T08:28:59.995+0300    INFO    sectors storage-sealing/input.go:278    Adding piece for deal 6333498 (publish msg: bafy2bzaceawz6tlxavlwdjiynfhsp2lrdnfeo66phthy42jaorhc5tbctalog)
    2022-05-19T08:29:00.567+0300    INFO    sectors storage-sealing/input.go:620    Upgrading sector        {"number": "957", "type": "deal", "proofType": 8, "expiration": "3374050", "pledge": "0.14650541070944655 FIL"}
    2022-05-19T08:29:00.568+0300    ERROR   sectors storage-sealing/input.go:514    we are trying to create a new sector with open sectors map[]
    2022-05-19T08:29:00.573+0300    INFO    sectors storage-sealing/input.go:144    starting to seal deal sector    {"sector": "957", "trigger": "wait-timeout"}
    2022-05-19T08:29:00.579+0300    INFO    sectors storage-sealing/states_sealing.go:48    performing filling up rest of the sector...     {"sector": "957"}
    2022-05-19T08:29:00.579+0300    WARN    sectors storage-sealing/states_sealing.go:72    Creating 1 filler pieces for sector 957
    2022-05-19T08:29:00.579+0300    INFO    sectors storage-sealing/states_sealing.go:88    Pledge {{127896 957} 8}, contains []
    2022-05-19T08:29:00.579+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 1 queued; 4 open windows
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched.go:457     SCHED windows: [{allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]}]
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched.go:458     SCHED Acceptable win: [[0 3]]
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched.go:480     SCHED try assign sqi:0 sector 957 to window 0 (awi:0)
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched.go:480     SCHED try assign sqi:0 sector 957 to window 3 (awi:1)
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched.go:521     SCHED ASSIGNED  {"sqi": 0, "sector": "957", "task": "seal/v0/addpiece", "window": 0, "worker": "80b730ae-31c9-4830-b176-b828a0df9dbe", "utilization": 0.2852374219200975}
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched_worker.go:369      assign worker sector 957
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 0 queued; 4 open windows
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched_worker.go:274      task done       {"workerid": "80b730ae-31c9-4830-b176-b828a0df9dbe"}
    2022-05-19T08:29:00.582+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 0 queued; 4 open windows
    2022-05-19T08:29:06.011+0300    INFO    miner   miner/miner.go:479      completed mineOne       {"tookMilliseconds": 7, "forRound": 1821059, "baseEpoch": 1821058, "baseDeltaSeconds": 6, "nullRounds": 0, "lateStart": false, "beaconEpoch": 1916903, "lookbackEpochs": 900, "networkPowerAtLookback": "19312307760272900096", "minerPowerAtLookback": "63971695689728", "isEligible": true, "isWinner": false, "error": null}
    2022-05-19T08:30:04.770+0300    ERROR   sectors storage-sealing/fsm.go:34       unhandled sector error (957): checkPieces sanity check error:
        github.com/filecoin-project/lotus/extern/storage-sealing.handleErrors
            /root/lotus/extern/storage-sealing/states_replica_update.go:295
      - sector 957 must have deals, but does not
    

    I trigger abort upgrade on the 2 sectors from above, and the deals got added to the new sectors (around 13:00 timestamp):

    2022-05-19T12:57:00.015+0300    INFO    sectors storage-sealing/input.go:726    aborting upgrade of sector      {"sector": "957", "trigger": "user"}
    2022-05-19T12:57:00.015+0300    WARN    sectors storage-sealing/fsm.go:763      sector 957 got error event sealing.SectorAbortUpgrade: triggered by user
    2022-05-19T12:57:00.021+0300    ERROR   sectors storage-sealing/input.go:514    we are trying to create a new sector with open sectors map[]
    2022-05-19T12:57:00.022+0300    INFO    sectors storage-sealing/input.go:674    new deal sector decision        {"sealing": 1, "maxSeal": 2, "maxUpgrade": 2, "preferNew": false, "canCreate": true, "canUpgrade": true, "shouldUpgrade": true}
    2022-05-19T12:57:00.592+0300    INFO    sectors storage-sealing/input.go:620    Upgrading sector        {"number": "978", "type": "deal", "proofType": 8, "expiration": "3374050", "pledge": "0.146720918263824136 FIL"}
    2022-05-19T12:57:00.599+0300    ERROR   sectors storage-sealing/input.go:514    we are trying to create a new sector with open sectors map[{127896 978}:0xc01d2c3520]
    2022-05-19T12:57:00.600+0300    INFO    sectors storage-sealing/input.go:674    new deal sector decision        {"sealing": 2, "maxSeal": 2, "maxUpgrade": 2, "preferNew": false, "canCreate": false, "canUpgrade": false, "shouldUpgrade": false}
    2022-05-19T12:57:00.600+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 1 queued; 4 open windows
    2022-05-19T12:57:00.602+0300    DEBUG   advmgr  sector-storage/sched.go:457     SCHED windows: [{allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]} {allocated:{memUsedMin:0 memUsedMax:0 gpuUsed:0 cpuUse:0 cond:<nil> waiting:0} todo:[]}]
    2022-05-19T12:57:00.602+0300    DEBUG   advmgr  sector-storage/sched.go:458     SCHED Acceptable win: [[2 3]]
    2022-05-19T12:57:00.602+0300    DEBUG   advmgr  sector-storage/sched.go:480     SCHED try assign sqi:0 sector 978 to window 2 (awi:0)
    2022-05-19T12:57:00.603+0300    DEBUG   advmgr  sector-storage/sched.go:480     SCHED try assign sqi:0 sector 978 to window 3 (awi:1)
    2022-05-19T12:57:00.603+0300    DEBUG   advmgr  sector-storage/sched.go:521     SCHED ASSIGNED  {"sqi": 0, "sector": "978", "task": "seal/v0/addpiece", "window": 2, "worker": "80b730ae-31c9-4830-b176-b828a0df9dbe", "utilization": 0.2852374219200975}
    2022-05-19T12:57:00.603+0300    DEBUG   advmgr  sector-storage/sched_worker.go:369      assign worker sector 978
    2022-05-19T12:57:00.603+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 0 queued; 4 open windows
    2022-05-19T12:57:00.603+0300    DEBUG   advmgr  sector-storage/sched_worker.go:274      task done       {"workerid": "80b730ae-31c9-4830-b176-b828a0df9dbe"}
    2022-05-19T12:57:00.603+0300    DEBUG   advmgr  sector-storage/sched.go:361     SCHED 0 queued; 4 open windows
    2022-05-19T12:57:02.688+0300    INFO    sectors storage-sealing/input.go:726    aborting upgrade of sector      {"sector": "1007", "trigger": "user"}
    2022-05-19T12:57:02.688+0300    WARN    sectors storage-sealing/fsm.go:763      sector 1007 got error event sealing.SectorAbortUpgrade: triggered by user
    2022-05-19T12:57:02.696+0300    ERROR   sectors storage-sealing/input.go:514    we are trying to create a new sector with open sectors map[{127896 978}:0xc01d2c3520]
    2022-05-19T12:57:02.697+0300    INFO    sectors storage-sealing/input.go:674    new deal sector decision        {"sealing": 1, "maxSeal": 2, "maxUpgrade": 2, "preferNew": false, "canCreate": true, "canUpgrade": true, "shouldUpgrade": true}
    2022-05-19T12:57:03.201+0300    INFO    sectors storage-sealing/input.go:620    Upgrading sector        {"number": "875", "type": "deal", "proofType": 8, "expiration": "3374050", "pledge": "0.146775113865666972 FIL"}
    

    Logs from lotus-worker

    2022-05-19T06:06:31.211+0300    DEBUG   advmgr  sector-storage/worker_local.go:158      acquired sector {{127896 1007} 8} (e:0; a:1): {{0 0} /root/.lotusworker/unsealed/s-t0127896-1007    }
    2022-05-19T06:06:34.205+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -12288
    2022-05-19T06:06:44.206+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -12288
    2022-05-19T06:06:54.208+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -12288
    2022-05-19T06:07:04.209+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -32768
    2022-05-19T06:07:14.218+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -36864
    2022-05-19T06:07:24.220+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -36864
    2022-05-19T06:07:34.221+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -40960
    2022-05-19T08:29:00.587+0300    DEBUG   advmgr  sector-storage/worker_local.go:158      acquired sector {{127896 957} 8} (e:0; a:1): {{0 0} /root/.lotusworker/unsealed/s-t0127896-957    }
    2022-05-19T08:29:05.281+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -16384
    2022-05-19T08:29:15.282+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -16384
    2022-05-19T08:29:25.283+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -16384
    2022-05-19T08:29:35.283+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -32768
    2022-05-19T08:29:45.298+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -49152
    2022-05-19T08:29:55.302+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -49152
    
    
    2022-05-19T12:57:00.609+0300    DEBUG   advmgr  sector-storage/worker_local.go:158      acquired sector {{127896 978} 8} (e:0; a:1): {{0 0} /root/.lotusworker/unsealed/s-t0127896-978    }
    2022-05-19T12:57:03.221+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -28672
    2022-05-19T12:57:03.221+0300    DEBUG   advmgr  sector-storage/worker_local.go:158      acquired sector {{127896 875} 8} (e:0; a:1): {{0 0} /root/.lotusworker/unsealed/s-t0127896-875    }
    2022-05-19T12:57:07.218+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -73728
    2022-05-19T12:57:17.219+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -77824
    2022-05-19T12:57:27.221+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -86016
    2022-05-19T12:57:37.222+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -86016
    2022-05-19T12:57:47.223+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -86016                                                                     2022-05-19T12:57:57.224+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -86016
    2022-05-19T12:58:07.226+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -86016
    2022-05-19T12:58:17.227+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -86016                                                                     2022-05-19T12:58:27.228+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -86016
    2022-05-19T12:58:37.229+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -94208
    2022-05-19T12:58:47.230+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -106496
    2022-05-19T12:58:57.231+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -106496
    2022-05-19T12:59:07.232+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -106496
    2022-05-19T12:59:17.233+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -110592
    2022-05-19T12:59:27.237+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -110592
    2022-05-19T12:59:37.238+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -110592
    2022-05-19T12:59:47.239+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -110592
    2022-05-19T12:59:57.240+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -110592
    2022-05-19T13:00:07.241+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -110592                                                                    2022-05-19T13:00:17.245+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -110592
    2022-05-19T13:00:27.246+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=68719476736, reserved: -110592
    2022-05-19T13:00:31.235+0300    DEBUG   advmgr  sector-storage/worker_local.go:158      acquired sector {{127896 978} 8} (e:1; a:0): {{0 0} /root/.lotusworker/unsealed/s-t0127896-978    }                     2022-05-19T13:00:37.247+0300    WARN    stores  stores/local.go:130     negative reserved storage: p.reserved=34359738368, reserved: -53248
    2022-05-19T13:00:45.312+0300    DEBUG   advmgr  sector-storage/worker_local.go:158      acquired sector {{127896 875} 8} (e:1; a:0): {{0 0} /root/.lotusworker/unsealed/s-t0127896-875    }
    2022-05-19T13:04:18.534+0300    INFO    stores  stores/http_handler.go:189      SERVE Alloc check /remote/unsealed/s-t0127896-978/8/allocated/0/17045651456
    2022-05-19T13:04:18.535+0300    DEBUG   stores  stores/http_handler.go:283      returning ok: worker has unsealed file with unsealed piece, sector:{Miner:127896 Number:978}, offset:0, size:17045651456
    2022-05-19T13:04:18.539+0300    INFO    stores  stores/http_handler.go:189      SERVE Alloc check /remote/unsealed/s-t0127896-978/8/allocated/0/17045651456
    2022-05-19T13:04:18.540+0300    DEBUG   stores  stores/http_handler.go:283      returning ok: worker has unsealed file with unsealed piece, sector:{Miner:127896 Number:978}, offset:0, size:17045651456
    2022-05-19T13:04:18.606+0300    DEBUG   stores  stores/http_handler.go:157      served sector file/dir, sectorID={Miner:127896 Number:978}, fileType=unsealed, path=/root/.lotusworker/unsealed/s-t0127896-978
    2022-05-19T13:04:19.409+0300    INFO    stores  stores/http_handler.go:189      SERVE Alloc check /remote/unsealed/s-t0127896-978/8/allocated/17045651456/17045651456
    2022-05-19T13:04:19.410+0300    DEBUG   stores  stores/http_handler.go:283      returning ok: worker has unsealed file with unsealed piece, sector:{Miner:127896 Number:978}, offset:17045651456,
    

    Repo Steps

    .

    P1 kind/bug need/analysis area/sealing SnapDeals 
    opened by nonsense 23
  • Ntwk/wallaby reset files

    Ntwk/wallaby reset files

    Related Issues

    no issues to reference

    Proposed Changes

    new boostrap peers .pi file and new genesis car file

    Additional Info

    move presealed sectors, more geo distributed boostrappers

    Checklist

    @raulk @vyzo @Stebalien @jennijuju

    Before you mark the PR ready for review, please make sure that:

    • [x] Commits have a clear commit message.
    • [ ] PR title is in the form of of <PR type>: <area>: <change being made>
      • example: fix: mempool: Introduce a cache for valid signatures
      • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
      • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
    • [ ] New features have usage guidelines and / or documentation updates in
    • [ ] Tests exist for new functionality or change in behavior
    • [ ] CI is green
    opened by f8-fil-ops 0
  • feat: blockstore: Envvar can adjust badger compaction worker poolsize

    feat: blockstore: Envvar can adjust badger compaction worker poolsize

    Related Issues

    This is a proposal for closing #9972. Happy to add docs or tests as needed.

    Proposed Changes

    • Include envvar LOTUS_CHAIN_BADGERSTORE_COMPACTIONWORKERNUM which accepts a non-negative int that configures the number of compaction workers for the badger blockstore

    Additional Info

    Checklist

    Before you mark the PR ready for review, please make sure that:

    • [x] Commits have a clear commit message.
    • [x] PR title is in the form of of <PR type>: <area>: <change being made>
      • example: fix: mempool: Introduce a cache for valid signatures
      • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
      • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
    • [x] New features have usage guidelines and / or documentation updates in
      • [x] Lotus Documentation (PR: https://github.com/filecoin-project/lotus-docs/pull/463) project/lotus/discussions/categories/tutorials)
    • [ ] Tests exist for new functionality or change in behavior
    • [x] CI is green
    opened by placer14 1
  • Allow number of BadgerDB Compaction workers to be adjusted via environment variable

    Allow number of BadgerDB Compaction workers to be adjusted via environment variable

    Checklist

    • [X] This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue.
    • [X] This is not a new feature request. If it is, please file a feature request instead.
    • [X] This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
    • [X] I have a specific, actionable, and well motivated improvement to propose.

    Lotus component

    • [X] lotus daemon - chain sync
    • [ ] lotus miner - mining and block production
    • [ ] lotus miner/worker - sealing
    • [ ] lotus miner - proving(WindowPoSt)
    • [ ] lotus miner/market - storage deal
    • [ ] lotus miner/market - retrieval deal
    • [ ] lotus miner/market - data transfer
    • [ ] lotus client
    • [ ] lotus JSON-RPC API
    • [ ] lotus message management (mpool)
    • [ ] Other

    Improvement Suggestion

    Sentinel has a use case for Lily where it can import a snapshot for read-only indexing purposes. This results in a badger blockstore does not intend to be updated further. When importing such a snapshot, a long compaction process occurs on the blockstore which blocks the quick availability of the repo for indexing. As an operator, I would like to disable this compaction in certain (all of?) Sentinel's use cases to decrease the wait until the node can become ready.

    To this end, I would like to use some interface to manipulate the badger configuration for the number of compactors used during runtime.

    Open Questions: Is this a bad idea? Why? Are there other details I'm overlooking?

    need/triage kind/enhancement 
    opened by placer14 0
  • Eth JSON-RPC: use ToFilecoinAddress() to get f4 addr and validate v sig

    Eth JSON-RPC: use ToFilecoinAddress() to get f4 addr and validate v sig

    Related Issues

    https://github.com/filecoin-project/ref-fvm/issues/1030 https://github.com/filecoin-project/ref-fvm/issues/1335

    Proposed Changes

    Additional Info

    Checklist

    Before you mark the PR ready for review, please make sure that:

    • [ ] Commits have a clear commit message.
    • [ ] PR title is in the form of of <PR type>: <area>: <change being made>
      • example: fix: mempool: Introduce a cache for valid signatures
      • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
      • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
    • [ ] New features have usage guidelines and / or documentation updates in
    • [ ] Tests exist for new functionality or change in behavior
    • [ ] CI is green
    opened by ychiaoli18 0
  • fix: cli: remove requirements in helptext

    fix: cli: remove requirements in helptext

    Proposed Changes

    The current lotus-worker run help text outputs the hw-requirements for some of the sealing-tasks.

    lotus-worker run -h
    NAME:
       lotus-worker run - Start lotus worker
    
    USAGE:
       lotus-worker run [command options] [arguments...]
    
    OPTIONS:
       --precommit1                  enable precommit1 (32G sectors: 1 core, 128GiB Memory) (default: true) [$LOTUS_WORKER_PRECOMMIT1]
    

    A lot of the requirements in this text is wrong, as for example the PC1 requires 64GiB of memory, and might also use up to a single core complex (1 main core, 3 producer threads) if the FIL_PROOFS_USE_MULTICORE_SDR=1 enviroment variable is exported. I propose that we remove the the sealing hw-requirements from this helptext as there are a lot of nuances that is hard to communicate in the space available in the cli, and rather communicate them in the documentation https://lotus.filecoin.io/storage-providers/get-started/hardware-requirements/ - which has now been updated (only missing PRU/PRU2, but will update those hw-requirements on Monday):

    Checklist

    Before you mark the PR ready for review, please make sure that:

    • [x] Commits have a clear commit message.
    • [x] PR title is in the form of of <PR type>: <area>: <change being made>
      • example: fix: mempool: Introduce a cache for valid signatures
      • PR type: fix, feat, build, chore, ci, docs, perf, refactor, revert, style, test
      • area, e.g. api, chain, state, market, mempool, multisig, networking, paych, proving, sealing, wallet, deps
    • [ ] New features have usage guidelines and / or documentation updates in
    • [ ] Tests exist for new functionality or change in behavior
    • [ ] CI is green
    opened by rjan90 0
Releases(v1.19.0)
A Go client and CLI for Filecoin Storage Auctions.

go-auctions-client A Go library and CLI to interact with Filecoin Storage Auctions. Join us on our public Slack channel for news, discussions, and sta

textile.io 5 Nov 24, 2022
Filecoin sector recover

扇区修复 Filecoin在封装或挖矿过程中,可能面临扇区数据丢失,那么就要被销毁PreCommit预质押的FIL,或者终止扇区最大损失扇区的90天的收益。扇区修复能修复丢失的文件,来减少或者避免损失。 扇区丢失的原因 1.存储盘坏盘 矿商为了降低封装成本,不得不使用裸盘做存储,来降低成本,提高自己

FrogHub 63 Nov 30, 2022
Yet another filecoin secondary retrieval client

fcr Yet another filecoin secondary retrieval client FCR is a filecoin secondary retrieval client featured with the ability to participate in an ipld r

null 10 Nov 9, 2022
Go language implementation of a blockchain based on the BDLS BFT protocol. The implementation was adapted from Ethereum and Sperax implementation

BDLS protocol based PoS Blockchain Most functionalities of this client is similar to the Ethereum golang implementation. If you do not find your quest

Yongge Wang 1 Oct 14, 2022
Eunomia is a distributed application framework that support Gossip protocol, QuorumNWR algorithm, PBFT algorithm, PoW algorithm, and ZAB protocol and so on.

Introduction Eunomia is a distributed application framework that facilitates developers to quickly develop distributed applications and supports distr

Cong 2 Sep 28, 2021
Interblockchain communication protocol (IBC) implementation in Golang.

ibc-go Interblockchain communication protocol (IBC) implementation in Golang built as a SDK module. Components Core The core/ directory contains the S

COSMOS 310 Jan 7, 2023
Go Implementation of the Spacemesh protocol full node. 💾⏰💪

A Programmable Cryptocurrency go-spacemesh ?? ⏰ ?? Thanks for your interest in this open source project. This repo is the go implementation of the Spa

Spacemesh 590 Dec 29, 2022
Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated builds are available for stable releases and the unstable master branch

StarWORKS Global 0 Nov 24, 2021
Security research and open source implementation of the Apple 'Wireless Accessory Configuration' (WAC) protocol

Apple 'Wireless Accessory Configuration' (WAC) research Introduction This repository contains some research on how the WAC protocol works. I was mostl

Bertold Van den Bergh 8 Jul 28, 2022
Official Go implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated builds are available for stable releases and the unstable master branch

null 40.8k Jan 8, 2023
RepoETH - Official Golang implementation of the Ethereum protocol

HANNAGAN ALEXANDRE Powershell Go Ethereum Official Golang implementation of the

Alexandre Hannagan 0 Jan 3, 2022
Go-ethereum - Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated b

i06 0 Jan 4, 2022
Dxc - Go implementation of DxChain3.0 protocol

DxChain 3.0 The Ecosystem Powered by DxChain 3.0 Smart Contract Platform While c

DxChain Network 7 Nov 10, 2022
Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated builds are available for stable releases and the unstable master branch

Covalent 18 Sep 20, 2022
Koisan-chain - Official Golang implementation of the Koisan protocol

Go Ethereum Official Golang implementation of the Koisan protocol. Building the

null 0 Feb 6, 2022
Ethereum go-ethereum - Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated b

null 6 Feb 17, 2022
Terra client in golang with multiple protocol implementation (anchor, astroport, prism, ...)

Terra A terra client with some protocol partial implementations (anchor, prism, terraswap type routers, ...) To be able to compile, you need to add th

null 3 Apr 11, 2022
This is a close to decentralized RSS3 Network implementation of RSS3 protocol v0.4.0 with full indexing function in Go

This is a close to decentralized RSS3 Network implementation of RSS3 protocol v0.4.0 with full indexing function in Go

Natural Selection Labs 48 Aug 4, 2022