A db for bitcoin-sv & BTC

Related tags

Cryptography go-svdb
Overview

Welcome to go-svdb Project

===========

Boquan Team

The Boquan is a team dedicated to promoting and developing true bitcoin.

The team has successfully developed:

Bitcoin mine pool mempool.com (https://pool.mempool.com), the mine pool supports BTC and BSV mining, and is currently the largest BSV mine in China.

Bitcoin wallet Lastpusre ( https://lastpurse.com ).

Introduce

go-svdb Project is a bitcoin database project. It implements a query service on bitcoin-sv. It also supports BTC.

go-svdb is a software based on the golang.

The purpose of the go-svdb project is that Bitdb did not meet the needs of the boquan team for the project.

The go-svdb service is currently used in three projects of the boquan team:   

  • mempool mining pool: pool.mempool.com, the Chinese largest bitcoin-sv mine pool.
  • LastPurse: lastpurse.com/ddpurse.com, the WeChat bsv wallet witch has most user on the world.
  • btc/bsv blockchain browser: bsv.mempool.com.

It also works for BTC.

Because a reliable, complete go-svdb service requires more servers, the next boquan team will launch a Metanet-based query service.

Technical briefing

go-svdb mainly crawls data on the chain through three processes, block_crawler, output_crawler, and input_crawler.

The input_crawler is designed to depend on the output_crawler, and the output_crawler depends on the block_crawler.

Process division:    + block_crawler crawls the header information of the block, so block_crawler crawls fastest;    + output_crawler crawls the output of each transaction;    + input_crawler organizes the transactions on the block into two tables: one is the address inventory table with the address as the index, and the other is the transaction table with the txid as the index;    + monitored_addr_crawler takes business-related data to an acceleration table;    + tx_listener listens for transactions in the memory pool.

This way of crawling for crawling with only one process has several advantages:

If you use only one process,or just get data block by block, then the database can not be aware of the new block in time.

In the case of continuous new blocks, crawling a complete block takes a certain amount of time, so that the program cannot perceive the new block as early as possible.

This can lead to some problems: for example, the orphon block can't be found immediately, or the browser fails to keep up with the block height in time. Svdb's block_crawler only crawls the head, which is extremely efficient and can solve the above problems on some level.

In addition, due to the characteristics of the Bitcoin network, the full node only stores utxo, and does not care about the input of a particular transaction in history. Therefore, svdb first uses the output_crawler to quickly crawl all the output first so that input_crawler can crawls complete block, parsing the input of the transaction easier

Finally, the input_crawler according to the data prepared by block_crawler and output_crawler, along the blockchain all the way from low to high, all the transactions on the chain are counted into two tables according to the address(addr_tx) and txid(tx).

The block_crawler, output_crawler, and input_crawler processes establish a complete set of data on the chain. On this basis, the monitored_addr_crawler is for the business we need.

Because the data on the entire blockchain is huge, in order to optimize the business of the Boquan team, we use another process to crawl the data from the table addr_tx and table tx, and the data of the business-related will be insert into another table.

In addition, because the Bitcoin network has a new block in about 10 minutes, in order to obtain the corresponding transaction information as early as possible, and zero confirmation , the tx_listener process will continuously listen to all transactions in the entire node memory pool. Transactions that are already in the memory pool are first populated into the database by height equal -1.

go-svdb is currently able to query all data on the chain and correct the data if the data such as a lone block is rolled back.

How to deploy

At present, go-svdb is deployed in the Boquan team, which is deployed in 4 8 Core CPUs + 64G RAM + 4T DISK, two of which run BSV and two run BTC.

The database uses Docker to run Mongo.

The size of the hard drive grows with the size of the blockchain. It is recommended to use lvm technology.

The operating system uses CentOS 7.x.

When you run the blockchain data at full speed, you can basically take advantage of the eight-core CPU.

Route Map

  1. Large block support, the current go-svdb internal version has been tested on the STN network.
  2. Unlimited capacity support, since go-svdb uses mongodb to store data.
  3. A public inquiry service based on Metanet.
  4. Support parse the Tokenized on blockchain.

Permisson

go-svdb is based on the Open BSV license. Please read LINCENSE.

License

go-svdb is released under the terms of the Open BSV license. See LICENSE for more information.

Owner
mempool.com team
created mempool.com and dotwallet.com
mempool.com team
Go implementation of a vanity attempt to generate Bitcoin private keys and subsequently checking whether the corresponding Bitcoin address has a non-zero balance.

vanity-BTC-miner Go implementation of a vanity attempt to generate Bitcoin private keys and subsequently checking whether the corresponding Bitcoin ad

Lih Ingabo 1 Jun 3, 2022
A mining pool proxy tool, support BTC, ETH, ETC, XMR mining pool, etc.

Tier2Pool A mining pool proxy tool, support BTC, ETH, ETC, XMR mining pool, etc. Build I use Ubuntu as a demo. sudo update sudo apt install git make s

Tier2Pool 6 Jul 29, 2022
A full node Bitcoin (BSV) implementation written in Go

bsvd bsvd is a full node Bitcoin (BSV) implementation written in Go (golang). This project is a port of the bchd codebase to Bitcoin (BSV). It provide

null 41 Feb 7, 2022
Moeing chain is an EVM&Web3 compatible sidechain for Bitcoin Cash

Full node client of smartBCH This repository contains the code of the full node client of smartBCH, an EVM&Web3 compatible sidechain for Bitcoin Cash.

null 118 Jul 12, 2022
Store data on Bitcoin for 350 sats/KB up to 185 KB by using P2SH-P2WSH witness scripts

Bitcandle Store data on Bitcoin for 350 sats/KB up to 185 kB by using P2SH-P2WSH witness scripts. 225ed8bc432d37cf434f80717286fd5671f676f12b573294db72

Aurèle Oulès 10 Nov 14, 2021
A curated Golang toolkit for creating Bitcoin SV powered apps

bsv A curated Golang toolkit for creating Bitcoin SV powered apps Table of Contents Installation Maintainers License Installation bsv requires a suppo

null 21 May 10, 2022
A work-in-progress Bitcoin wallet based on Output Descriptors

go-wallet A work-in-progress Bitcoin wallet Descriptors go-wallet is designed around Bitcoin Descriptors. It implements a Output Script Descriptors la

Gustavo Chaín 5 May 4, 2022
The go-to Bitcoin Node (BN) Go library.

go-bitcoin Go wrapper for bitcoin RPC RPC services Start by creating a connection to a bitcoin node b, err := New("rcp host", rpc port, "rpc usernam

null 3 Feb 13, 2022
Bitcoin CPU miner written in Go.

CPU Miner Bitcoin CPU miner written in Go. Introduction This is a CPU miner written in Go. It is a proof of concept and is not intended for production

Yaroslav Gaponov 0 Jan 6, 2022
A fully validating Bitcoin node with Utreexo support

btcd btcd is an alternative full node bitcoin implementation written in Go (golang). This project is currently under active development and is in a Be

Utreexo 11 Jul 7, 2022
Bitcoin futures curve from Deribit as a JSON webservice

Curve Bitcoin futures curve from Deribit as a JSON webservice Building go build . Running ./curve Expiration date and annualised yield of each contr

Steven Wilkin 0 Dec 13, 2021
Bitcoin Core integration/staging tree

Bitcoin Core integration/staging tree https://bitcoincore.org For an immediately usable, binary version of the Bitcoin Core software, see https://bitc

Bitcoin 65.5k Aug 2, 2022
Mastering Bitcoin 2nd Edition - Programming the Open Blockchain

Code Examples: Mastering Bitcoin Mastering Bitcoin is a book for developers, although the first two chapters cover bitcoin at a level that is also app

Mastering Bitcoin 20.1k Aug 1, 2022
Bitcoin Improvement Proposals

People wishing to submit BIPs, first should propose their idea or document to the [email protected] mailing list (do not assign a

Bitcoin 7.1k Aug 9, 2022
Full bitcoin solution written in Go (golang)

About Gocoin Gocoin is a full Bitcoin solution written in Go language (golang). The software architecture is focused on maximum performance of the nod

Piotr Narewski 873 Jul 30, 2022
A simple, concurrent bitcoin miner framework implemented in Go.

Bitcoin Miner A simple, concurrent bitcoin miner framework implemented in Go. Disclaimer: this is not a product intended to be used for real mining, s

null 4 May 30, 2022
Bitcoin address balance checker on steroids.

BTCSteroids Bitcoin address balance checker on steroids. Table of contents Quick start What's included Use Cases Thanks Copyright and license Quick st

null 7 Jun 2, 2022
Bitcoin UTXO & xPub Management Suite

BUX Bitcoin UTXO & xPub Management Suite Table of Contents About Installation Documentation Examples & Tests Benchmarks Code Standards Usage Contribut

BUX 14 Jul 27, 2022