A Prometheus exporter for Ethereum Execution & Consensus clients

Overview

πŸ¦„ Ethereum Metrics Exporter πŸ¦„

A Prometheus metrics exporter for Ethereum execution & consensus nodes

Ethereum client implementations expose extensive Prometheus metrics however there is minimal standardization around the metrics structure. This makes observability across multiple clients a painful experience. This exporter hopes to help alleviate this problem by creating a client-agnostic set of metrics that operators can run without any additional configuration to dashboards or alerting.

To provide these client-agnostic metrics the exporter relies entirely on these well-defined APIs:

This means that the exporter is limited to metrics that are exposed by these APIs.

Built With

Usage

A tool to report the state of ethereum nodes

Usage:
  ethereum-metrics-exporter [flags]
  ethereum-metrics-exporter [command]

Available Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  serve       Run a metrics server and poll the configured clients.

Flags:
      --config string                   config file (default is $HOME/.ethereum-metrics-exporter.yaml)
      --consensus-url string            (optional) URL to the consensus node
      --execution-url string            (optional) URL to the execution node
  -h, --help                            help for ethereum-metrics-exporter
      --metrics-port int                Port to serve Prometheus metrics on (default 9090)
      --monitored-directories strings   (optional) directories to monitor for disk usage
  -t, --toggle                          Help message for toggle

Getting Started

Docker

Available as a docker image at samcm/ethereum-metrics-exporter

Quick start

docker run --name ethereum-metrics-exporter -p 9090:9090 -it samcm/ethereum-metrics-exporter --consensus-url=http://localhost:5052 --execution-url=http://192.168.0.188:8545

With a config file

docker run --name ethereum-metrics-exporter -v $HOST_DIR_CHANGE_ME/config.yaml:/opt/exporter/config.yaml -p 9090:9090 -it samcm/ethereum-metrics-exporter --config /opt/exporter/config.yaml

Standalone

Downloading a release

Coming soon.

Building yourself (requires Go)

  1. Clone the repo
    go get github.com/samcm/ethereum-metrics-exporter
  2. Change directories
    cd ./ethereum-metrics-exporter
  3. Build the binary
     go build -o ethereum-metrics-exporter .
  4. Run the exporter
     ./ethereum-metrics-exporter

Contributing

Contributions are greatly appreciated! Pull requests will be reviewed and merged promptly if you're interested in improving the exporter!

  1. Fork the project
  2. Create your feature branch:
    • git checkout -b feat/new-metric-profit
  3. Commit your changes:
    • git commit -m 'feat(profit): Export new metric: profit
  4. Push to the branch: -git push origin feat/new-metric-profit
  5. Open a pull request

Contact

Sam - @samcmau

Issues
  • txpool_status does not exist

    txpool_status does not exist

    It seems like this project is calling a txpool_status method on the execution RPC endpoing which does not exist on Geth 1.10.18-stable.

    Geth is complaining with these messages in the log:

    WARN [06-06|19:45:59.953] Served txpool_status                     conn=127.0.0.1:55350 reqid=0 duration="37.21Β΅s"   err="the method txpool_status does not exist/is not available"
    
    opened by remyroy 2
  • feat(pair): Add consensus_mechanism metric

    feat(pair): Add consensus_mechanism metric

    Adds a new exporter job, pair, which requires both the execution & consensus modules to be enabled. There is a single metric exported:

    eth_pair_consensus_mechanism{consensus_mechanism="Proof of Stake",consensus_mechanism_short="PoS"} 3
    
    opened by samcm 0
  • feat(consensus): Create beacon.Node

    feat(consensus): Create beacon.Node

    Creates a new package consensus/beacon. This package is intended to be the central point of reference for the jobs in the jobs packages.

    This will faciliate us exporting complex metrics, like expected slot/epoch, missed blocks, etc.

    opened by samcm 0
  • feat(consensus): Export state roots as a bounded hash range

    feat(consensus): Export state roots as a bounded hash range

    This change adds two new metrics:

    • eth_con_beacon_finality_checkpoint_hash following the head checkpoints for finalized/justified/previous_justified
    • eth_con_beacon_head_slot_hash following the head slot

    These metrics observe the state roots, and consistently convert them in to an integer between 0 and 65536. These metrics can then be used to observe which chain a beacon node is following by comparing this value at the current time to other nodes.

    To work around scrape intervals, a query like this can be used to try and avoid noise: max(eth_con_beacon_head_slot_hash)[1m].

    opened by samcm 0
  • feat(consensus): Reduce polling frequency

    feat(consensus): Reduce polling frequency

    • Rely on events entirely for following the consensus slots & finality checkpoints
    • Added a new metric that reports when the last subscription event we received was (roughly)
    • Added reconnect logic to the subscription in case the CL is restarted
    opened by samcm 0
  • Failed to request signed beacon block

    Failed to request signed beacon block

    Hi,

    Trying out this awesome exporter with Prometheus and Grafana and got it working, but still see an error in the logs. Using Geth (latest) for EL and Prysm (latest) for CL.

    The error in the logs show this:

    ethereum-metrics-exporter[25186]: {"component":"exporter","error":"failed to request signed beacon block: GET failed with status 500: {\"message\":\"GetBlockV2: rpc error: code = NotFound desc = Could not find requested block: signed beac on block can't be nil\",\"code\":500}","level":"error","module":"consensus/beacon","msg":"Subscriber error","time":"2022-08-01T11:21:00+02:00","topic":"block"}

    Is this a misconfiguration from my side or could it be something else?

    Flags used: --consensus-url=http://localhost:3500 --execution-url=http://localhost:8545

    Prysm public API: https://docs.prylabs.network/docs/how-prysm-works/prysm-public-api

    opened by Jor-Tech 0
  • Add metric for basefee

    Add metric for basefee

    The EL blocks contain a field called basefee. This basefee fluctuation is something we'd like to track on shadow forks. It'd be great if we can add this and update the dashboard to visualize the fluctuations.

    opened by parithosh 1
Releases(v0.16.5)
  • v0.16.5(Jul 21, 2022)

    What's Changed

    • fix(consensus): Reduce duplicate subscriptions by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/40

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.16.4...v0.16.5

    Source code(tar.gz)
    Source code(zip)
  • v0.16.4(Jul 21, 2022)

    What's Changed

    • chore: increase golangci-lint timeout and set verbose logging by @skylenet in https://github.com/samcm/ethereum-metrics-exporter/pull/38
    • chore: run docker build on PRs by @skylenet in https://github.com/samcm/ethereum-metrics-exporter/pull/37
    • feat(consensus): Sleep the subscription loop until genesis by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/39

    New Contributors

    • @skylenet made their first contribution in https://github.com/samcm/ethereum-metrics-exporter/pull/38

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.16.3...v0.16.4

    Source code(tar.gz)
    Source code(zip)
  • v0.16.3(Jul 14, 2022)

    What's Changed

    • fix(execution): Panic fix - check for nil block by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/36

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.16.2...v0.16.3

    Source code(tar.gz)
    Source code(zip)
  • v0.16.2(Jul 12, 2022)

    What's Changed

    • fix(consensus): No-op event resub if we are pre-genesis by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/35

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.16.1...v0.16.2

    Source code(tar.gz)
    Source code(zip)
  • v0.16.1(Jul 12, 2022)

  • v0.16.0(Jul 12, 2022)

    What's Changed

    • feat(consensus): Create beacon.Node by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/31

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.15.0...v0.16.0

    Source code(tar.gz)
    Source code(zip)
  • v0.15.0(Jul 5, 2022)

    What's Changed

    • feat(pair): Add consensus_mechanism metric by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/32

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.14.0...v0.15.0

    Source code(tar.gz)
    Source code(zip)
  • v0.14.0(Jun 29, 2022)

    What's Changed

    • feat(consensus): Export state roots as a bounded hash range by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/30

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.13.1...v0.14.0

    Source code(tar.gz)
    Source code(zip)
  • v0.13.1(Jun 23, 2022)

    What's Changed

    • feat(consensus): Slow down resubscribes by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/29

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.13.0...v0.13.1

    Source code(tar.gz)
    Source code(zip)
  • v0.13.0(Jun 23, 2022)

    What's Changed

    • feat(consensus): Reduce polling frequency by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/27

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.12.0...v0.13.0

    Source code(tar.gz)
    Source code(zip)
  • v0.12.0(Jun 14, 2022)

    What's Changed

    • feat(consensus): Remove slots from general exporter by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/25

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.11.1...v0.12.0

    Source code(tar.gz)
    Source code(zip)
  • v0.11.1(Jun 11, 2022)

    What's Changed

    • feat(consensus): Remove justified beacon slot fetching by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/24

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.11.0...v0.11.1

    Source code(tar.gz)
    Source code(zip)
  • v0.11.0(Jun 7, 2022)

    What's Changed

    • feat(consensus): Add Peers metrics by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/22
    • feat(execution): Disable TXPool module by default by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/23

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.10.0...v0.11.0

    Source code(tar.gz)
    Source code(zip)
  • v0.10.0(Jun 6, 2022)

    What's Changed

    • feat(consensus): Add Beacon and Event jobs by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/20

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.9.0...v0.10.0

    Source code(tar.gz)
    Source code(zip)
  • v0.9.0(Jun 3, 2022)

    What's Changed

    • feat(consensus): Expose finality checkpoints by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/19

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.8.4...v0.9.0

    Source code(tar.gz)
    Source code(zip)
  • v0.8.4(May 19, 2022)

    What's Changed

    • fix(consensus): Fix a panic if a bad consensus block is fetched by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/18

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.8.3...v0.8.4

    Source code(tar.gz)
    Source code(zip)
  • v0.8.3(May 18, 2022)

    What's Changed

    • fix: panic when registering uninitialized metric by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/17

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.8.2...v0.8.3

    Source code(tar.gz)
    Source code(zip)
  • v0.8.1(May 18, 2022)

  • v0.8.0(May 18, 2022)

    What's Changed

    • add net and web3 modules, disable admin module by default by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/15
    • chore: linting + add golangci-lint job by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/16

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.7.0...v0.8.0

    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(May 17, 2022)

    What's Changed

    • feat(execution): Fetch total difficulty from the Head block rather th… by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/14

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.6.0...v0.7.0

    Source code(tar.gz)
    Source code(zip)
  • v0.6.0(May 16, 2022)

    What's Changed

    • fix: execution head block transaction count by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/13

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.5.0...v0.6.0

    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(May 16, 2022)

    What's Changed

    • feat(execution): Add most recent execution block stats by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/12

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.4.0...v0.5.0

    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(May 16, 2022)

    What's Changed

    • Feat/current epoch by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/10
    • chore: Add missing comments by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/11

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.3.0...v0.4.0

    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(May 16, 2022)

    What's Changed

    • feat(args): Add execution-modules argument by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/9

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/v0.2.0...v0.3.0

    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(May 13, 2022)

    What's Changed

    • feat(cmd): Run serve from root command and support url args by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/1
    • Feat(exporter): disk usage & node info by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/2
    • feat(network): Add metrics about the spec and chain of the network by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/3
    • chore(jobs): Break metrics up in to jobs by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/4
    • feat(network): Add General/Network level metrics by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/5
    • feat(merge): Add total difficulty from exe by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/6
    • chore: Restructure execution jobs by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/7
    • feat(consensus): Add chain reorgs by @samcm in https://github.com/samcm/ethereum-metrics-exporter/pull/8

    New Contributors

    • @samcm made their first contribution in https://github.com/samcm/ethereum-metrics-exporter/pull/1

    Full Changelog: https://github.com/samcm/ethereum-metrics-exporter/compare/latest...v0.2.0

    Source code(tar.gz)
    Source code(zip)
Owner
Sam Calder-Mason
DevOps @ethereum
Sam Calder-Mason
Ethereum go-ethereum - Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated b

null 6 Feb 17, 2022
A phoenix Chain client based on the go-ethereum fork,the new PoA consensus engine is based on the VRF algorithm.

Phoenix Official Golang implementation of the Phoenix protocol. !!!The current version is for testing and developing purposes only!!! Building the sou

g_master 14 Apr 28, 2022
Tool for monitoring your Ethereum clients. Client-agnostic as it queries the standardized JSON-RPC APIs

e7mon Tool for monitoring your Ethereum clients. Client-agnostic as it queries the standardized JSON-RPC APIs. However, the execution client should be

null 25 Jul 7, 2022
Ethereum clients monitor

e7mon Tool for monitoring your Ethereum clients. Client-agnostic as it queries the standardized JSON-RPC APIs. Requires the following APIs to be expos

null 25 Jul 7, 2022
ConsenSys Software 8 Jan 18, 2022
LEO (Low Ethereum Orbit) is an Ethereum Portal Network client.

LEO LEO (Low Ethereum Orbit) is an Ethereum Portal Network client. What makes LEO different from other Portal Network clients is that it uses libp2p f

Valist, Inc. 10 Apr 19, 2022
Ethereum-vanity-wallet - A fork of https://github.com/meehow/ethereum-vanity-wallet but the key can be exported to a JSON keystore file

ethereum-vanity-wallet See https://github.com/meehow/ethereum-vanity-wallet This version: doesn't display the private key let's you interactively expo

null 0 Jan 2, 2022
Go-ethereum - Official Golang implementation of the Ethereum protocol

Go Ethereum Official Golang implementation of the Ethereum protocol. Automated b

i06 0 Jan 4, 2022
This library aims to make it easier to interact with Ethereum through de Go programming language by adding a layer of abstraction through a new client on top of the go-ethereum library.

Simple ethereum client Simple ethereum client aims to make it easier for the developers to interact with Ethereum through a new layer of abstraction t

Jero 3 May 1, 2022
A blockcypher exporter to monitor your blockchain nodes to Prometheus.

A blockcypher exporter to monitor your blockchain nodes to Prometheus. Metrics are retrieved using the BlockCypher REST API.

iderr 0 Nov 2, 2021
Rei chain fork from quorum using raft consensus

GoQuorum is an Ethereum-based distributed ledger protocol with transaction/contract privacy and new consensus mechanisms. GoQuorum is a fork of go-eth

Moon Rhythm 12 Mar 31, 2022
Slides about IOTA transactions, validation and consensus

IOTA Donations: NJNCOKJOE9SMCYTSBZVTGWMAABPBYELV9SBPUYLKWSTCXQQZDUWHTFLTVKKRBBWSZKPDMNQALJMJX9CG9KAMOJXQVW IOTA Transactions, Confirmation and Consens

null 106 Mar 15, 2022
Go-chain - EVM-compatible chain secured by the Lachesis consensus algorithm

ICICB galaxy EVM-compatible chain secured by the Lachesis consensus algorithm. B

Talented Blockchain Developer 4 Jun 8, 2022
Pure Go Kerberos library for clients and services

gokrb5 It is recommended to use the latest version: Development will be focused on the latest major version. New features will only be targeted at thi

Jonathan Turner 595 Jul 28, 2022
Supports the safe and convenient execution of asynchronous computations with goroutines and provides facilities for the safe retrieval of the computation results.

Rendezvous The Rendezvous library supports the safe and convenient execution of asynchronous computations with goroutines and provides facilities for

Paulo Villela 0 Dec 29, 2021
Huobi Eco Chain client based on the go-ethereum fork

The Huobi Open Platform is a unified infrastructure platform based on the technical, traffic and ecological resources of the Huobi Group, and will be gradually open to the blockchain industry.

null 236 Aug 2, 2022
Streaming Fast on Ethereum

Stream Ethereum data like there's no tomorrow

StreamingFast 69 Jul 9, 2022
a Golang sdk for working with DeFi protocols, and ethereum compatible blockchains

A golang sdk for working with DeFi protocols and general utilities for working with ethereum-compatible blockchains. packages bclient bindings cli con

bonedaddy 65 Jul 30, 2022
run ABI encoded data against the ethereum blockchain

Run EVM code against a database at a certain block height - Note You can't run this against a running geth node - because that would share the db and

Edgar Aroutiounian 60 Nov 11, 2021