Type-safe Prometheus metrics builder library for golang

Overview

gotoprom

A Prometheus metrics builder

Build Status Coverage Status GoDoc Mentioned in Awesome Go

gotoprom offers an easy to use declarative API with type-safe labels for building and using Prometheus metrics. It doesn't replace the official Prometheus client but adds a wrapper on top of it.

gotoprom is built for developers who like type safety, navigating the code using IDEs and using a “find usages” functionality, making refactoring and debugging easier at the cost of performance and writing slightly more verbose code.

Motivation

Main motivation for this library was to have type-safety on the Prometheus labels, which are just a map[string]string in the original library, and their values can be reported even without mentioning the label name, just relying on the order they were declared in.

For example, it replaces:

httpReqs := prometheus.NewCounterVec(
    prometheus.CounterOpts{
        Name: "http_requests_total",
        Help: "How many HTTP requests processed, partitioned by status code and HTTP method.",
    },
    []string{"code", "method"},
)
prometheus.MustRegister(httpReqs)

// ...

httpReqs.WithLabelValues("404", "POST").Add(42)

With:

var metrics = struct{
	Reqs func(labels) prometheus.Counter `name:"requests_total" help:"How many HTTP requests processed, partitioned by status code and HTTP method."`
}

type labels struct {
	Code   int    `label:"code"`
	Method string `label:"method"`
}

gotoprom.MustInit(&metrics, "http")

// ...

metrics.Reqs(labels{Code: 404, Method: "POST"}).Inc()

This way it's impossible to mess the call by exchanging the order of "POST" & "404" params.

Usage

Define your metrics:

var metrics struct {
	SomeCounter                      func() prometheus.Counter   `name:"some_counter" help:"some counter"`
	SomeHistogram                    func() prometheus.Histogram `name:"some_histogram" help:"Some histogram with default prometheus buckets" buckets:""`
	SomeHistogramWithSpecificBuckets func() prometheus.Histogram `name:"some_histogram_with_buckets" help:"Some histogram with custom buckets" buckets:".01,.05,.1"`
	SomeGauge                        func() prometheus.Gauge     `name:"some_gauge" help:"Some gauge"`
	SomeSummaryWithSpecificMaxAge    func() prometheus.Summary   `name:"some_summary_with_specific_max_age" help:"Some summary with custom max age" max_age:"20m" objectives:"0.50,0.95,0.99"`

	Requests struct {
		Total func(requestLabels) prometheus.Count `name:"total" help:"Total amount of requests served"`
	} `namespace:"requests"`
}

type requestLabels struct {
	Service    string `label:"service"`
	StatusCode int    `label:"status"`
	Success    bool   `label:"success"`
}

Initialize them:

func init() {
	gotoprom.MustInit(&metrics, "namespace")
}

Measure stuff:

metrics.SomeGauge().Set(100)
metrics.Requests.Total(requestLabels{Service: "google", StatusCode: 404, Success: false}).Inc()

Custom metric types

By default, only some basic metric types are registered when gotoprom is intialized:

  • prometheus.Counter
  • prometheus.Histogram
  • prometheus.Gauge
  • prometheus.Summary

You can extend this by adding more types, for instance, if you want to observe time and want to avoid repetitive code you can create a prometheusx.TimeHistogram:

package prometheusx

import (
	"reflect"
	"time"

	"github.com/cabify/gotoprom"
	"github.com/cabify/gotoprom/prometheusvanilla"
	"github.com/prometheus/client_golang/prometheus"
)

var (
	// TimeHistogramType is the reflect.Type of the TimeHistogram interface
	TimeHistogramType = reflect.TypeOf((*TimeHistogram)(nil)).Elem()
)

func init() {
	gotoprom.MustAddBuilder(TimeHistogramType, RegisterTimeHistogram)
}

// RegisterTimeHistogram registers a TimeHistogram after registering the underlying prometheus.Histogram in the prometheus.Registerer provided
// The function it returns returns a TimeHistogram type as an interface{}
func RegisterTimeHistogram(name, help, namespace string, labelNames []string, tag reflect.StructTag) (func(prometheus.Labels) interface{}, prometheus.Collector, error) {
	f, collector, err := prometheusvanilla.BuildHistogram(name, help, namespace, labelNames, tag)
	if err != nil {
		return nil, nil, err
	}

	return func(labels prometheus.Labels) interface{} {
		return timeHistogramAdapter{Histogram: f(labels).(prometheus.Histogram)}
	}, collector, nil
}

// TimeHistogram offers the basic prometheus.Histogram functionality
// with additional time-observing functions
type TimeHistogram interface {
	prometheus.Histogram
	// Duration observes the duration in seconds
	Duration(duration time.Duration)
	// Since observes the duration in seconds since the time point provided
	Since(time.Time)
}

type timeHistogramAdapter struct {
	prometheus.Histogram
}

// Duration observes the duration in seconds
func (to timeHistogramAdapter) Duration(duration time.Duration) {
	to.Observe(duration.Seconds())
}

// Since observes the duration in seconds since the time point provided
func (to timeHistogramAdapter) Since(duration time.Time) {
	to.Duration(time.Since(duration))
}

So you can later define it as:

var metrics struct {
	DurationSeconds func() prometheusx.TimeHistogram `name:"duration_seconds" help:"Duration in seconds" buckets:".001,.005,.01,.025,.05,.1"`
}

func init() {
	gotoprom.MustInit(&metrics, "requests")
}

And use it as:

// ...
defer metrics.DurationSeconds().Since(t0)
// ...

Replacing metric builders

If you don't like the default metric builders, you can replace the DefaultInitializer with your own one.

Performance

Obviously, there's a performance cost to perform the type-safety mapping magic to the original Prometheus client's API.

In general terms, it takes 3x to increment a counter than with vanilla Prometheus, which is around 600ns (we're talking about a portion of a microsecond, less than a thousandth of a millisecond)

$ go test -bench . -benchtime 3s
goos: darwin
goarch: amd64
pkg: github.com/cabify/gotoprom
BenchmarkVanilla-4    	10000000	       387 ns/op
BenchmarkGotoprom-4   	 5000000	      1049 ns/op
PASS
ok  	github.com/cabify/gotoprom	10.611s

In terms of memory, there's a also a 33% increase in terms of space, and 3x increase in allocations:

$ go test -bench . -benchmem
goos: darwin
goarch: amd64
pkg: github.com/cabify/gotoprom
BenchmarkVanilla-4    	 5000000	       381 ns/op	     336 B/op	       2 allocs/op
BenchmarkGotoprom-4   	 1000000	      1030 ns/op	     432 B/op	       6 allocs/op
PASS
ok  	github.com/cabify/gotoprom	3.369s

This costs are probably assumable in most of the applications, especially when measuring network accesses, etc. which are magnitudes higher.

Issues
  • Included the implementation builder for summaries

    Included the implementation builder for summaries

    This pull request adds prometheu.Summarys to the library.

    opened by mapno 2
  • Adding support for unsigned integers

    Adding support for unsigned integers

    Adding support for unsigned integers, the same way we already have support for signed ones.

    opened by juananrey 1
  • Remove the default DefaultBuckets, make them configurable

    Remove the default DefaultBuckets, make them configurable

    When buckets for a Histogram are not defined explicitly, we set some DefaultBuckets: https://github.com/cabify/gotoprom/blob/b6bf805c5c9ff2e7dad23c127ef5e9876453c522/prometheusvanilla/builders.go#L123-L125

    Those may fit you or may not, fit you. We actually already changed them once and that was a breaking change for our customers.

    I'd say that there should not be any default values and everyone should provide explicitly the buckets they want.

    OTOH, since there's a clear wish for having default values here, we can make the default values being optional.

    So I think that the solution is a Builder factory, like:

    // HistogramBuilder returns a builder for the `prometheus.Histogram` metric type with the
    // default buckets set to the provided ones.
    // Default buckets will be used if specific ones were not provided through a tag.
    // If default buckets are set to nil and metric does not provide ones, metric building will fail. 
    func HistogramBuilder(defaultBuckets []float64) gotoprom.Builder {
        ...
    }
    

    With default initializer configured as prometheusvanilla.HistogramBuilder(nil)

    opened by colega 1
  • Release v1.0.0

    Release v1.0.0

    We've been stuck to v0.x.y because github.com/prometheus/client_golang was on v0.i.j, but since official prometheus v1.0.0 is out and considered stable, we can release a stable version too (with some breaking changes, see the milestone)

    opened by colega 1
  • Change the builder for prometheus.Observer by a builder for prometheus.Histogram

    Change the builder for prometheus.Observer by a builder for prometheus.Histogram

    Initially we didn't offer a builder for summaries and we used the prometheus.Observer target interface to bind the builder for prometheus.Histogram. That's not exactly correct and the v1.0.0 release is a good moment to fix that.

    This would be a breaking change so we should reduce the pain on this for our customers (since upgrading from v0 to v1 does not change the package).

    I'd propose to rename BuildObserver to BuildHistogram and register that builder for prometheus.Histogram and stop registering it for prometheus.Observer.

    State that in the release notice allowing people to:

    • Change their code to the new type in the builder (otherwise it would not build the metrics)
    • Register BuildHistogram as a builder for prometheus.Observer in init() of their main package if someone depends on a third party code building the metrics which doesn't update.
    opened by colega 1
  • Allow passing summary objectives through the tags

    Allow passing summary objectives through the tags

    Prometheus removed the default 0.5, 0.9 and 0.95 summary objectives in the v1.0.0, so now the summaries this library builds have no objectives at all, and act like an histogram with no buckets (only a sum and a count are exposed).

    We should add the ability of providing those objectives through the tags of the summary when it's defined.

    good first issue 
    opened by colega 1
  • `prometheusvanilla` is not included in coveralls report

    `prometheusvanilla` is not included in coveralls report

    Make sure it's tested in the CI, and included in the coverage report.

    opened by colega 1
  • Added support of empty values for labels

    Added support of empty values for labels

    Maybe I should call it "default"? We can explicitly report a value if none was set on a string. It can become too repetitive in the code if you have 5 labels in a metric.

    opened by colega 1
  • Initial README.md

    Initial README.md

    Lets start moving this or something

    opened by colega 1
Releases(v1.1.0)
  • v1.1.0(Jan 29, 2020)

  • v1.0.0(Jan 10, 2020)

    Added

    • Support for empty buckets tag, which will generate nil buckets for the prometheus Histogram and use default prometheus buckets.
    • Support for empty objectives tag, which will generate nil objectives for the prometheus Summary and use an empty objectives map after all.

    Changed

    • Breaking: prometheus.Histogram is now used to build histograms, instead of prometheus.Observer, which means that previous code building prometheus.Observer won't compile anymore.

    Removed

    • Breaking: default buckets on histograms. All histogram should explicitly specify their buckets now or they will fail to build.
    • Breaking: default objectives on summaries. All summaries should explicitly specify their objectives now or they will fail to build.

    Fixed

    • Summary building was not failing with malformed objectives.

    Migration instructions

    If you're migrating from a v0.x.y, you will need to:

    • Replace Metric func() prometheus.Observer by Metric func() prometheus.Histogram
    • On prometheus.Histogram metrics, add buckets:"" which will use the prometheus.DefBuckets bucekts, or even better, define yours.
    • On prometheus.Summary metrics, add objectives:".5,.95,.99" to keep using the same objectives, or define yours.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Oct 10, 2019)

  • v0.2.1(Jun 5, 2019)

  • v0.1.1(May 8, 2019)

Owner
Cabify
Cabify
Generate type-safe Go converters by simply defining an interface

goverter a "type-safe Go converter" generator goverter is a tool for creating type-safe converters. All you have to do is create an interface and exec

Jannis Mattheis 92 Nov 30, 2021
Type-safe atomic values for Go

Type-safe atomic values for Go One issue with Go's sync/atomic package is that there is no guarantee from the type system that operations on an intege

Alec Thomas 9 Dec 4, 2021
[TOOL, CLI] - Filter and examine Go type structures, interfaces and their transitive dependencies and relationships. Export structural types as TypeScript value object or bare type representations.

typex Examine Go types and their transitive dependencies. Export results as TypeScript value objects (or types) declaration. Installation go get -u gi

Daniel T. Gorski 135 Nov 23, 2021
A tool to run queries in defined frequency and expose the count as prometheus metrics.

A tool to run queries in defined frequency and expose the count as prometheus metrics. Supports MongoDB and SQL

S Santhosh Nagaraj 19 Oct 21, 2020
Prometheus support for go-metrics

go-metrics-prometheus This is a reporter for the go-metrics library which will post the metrics to the prometheus client registry . It just updates th

Csergő Bálint 60 Oct 7, 2021
rsync wrapper (or output parser) that pushes metrics to prometheus

rsync-prom An rsync wrapper (or output parser) that pushes metrics to prometheus. This allows you to then build dashboards and alerting for your rsync

Michael Stapelberg 17 Nov 11, 2021
🚀 Backend for Online Courses Builder | SaaS Product

Backend app for Online Course Constructor Platform Build & Run (Locally) Prerequisites go 1.15 docker golangci-lint (optional, used to run code checks

Maksim Zhashkevych 140 Nov 29, 2021
A Visual Go REST API boilerplate builder.

A Visual Go REST API boilerplate builder. The boilerplate builder will export a Go web server with 0 dependencies, besides the ones you add.

The Strukture IDE 49 Nov 4, 2021
Go port of Coda Hale's Metrics library

go-metrics Go port of Coda Hale's Metrics library: https://github.com/dropwizard/metrics. Documentation: http://godoc.org/github.com/rcrowley/go-metri

Richard Crowley 3.1k Nov 25, 2021
atomic measures + Prometheus exposition library

About Atomic measures with Prometheus exposition for the Go programming language. This is free and unencumbered software released into the public doma

Pascal S. de Kloe 22 Nov 1, 2021
Prometheus instrumentation library for Go applications

Prometheus Go client library This is the Go client library for Prometheus. It has two separate parts, one for instrumenting application code, and one

Prometheus 3.5k Nov 29, 2021
Toy program for benchmarking safe and unsafe ways of saving a file

save-a-file benchmarks the many strategies an editor could use to save a file. Example output on a SSD: ext4: $ ./save-a-file ~/tmp/foo 29.195µs per s

null 2 Sep 13, 2021
Analyze the binary outputted by `go build` to get type information etc.

Analyze the binary outputted by go build to get type information etc.

Masaaki Goshima 16 Aug 9, 2021
IBus Engine for GoVarnam. An easy way to type Indian languages on GNU/Linux systems.

IBus Engine For GoVarnam An easy way to type Indian languages on GNU/Linux systems. goibus - golang implementation of libibus Thanks to sarim and haun

Varnamproject 9 Oct 30, 2021
Lithia is an experimental functional programming language with an implicit but strong and dynamic type system.

Lithia is an experimental functional programming language with an implicit but strong and dynamic type system. Lithia is designed around a few core concepts in mind all language features contribute to.

Valentin Knabel 4 Nov 12, 2021
A tool to generate Pulumi Package schemas from Go type definitions

MkSchema A tool to generate Pulumi Package schemas from Go type definitions. This tool translates annotated Go files into Pulumi component schema meta

Joe Duffy 1 Oct 18, 2021
a tool for getting metrics in containers

read metrics in container if environment is container, the cpu ,memory is relative to container, else the metrics is relative to host. juejing link :

null 6 Nov 30, 2021
Collect and visualize metrics from Brigade 2

Brigade Metrics: Monitoring for Brigade 2 Brigade Metrics adds monitoring capabilities to a Brigade 2 installation. It utilizes Brigade APIs to export

Brigade 3 Nov 19, 2021
Count Dracula is a fast metrics server that counts entries while automatically expiring old ones

In-Memory Expirable Key Counter This is a fast metrics server, ideal for tracking throttling. Put values to the server, and then count them. Values ex

Mailsac 13 Dec 3, 2021