A proxy to add `/federate` to Thanos monitoring

Related tags

thanos-federate-proxy
Overview

Thanos Federate Proxy

A proxy to convert /federate queries to /v1/api/query and respond in open metrics format.

The most common use case for this proxy is to be used as a side car with Thanos, to provide /federate api for thanos (currently Thanos does not support it). So this way you can add Thanos as a federation source in another prometheus. Also Thanos does not support remote write (Note that being able to write metrics remotely from thanos to another prometheus is a different concept than thanos receiver component).

Usage

Docker

sudo docker run -p 9099:9099 ghcr.io/snapp-incubator/thanos-federate-proxy:main -insecure-listen-address="0.0.0.0:9099"

Binary releases

export VERSION=0.1.0
wget https://github.com/snapp-incubator/thanos-federate-proxy/releases/download/v${VERSION}/thanos-federate-proxy-${VERSION}.linux-amd64.tar.gz
tar xvzf thanos-federate-proxy-${VERSION}.linux-amd64.tar.gz thanos-federate-proxy-${VERSION}.linux-amd64/thanos-federate-proxy

From source

git clone https://github.com/snapp-incubator/thanos-federate-proxy
cd thanos-federate-proxy && go build
./thanos-federate-proxy <optional-extra-flags>

Configuration

Flags:

  -insecure-listen-address string
        The address which proxy listens on (default "127.0.0.1:9099")
  -tlsSkipVerify
        Skip TLS Verfication (default false)
  -upstream string
        The upstream thanos URL (default "http://127.0.0.1:9090")

Sample k8s deployment (as a side car with thanos or prometheus):

containers:
  ...
- name: thanos-federate-proxy
  image: ghcr.io/snapp-incubator/thanos-federate-proxy:main
  args:
  - -insecure-listen-address=0.0.0.0:9099
  - -upstream=http://127.0.0.1:9090
  ports:
  - containerPort: 9099
    name: fedproxy
    protocol: TCP

Sample prometheus config for federation:

scrape_configs:
- job_name: 'thanos-federate'
    scrape_timeout: 1m
    metrics_path: '/federate'
    params:
    'match[]':
    - 'up{namespace=~"perfix.*"}'
    static_configs:
    - targets:
        - 'thanos.svc.cluster:9099'

Limitations

The following limitations will be addressed in future releases (see Roadmap):

  • At the moment thanos-federate-proxy does not support multiple matcher queries and will only apply the first one.

  • You can not pass empty matcher to prometheus for scraping all the metrics (see this prometheus issue). A workaround is to use the following matcher:

    'match[]':
    - '{__name__=~".+"}'
    

    Note that {__name__=~".*"} won't also work and you should use ".+" instead of ".*".

Roadmap

  • test federation
  • tlsSkipVerify flag
  • Dockerfile
  • github actions
  • metrics
  • support multiple matchers
  • support empty matchers
  • support store-API for better performance
  • return error message instead of only logging it (??)
  • remove space after comma in metrics (causing no issues)

Metrics

Metric Notes
thanosfederateproxy_scrape_duration_seconds_count Total number of scrape requests with response code
thanosfederateproxy_scrape_duration_seconds_sum Duration of scrape requests with response code
thanosfederateproxy_scrape_duration_seconds_bucket Count of scrape requests per bucket (for calculating percentile)

Security

Reporting security vulnerabilities

If you find a security vulnerability or any security related issues, please DO NOT file a public issue, instead send your report privately to [email protected]. Security reports are greatly appreciated and we will publicly thank you for it.

License

Apache-2.0 License, see LICENSE.

Issues
  • chore(deps): bump github.com/prometheus/common from 0.26.0 to 0.30.0

    chore(deps): bump github.com/prometheus/common from 0.26.0 to 0.30.0

    Bumps github.com/prometheus/common from 0.26.0 to 0.30.0.

    Release notes

    Sourced from github.com/prometheus/common's releases.

    v0.30.0

    Add the ability to enable HTTP2 again via an env variable

    v0.29.0

    Add WithIdleConnTimeout HTTP client option

    v0.28.0

    Client golang dependency update.

    v0.27.0 / 2021-06-03

    • Fix marshalling of URL wrapper with nil value. #303
    • Remove deprecated log package #306
    Commits
    • 8d1c9f8 Merge pull request #286 from roidelapluie/http2-env
    • 5f27aa2 Merge pull request #316 from prometheus/mem/fix_linter_errors
    • b5c3589 Enable HTTP 2 via env variable
    • 08ac5ff Fix linter errors
    • a1b6ede Merge pull request #313 from roidelapluie/refactorsigv4
    • a44452d Refactor sigv4 in tests
    • 448ba39 Add sigv4 module (#310)
    • 8281fb2 Merge pull request #308 from austince/feat/with-idle-conn-timeout
    • 5018d4d Add WithIdleConnTimeout HTTP client option
    • 9e276a1 Merge pull request #307 from sagikazarmark/update-client-golang
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    opened by dependabot[bot] 1
Releases(v0.1.0-alpha)
Owner
Snapp Cab Incubators
Growing snapp-cab open source projects
Snapp Cab Incubators
An incredibly fast proxy checker & IP rotator with ease.

An incredibly fast proxy checker & IP rotator with ease.

Kitabisa 443 Sep 20, 2021
:alarm_clock: :fire: A TCP proxy to simulate network and system conditions for chaos and resiliency testing

Toxiproxy Toxiproxy is a framework for simulating network conditions. It's made specifically to work in testing, CI and development environments, supp

Shopify 5.8k Sep 16, 2021
A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.

frp README | 中文文档 What is frp? frp is a fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet. As of now, it s

null 48.5k Sep 22, 2021
Axiom Honeycomb Proxy ships logs to Axiom and Honeycomb simultaneously.

Axiom Honeycomb Proxy Table of Contents Introduction Usage Contributing License Introduction Axiom Honeycomb Proxy ships logs to Axiom and Honeycomb s

Axiom, Inc. 5 Sep 8, 2021
Standalone client for proxies of Opera VPN

opera-proxy Standalone Opera VPN client. Younger brother of hola-proxy. Just run it and it'll start a plain HTTP proxy server forwarding traffic throu

null 226 Sep 20, 2021
An experimental Tor-Proxy serivce written in Go using Go-proxy and Go-libtor.

tor-proxy An experimental standalone tor-proxy service built with Go, using go-proxy, go-libtor and bine. This is a simple replacement to Tor's origin

Narasimha Prasanna HN 30 Sep 8, 2021
Standalone client for proxies of Windscribe browser extension

windscribe-proxy Standalone Windscribe proxy client. Younger brother of opera-proxy. Just run it and it'll start a plain HTTP proxy server forwarding

null 19 Sep 16, 2021
Reverse cwmp proxy

cwmp-proxy Integration of the proxy will provide you the ability to place CPEs and ACS servers in different networks. What about if the devices are pl

Ivan Stefanov 14 Feb 21, 2020
Highly-opionated MTPROTO proxy for Telegram.

Highly-opionated (ex-bullshit-free) MTPROTO proxy for Telegram.

Sergey Arkhipov 1.1k Sep 25, 2021
A CLI-based HTTP intercept and replay proxy

GLORP Glorp is an HTTP intercept proxy, allowing the inspection and replaying of HTTP requests. The layout and flow was designed to function similar t

DoI 202 Sep 21, 2021
Judas is a pluggable phishing proxy.

Judas is a pluggable phishing proxy.

cances 30 Aug 14, 2021
Squzy - is a high-performance open-source monitoring, incident and alert system written in Golang with Bazel and love.

Squzy - opensource monitoring, incident and alerting system About Squzy - is a high-performance open-source monitoring and alerting system written in

Squzy 365 Sep 13, 2021
Use Consul to do service discovery, use gRPC +kafka to do message produce and consume. Use redis to store result.

目录 gRPC/consul/kafka简介 gRPC+kafka的Demo gRPC+kafka整体示意图 限流器 基于redis计数器生成唯一ID kafka生产消费 kafka生产消费示意图 本文kafka生产消费过程 基于pprof的性能分析Demo 使用pprof统计CPU/HEAP数据的

null 38 Jan 31, 2021
A secure image proxy server

go-camo Contents About How it works Differences from Camo Installing pre-built binaries Building Running Running on Heroku Securing an installation Co

null 164 Sep 16, 2021
A small TCP proxy written in Go

tcp-proxy A small TCP proxy written in Go This project was intended for debugging text-based protocols. The next version will address binary protocols

Jaime Pillora 546 Sep 23, 2021
HTTP IP Proxy Pool

HTTP IP Proxy Pool

null 9 Sep 26, 2021
Modern network boot server.

bofied demo.mp4 Modern network boot server. Overview bofied is a network boot server. It provides everything you need to PXE boot a node, from a (prox

Felix Pojtinger 77 Sep 20, 2021
Open Source HTTP Reverse Proxy Cache and Time Series Dashboard Accelerator

Trickster is an HTTP reverse proxy/cache for http applications and a dashboard query accelerator for time series databases. Learn more below, and chec

null 1.5k Sep 17, 2021
Easy proxy for redis sentinel

sentinel-proxy Easy proxy for redis sentinel. Main purpose of the proxy is easy work with redis sentinel without changing the client code. Especially

Anatoly 3 Jun 2, 2021