support clickhouse

Last update: Dec 7, 2022

Related tags

Overview

Remote storage adapter

This is a write adapter that receives samples via Prometheus's remote write protocol and stores them in Graphite, InfluxDB, clickhouse or OpenTSDB. It is meant as a replacement for the built-in specific remote storage implementations that have been removed from Prometheus.

For InfluxDB, this binary is also a read adapter that supports reading back data through Prometheus via Prometheus's remote read protocol.

Building

go build

Running

Graphite example:

./remote_storage_adapter --graphite-address=localhost:8080

OpenTSDB example:

./remote_storage_adapter --opentsdb-url=http://localhost:8081/

InfluxDB example:

./remote_storage_adapter --influxdb-url=http://localhost:8086/ --influxdb.database=prometheus --influxdb.retention-policy=autogen

Clickhouse example:

./remote_storage_adapter --clickhouse.url=localhost:9000

sql for clickhouse

# note: replace {shard} and {replica} and run on each server

CREATE DATABASE prometheus ON CLUSTER you_cluster

DROP TABLE IF EXISTS prometheus.metrics;
CREATE TABLE IF NOT EXISTS prometheus.metrics ON CLUSTER you_cluster
(
     date Date DEFAULT toDate(0),
     name String,
     tags Array(String),
     val Float64,
     ts DateTime,
     updated DateTime DEFAULT now()
)
ENGINE = ReplicatedGraphiteMergeTree(
     '/clickhouse/tables/{shard}/prometheus.metrics',
     '{replica}', date, (name, tags, ts), 8192, 'graphite_rollup'
)

xml example for clickhouse server config

 <graphite_rollup>
        <path_column_name>tags</path_column_name>
        <time_column_name>ts</time_column_name>
        <value_column_name>val</value_column_name>
        <version_column_name>updated</version_column_name>
        <default>
                <function>avg</function>
                <retention>
                        <age>0</age>
                        <precision>10</precision>
                </retention>
                <retention>
                        <age>86400</age>
                        <precision>30</precision>
                </retention>
                <retention>
                        <age>172800</age>
                        <precision>300</precision>
                </retention>
        </default>
  </graphite_rollup>

To show all flags:

./remote_storage_adapter -h

Configuring Prometheus

To configure Prometheus to send samples to this binary, add the following to your prometheus.yml:

# Remote write configuration (for Graphite, OpenTSDB, or InfluxDB).
remote_write:
  - url: "http://localhost:9201/write"

# Remote read configuration (for InfluxDB only at the moment).
remote_read:
  - url: "http://localhost:9201/read"

Worker failover support for PostgreSQL Citus extension using pg_auto_failover.

citus-failover Worker failover support for citus community version using pg_auto_failover. What is this? This is a simple service to monitor changes i

23 Dec 7, 2022

a lightweight distributed transaction management service, support xa tcc saga

8.1k Dec 29, 2022

MySQL Storage engine conversion，Support mutual conversion between MyISAM and InnoDB engines.

econvert MySQL Storage engine conversion 简介此工具用于MySQL存储引擎转换，支持CTAS和ALTER两种模式，目前只支持MyISAM和InnoDB存储引擎相互转换，其它引擎尚不支持。注意：当对表进行引擎转换时，建议业务停止访问或者极少量访问时进行。原

5 Oct 25, 2021

A go Library for scan database/sql rows to struct、slice、other types. And it support multiple databases connection management

ploto A go Library for scan database/sql rows to struct、slice、other types. And it support multiple databases connection management It's not an ORM. wo

3 Nov 3, 2022

Support MySQL or MariaDB for gopsql/psql and gopsql/db

mysql Support MySQL or MariaDB for github.com/gopsql/psql. You can make MySQL SELECT, INSERT, UPDATE, DELETE statements with this package. NOTE: Pleas

0 Dec 9, 2021

A go package to add support for data at rest encryption if you are using the database/sql.

go-lockset A go package to add support for data at rest encryption if you are using the database/sql to access your database. Installation In your Gol

0 Jan 30, 2022

Go-clickhouse - ClickHouse client for Go

ClickHouse client for Go 1.18+ This client uses native protocol to communicate w

161 Jan 9, 2023

support clickhouse

Remote storage adapter This is a write adapter that receives samples via Prometheus's remote write protocol and stores them in Graphite, InfluxDB, cli

30 Dec 7, 2022

Clickhouse support for GORM

clickhouse Clickhouse support for GORM Quick Start package main import ( "fmt" "github.com/sweetpotato0/clickhouse" "gorm.io/gorm" ) // User

1 Oct 18, 2022

ClickHouse http proxy and load balancer

chproxy English | 简体中文 Chproxy, is an http proxy and load balancer for ClickHouse database. It provides the following features: May proxy requests to

1k Jan 3, 2023

Collects many small inserts to ClickHouse and send in big inserts

ClickHouse-Bulk Simple Yandex ClickHouse insert collector. It collect requests and send to ClickHouse servers. Installation Download binary for you pl

397 Dec 28, 2022

GoRose(go orm), a mini database ORM for golang, which inspired by the famous php framwork laravle's eloquent. It will be friendly for php developer and python or ruby developer. Currently provides six major database drivers: mysql,sqlite3,postgres,oracle,mssql, Clickhouse.

GoRose ORM _______ ______ .______ ______ _______. _______ / _____| / __ \ | _ \ / __ \ / || ____| |

1.1k Dec 24, 2022

Bifrost ---- 面向生产环境的 MySQL 同步到Redis,MongoDB,ClickHouse,MySQL等服务的异构中间件

Comments

Replaced bytes.Buffer with strings.Builder

strings.Builder is more efficient in scopes of time and memory, see https://medium.com/@felipedutratine/string-concatenation-in-golang-since-1-10-bytes-buffer-vs-strings-builder-2b3081848c45 for more info.

Signed-off-by: Yury Frolov [email protected]

opened by EinKrebs 0
Small SQL fixes in README & github highlighting
made comment syntactically correct;

replaced you_cluster cluster name with '{cluster}', which is way more universal.

Signed-off-by: Yury Frolov [email protected]
opened by EinKrebs 0