Turn Nginx logs into Prometheus metrics

Overview

Nginx log parser and Prometheus exporter

Go Report Card Coverage Status Build Status Mentioned in Awesome Go

This service parses incoming syslog messages from Nginx sent over UDP and converts them into Prometheus metrics exported through the built-in HTTP server.

  • If the prefix-list is specified then per-URI statistics are generated. Using this feature without URI-prefix list is dangerous because it leads to an unbounded memory usage. It's therefore enabled only with a limited prefix list.

    The URI prefix list is a plain-text file with a single prefix per line, e.g.

    /api/call1.json
    /api/call2.json
    /api/call3.json

    URIs received from nginx are stripped of any query parameters - only the part before '?' is used.

  • It optionally supports country lookup for client IPs using MaxMind GeoIP database.

Nginx configuration snippet

log_format collector '$remote_addr|$scheme|$host|$request_method|$server_protocol|$request_uri|$status|$request_time|$request_length|$bytes_sent';
access_log syslog:server=1.1.1.1:1514,tag=nginx collector;

Getting it

  • Get RPM & DEB directly from releases, only linux-amd64 build currently available.

  • Build youself:

go get github.com/blind-oracle/nginx-prometheus
cd $GOPATH/src/github.com/blind-oracle/nginx-prometheus
go build

Usage example

If you're using RPM/DEB packages then options are set in /etc/default/nginx-prometheus

./nginx-prometheus \
-debug false \
-listenSyslog 0.0.0.0:1514 \
-listenHTTP 0.0.0.0:1514 \
-geoipCountryDB /etc/nginx-prometheus/country.mmdb \
-uriPrefixFile /etc/nginx-prometheus/uriPrefixes.txt
  • debug - prints every syslog message received if true (defaults to false)
  • listenSyslog - ip:port on which to listen for UDP Syslog messages (defaults to 0.0.0.0:1514)
  • listenHTTP - ip:port on which to listen for incoming HTTP requests from Prometheus (defaults to 0.0.0.0:11080)
  • geoipCountryDB - path to MaxMind country GeoIP database (optional)
  • uriPrefixFile - path to a URI prefix list (optional)

Grafana dashboard

Add Prometheus source to Grafana and import grafana-dashboard.json for most common graphs.

Releases(v1.1.0)
Owner
Igor Novgorodov
Software/System Engineer
Igor Novgorodov
Gin Server Demo, with Features: Swagger UI, Prometheus, Shell scripts

Gin Server Gin Server Features Swagger UI Docker Prometheus Shell generator Production Cypher Service And we can get the authorization key SECURITY_CA

song 0 Nov 1, 2021
nginx exporter for Prometheus copy code from ingress-nginx use for monit nginx

nginx_exporter 介绍 从ingress-nginx官方代码中的expoter迁移出来 用来监控虚拟机上的nginx的expoter 基于官方 controller-v0.49.3 版本移植的代码 软件架构 nginx_socket 通过lua模块monitor.lua 将nginx l

null 0 Dec 8, 2021
nginx exporter for Prometheus copy code from ingress-nginx use for monit nginx

nginx_exporter 介绍 从ingress-nginx官方代码中的expoter迁移出来 用来监控虚拟机上的nginx的expoter 基于官方 controller-v0.49.3 版本移植的代码 软件架构 nginx_socket 通过lua模块monitor.lua 将nginx l

嘉联支付有限公司 3 Dec 20, 2021
Json-log-exporter - A Nginx log parser exporter for prometheus metrics

json-log-exporter A Nginx log parser exporter for prometheus metrics. Installati

horan 0 Jan 5, 2022
Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer for Nginx.

Nginx-Log-Analyzer is a lightweight (simplistic) log analyzer, used to analyze Nginx access logs for myself.

Mao Mao 21 May 28, 2022
An implementation of an Ingress controller for NGINX and NGINX Plus

?? Help make the NGINX Ingress Controller better by participating in our survey!

null 0 Dec 17, 2021
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.

The open-source platform for monitoring and observability. Grafana allows you to query, visualize, alert on and understand your metrics no matter wher

Grafana Labs 49.7k Jun 27, 2022
Flowlogs2metrics - Transform flow logs into metrics

Overview Flow-Logs to Metrics (a.k.a. FL2M) is an observability tool that consum

null 15 Jun 12, 2022
Sensu-go-postgres-metrics - The sensu-go-postgres-metrics is a sensu check that collects PostgreSQL metrics

sensu-go-postgres-metrics Table of Contents Overview Known issues Usage examples

Scott Cupit 0 Jan 12, 2022
The metrics-agent collects allocation metrics from a Kubernetes cluster system and sends the metrics to cloudability

metrics-agent The metrics-agent collects allocation metrics from a Kubernetes cluster system and sends the metrics to cloudability to help you gain vi

null 0 Jan 14, 2022
Prometheus metrics for Go database/sql via VictoriaMetrics/metrics

sqlmetrics Prometheus metrics for Go database/sql via VictoriaMetrics/metrics Features Simple API. Easy to integrate. Install Go version 1.16+ go get

cristaltech 21 Jan 22, 2022
Export Prometheus metrics from journald events using Prometheus Go client library

journald parser and Prometheus exporter Export Prometheus metrics from journald events using Prometheus Go client library. For demonstration purposes,

Mike Sgarbossa 0 Jan 3, 2022
Mogo: a lightweight browser-based logs analytics and logs search platform for some datasource(ClickHouse, MySQL, etc.)

mogo Mogo is a lightweight browser-based logs analytics and logs search platform

Shimo HQ 670 Jun 22, 2022
Prometheus nginx socket exporter

stand-alone version of k8s/ingress-nginx' prometheus exporter based on openresty/lua-nginx-module how it works exporter opens socket (/tmp/prometheus-

Alex 2 Apr 4, 2022
Exporter your cypress.io dashboard into prometheus Metrics

Cypress.io dashboard Prometheus exporter Prometheus exporter for a project from Cypress.io dashboards, giving the ability to alert, make special opera

Romain Guilmont 4 Feb 8, 2022
Prometheus Common Data Exporter can parse JSON, XML, yaml or other format data from various sources (such as HTTP response message, local file, TCP response message and UDP response message) into Prometheus metric data.

Prometheus Common Data Exporter Prometheus Common Data Exporter 用于将多种来源(如http响应报文、本地文件、TCP响应报文、UDP响应报文)的Json、xml、yaml或其它格式的数据,解析为Prometheus metric数据。

null 7 May 18, 2022
Turn any program that uses STDIN/STDOUT into a WebSocket server. Like inetd, but for WebSockets.

websocketd websocketd is a small command-line tool that will wrap an existing command-line interface program, and allow it to be accessed via a WebSoc

Joe Walnes 16.3k Jun 27, 2022
turn a directory into a GUI, slash example of VNC-based GUI

dirgui @rsnous on Jan 11, 2021: "idea: filesystem<->GUI adapter, where a directory turns into a form, executable files inside that directory turn into

Tom Lieber 32 May 3, 2022
yml2dot - Turn YAML into beautiful Graph

yml2dot - Turn YAML into beautiful Graph Use Cases Visualize your YAML files as Graph Generate additional info from your source code

Luca Sepe 97 May 10, 2022
Turn shell commands into web services

webify Turn functions and commands into web services For a real world example, see turning a Python function into a web service. Overview webify is a

Michael Alexander 911 Jun 19, 2022
Turn asterisk-indented text lines into mind maps

Crumbs Turn asterisk-indented text lines into mind maps. Organize your notes in a hierarchical tree structure, using a simple text editor. an asterisk

Luca Sepe 94 Mar 19, 2022
A CLI to turn Go's benchmark output into pictures

benchdraw benchdraw allows you to make easy to read picture plots from data in Go's benchmark format, implemented in pure Go. Benchdraw does not try t

Jack Lindamood 342 Apr 28, 2022
GoPrisma - A Go wrapper for prisma to turn databases into GraphQL APIs using Go.

GoPrisma - a Go wrapper for the Prisma Engines What's this? Introspect a database and use it as a GraphQL API using Go. Supported Databases: SQLite Po

Jens Neuse 64 Jun 7, 2022
webify - Turn functions and commands into web services

webify is a very basic CGI server which forwards all requests to a single script. A design goal is to be as zero-config as possible.

Michael Alexander 916 Jun 29, 2022
Turn any key/value index into a high-performance two-dimensional spatial index

modular-spatial-index For the demo that this animated gif was generated from

SequentialRead.com 5 Mar 6, 2022
Turn a Spotify playlist into a YouTube playlist

spotlist Turn a Spotify playlist into a YouTube playlist How to use: Download latest version from Releases. (Quick Links: Mac/Windows) Run it Enter Sp

Hunter 4 Jul 18, 2021
A plugin that turn hashicorp vault into blockchain wallet.

dq-vault - Hashicorp vault BTC/ETH plugin This vault plugin stores a user's mnemonic inside vault in an encrypted manner. The plugin uses this stored

Deqode 7 Jun 1, 2022
Turn a heterogeneous pile of text docs into a single web page with good search.

Codex Codex turns an unstructured pile of heterogeneous documents into a single interactive web document. Your input documents maybe in markdown, TeX,

Amir Kadivar 4 Jan 2, 2022
Turn a grid of frames (stored in a single image) into a video file

reel2vid Convert a video that is encoded into one large image as many images sid

Alex Nichol 4 Dec 19, 2021