Simple edge server / reverse proxy

Overview

reproxy build Coverage Status Go Report Card Docker Automated build

Reproxy is simple edge HTTP(s) sever / reverse proxy supporting various providers (docker, static, file). One or more providers supply information about requested server, requested url, destination url and health check url. Distributed as a single binary or as a docker container.

Server can be set as FQDN, i.e. s.example.com or * (catch all). Requested url can be regex, for example ^/api/(.*) and destination url may have regex matched groups in, i.e. http://d.example.com:8080/$1. For the example above http://s.example.com/api/something?foo=bar will be proxied to http://d.example.com:8080/something?foo=bar.

For convenience, requests with the trailing / and without regex groups expanded to /(.*), and destinations in those cases expanded to /$1. I.e. /api/ -> http://127.0.0.1/service will be translated to ^/api/(.*) -> http://127.0.0.1/service/$1

Both HTTP and HTTPS supported. For HTTPS, static certificate can be used as well as automated ACME (Let's Encrypt) certificates. Optional assets server can be used to serve static files.

Starting reproxy requires at least one provider defined. The rest of parameters are strictly optional and have sane default.

example with a static provider:

reproxy --static.enabled --static.rule="example.com/api/(.*),https://api.example.com/$1"

Install

  • for a binary distribution pick the proper file in the release section
  • docker container available via docker hub (umputun/reproxy) as well as via github container registry (ghcr.io/umputun/reproxy). Latest stable version has :vX.Y.Z tag (with :latest alias) and the current master has :master tag.

Providers

User can sets multiple providers at the same time.

See examples of various providers in examples

Static

This is the simplest provider defining all mapping rules directly in the command line (or environment). Multiple rules supported. Each rule is 3 or 4 comma-separated elements server,sourceurl,destination,[ping-url]. For example:

  • *,^/api/(.*),https://api.example.com/$1 - proxy all request to any host/server with /api prefix to https://api.example.com
  • example.com,/foo/bar,https://api.example.com/zzz - proxy all requests to example.com and with /foo/bar url to https://api.example.com/zzz

The last (4th) element defines an optional ping url used for health reporting. I.e.*,^/api/(.*),https://api.example.com/$1,https://api.example.com/ping. See Health check section for more details.

File

reproxy --file.enabled --file.name=config.yml

example of config.yml:

default: # the same as * (catch-all) server
  - {route: "^/api/svc1/(.*)", dest: "http://127.0.0.1:8080/blah1/$1"}
  - {route: "/api/svc3/xyz", dest: "http://127.0.0.3:8080/blah3/xyz", "ping": "http://127.0.0.3:8080/ping"}
srv.example.com:
  - {route: "^/api/svc2/(.*)", dest: "http://127.0.0.2:8080/blah2/$1/abc"}

This is a dynamic provider and file change will be applied automatically.

Docker

Docker provider works with no extra configuration and by default redirects all requests like https://server/api/<container_name>/(.*) to the internal IP of the given container and the exposed port. Only active (running) containers will be detected.

This default can be changed with labels:

  • reproxy.server - server (hostname) to match
  • reproxy.route - source route (location)
  • reproxy.dest - destination path. Note: this is not full url, but just the path which will be appended to container's ip:port
  • reproxy.ping - ping path for the destination container.

By default all containers with exposed port will be considered as routing destinations. There are 2 ways to restrict it:

  • Exclude some containers explicitly with --docker.exclude, i.e. --docker.exclude=c1 --docker.exclude=c2 ...
  • Allow only a particular docker network with --docker.network

This is a dynamic provider and any change in container's status will be applied automatically.

SSL support

SSL mode (by default none) can be set to auto (ACME/LE certificates), static (existing certificate) or none. If auto turned on SSL certificate will be issued automatically for all discovered server names. User can override it by setting --ssl.fqdn value(s)

Logging

By default no request log generated. This can be turned on by setting --logger.enabled. The log (auto-rotated) has Apache Combined Log Format

Assets Server

User may turn assets server on (off by default) to serve static files. As long as --assets.location set it will treat every non-proxied request under assets.root as a request for static files.

Assets server can be used without any proxy providers. In this mode reproxy acts as a simple web server for a static context.

More options

  • --gzip enables gizp compression for responses.
  • --max=N allows to set the maximum size of request (default 64k)
  • --header sets extra header(s) added to each proxied request

Ping and health checks

reproxy provides 2 endpoints for this purpose:

  • /ping responds with pong and indicates what reproxy up and running
  • /health returns 200 OK status if all destination servers responded to their ping request with 200 or 417 Expectation Failed if any of servers responded with non-200 code. It also returns json body with details about passed/failed services.

All Application Options

  -l, --listen=                     listen on host:port (default: 127.0.0.1:8080) [$LISTEN]
  -t, --timeout=                    proxy timeout (default: 5s) [$TIMEOUT]
  -m, --max=                        max response size (default: 64000) [$MAX_SIZE]
  -g, --gzip                        enable gz compression [$GZIP]
  -x, --header=                     proxy headers [$HEADER]
      --no-signature                disable reproxy signature headers [$NO_SIGNATURE]
      --dbg                         debug mode [$DEBUG]

ssl:
      --ssl.type=[none|static|auto] ssl (auto) support (default: none) [$SSL_TYPE]
      --ssl.cert=                   path to cert.pem file [$SSL_CERT]
      --ssl.key=                    path to key.pem file [$SSL_KEY]
      --ssl.acme-location=          dir where certificates will be stored by autocert manager (default: ./var/acme) [$SSL_ACME_LOCATION]
      --ssl.acme-email=             admin email for certificate notifications [$SSL_ACME_EMAIL]
      --ssl.http-port=              http port for redirect to https and acme challenge test (default: 80) [$SSL_HTTP_PORT]
      --ssl.fqdn=                   FQDN(s) for ACME certificates [$SSL_ACME_FQDN]

assets:
  -a, --assets.location=            assets location [$ASSETS_LOCATION]
      --assets.root=                assets web root (default: /) [$ASSETS_ROOT]

logger:
      --logger.enabled              enable access and error rotated logs [$LOGGER_ENABLED]
      --logger.file=                location of access log (default: access.log) [$LOGGER_FILE]
      --logger.max-size=            maximum size in megabytes before it gets rotated (default: 100) [$LOGGER_MAX_SIZE]
      --logger.max-backups=         maximum number of old log files to retain (default: 10) [$LOGGER_MAX_BACKUPS]

docker:
      --docker.enabled              enable docker provider [$DOCKER_ENABLED]
      --docker.host=                docker host (default: unix:///var/run/docker.sock) [$DOCKER_HOST]
      --docker.network=             docker network (default: bridge) [$DOCKER_NETWORK]
      --docker.exclude=             excluded containers [$DOCKER_EXCLUDE]

file:
      --file.enabled                enable file provider [$FILE_ENABLED]
      --file.name=                  file name (default: reproxy.yml) [$FILE_NAME]
      --file.interval=              file check interval (default: 3s) [$FILE_INTERVAL]
      --file.delay=                 file event delay (default: 500ms) [$FILE_DELAY]

static:
      --static.enabled              enable static provider [$STATIC_ENABLED]
      --static.rule=                routing rules [$STATIC_RULES]

Help Options:
  -h, --help                        Show this help message
  

Status

The project is under active development and may have breaking changes till v1 released.

Comments
  • Performance with file uploads

    Performance with file uploads

    a simple 12Mb file takes 1.6 sec to upload with reproxy

    With reproxy:

    time curl -X POST "http://localhost/api/upload" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F "[email protected]"
    
    0.01s user 0.01s system 1% cpu 1.614 total
    

    without :

    time curl -X POST "http://localhost:8000/api/upload" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F "[email protected]"
    
    0.00s user 0.01s system 29% cpu 0.062 total
    

    conf:

      reproxy:
        image: umputun/reproxy:master
        container_name: reproxy
        hostname: reproxy
        ports:
          - "80:8080"
        volumes:
          - /var/run/docker.sock:/var/run/docker.sock:ro
          - ./files/static:/static
        environment:
          - LISTEN=0.0.0.0:8080
          - DOCKER_ENABLED=true
          - DEBUG=true
          - MAX_SIZE=20000000
    
    opened by vitalik 18
  • [question] is there a way to serve static files from multiple locatons ?

    [question] is there a way to serve static files from multiple locatons ?

    Is it possible to make some mapping like this:

    - site1.com/api/    ---> docker.container1
    - site1.com/static/ ---> static path /var/www/staticfolder1
    - site2.com/api/    ---> docker.container2
    - site2.com/static/ ---> /var/www/anotherfolder
    
    opened by vitalik 15
  • File event tests may occasionally fail

    File event tests may occasionally fail

    It looks like certain tests that rely on short timings may occasionally fail. This is coming from downstream NixOS CI, and in this case on macOS: https://hydra.nixos.org/build/143298376/log

    But I was unable to reproduce this locally on macOS, either in a normal build or in a build using Nix. I'm guessing it's a matter of tight tolerances and busy CI machine, maybe?

    Do you think it's safe to ignore these failures?

    opened by stephank 9
  • Unable to specify header with ',' in value

    Unable to specify header with ',' in value

    I want to add

    Access-Control-Allow-Headers:DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type

    but in response I got

    Access-Control-Allow-Headers:DNT

    if I specify

    Access-Control-Allow-Headers:"DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type"

    I got Access-Control-Allow-Headers:"DNT

    for Access-Control-Allow-Headers:'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type'

    I got Access-Control-Allow-Headers:'DNT

    opened by rychkov 8
  • Add a simple throttler/limiter

    Add a simple throttler/limiter

    This can be useful, and won't add much complexity. The throttler should be able to limit the total number of concurrent requests per reproxy server as well as per proxy destination. Not sure if limiting per destination server (virtual host) needed.

    It also should be able to limit req/sec per remote client for the same areas (total and destination).

    enhancement 
    opened by umputun 8
  • Support multiple route pairs in docker labels

    Support multiple route pairs in docker labels

    re: https://github.com/umputun/reproxy/discussions/76

    In some cases, one container may expose multiple endpoints, for example, public API and some admin API. They can be on different ports and the goal is to allow such functionality to be defined with docker provider (labels?)

    I can think of two ways to implement it:

    • add numeric suffix to reproxy.*.N, i.e. reproxy.server.1=example.com, reproxy.dest.1=blah and so on. This may increase the number of labels and could be ugly in extreme cases
    • similar to the first approach but combine the rule in this style of static provider, i.e. reproxy.1=m.example.com,/api/(.*)./someplace/$1,/ping. This is harder to read but will minimize the number of labels

    both approaches can be combined too, and the user can choose which one he likes better. The current labels should be supported too as we don't want to break compatibility for no good reason

    if anyone has better ideas pls share

    enhancement 
    opened by umputun 7
  • Replace fsouza/go-dockerclient with direct docker api calls

    Replace fsouza/go-dockerclient with direct docker api calls

    Just an idea - trying to minimize the number of third-party dependencies. The go-dockerclient seems to be a heavy one and brings a lot of transitive dependencies.

    All we do with docker is just event listener and container's list. It would be nice to replace go-dockerclient with internal calls to docker api

    help wanted 
    opened by umputun 7
  • some services in docker examples do not work on ARM platform

    some services in docker examples do not work on ARM platform

    When used on ARMv7 (odroid c1) all services works with exception of svc1 and svc2 that are based on amd64's hashicorp/http-echo. Probably it will be better to use a multi-platform http-echo image for svc1/svc2.

    Example of the problem log:

    # docker logs svc1
    standard_init_linux.go:219: exec user process caused: exec format error
    

    I researched for some other images and found teapow/http-echo which worked for me.

    Example of docker-compose.yml insert:

      echo1:
        image: teapow/http-echo
        container_name: echo1
        hostname: echo1
        ports:
          - "9090:9090"
        environment:
          - SERVER_PORT=9090
          - ECHO_MESSAGE="teapow pong"
          - SERVER_DEBUG=true
    

    test:

    # curl http://localhost/api/echo1/
    {
      "message": "\"teapow pong\""
    }
    
    opened by tigerrrr 5
  • Simplify usage (based on the real-life setup)

    Simplify usage (based on the real-life setup)

    I have tried to migrate one of my non-trivial configurations hosting multiple docker services with nginx-le proxy in front of this and with several static sites to reproxy. The good news - I was able to do most of it and the final configuration seems to be easier. No need for multiple configuration files (i.e. docker-compose + nginx configs) and generally the final compose readable and make sense.

    However, I've run into some unexpected complexity and most of it was self-induced and hopefully can be addressed with relatively easy changes:

    1. Mix of static rules and docker provider doesn't seem to work together. Not sure why, need to investigate
    2. All of my destination containers needed reproxy.route: /(.*) and reproxy.dest: /@1 labels and this pair seems to be a good candidate for a new default
    3. In order to allow the current label-less docker discovery I'm going to add a new bool parameter turning it on or off. Not sure which makes more sense as a default.
    4. Some containers needed to handle multiple servers, i.e. blah.com and www.blah.com, but currently, only a single server name supported. Should be doable to support multiple servers
    5. Attempt to serve fully static content with separate reproxy containers (see #12) worked, but looks a little bit odd. I'm not sure if I like it or maybe prefer some other approach
    6. I had to set LISTEN=0.0.0.0:port to each reproxy container because the default is localhost:port. From safety/security POV it makes sense, but practically every reproxy container will have to have LISTEN define explicitly. Maybe this is the good thing, not sure
    7. Additional reproxy servers (for static content) needed reproxy.enabled: y because the container itself sets it to false (see #16). This is logical (technically speaking) but unexpected and confusing in practice.
    opened by umputun 5
  • Support regex in host / server

    Support regex in host / server

    Main consideration is backward compatibility. example.com should be treated as an exact match, where possible. So current order is: exact host, regex host, * or ""

    opened by alek-sys 4
  • Cache and Expires by Media Type

    Cache and Expires by Media Type

    ASSETS_CACHE now sets only a Cache-control header for all static assets — images, CSS, scripts, and HTML. It would be useful at least to have an option to exclude HTML from the cache.

    More obvious way - to add Expires header and an option to set it by MediaType.

    I see index HTML pages served from the browser cache in the current configuration even after Hugo's rebuild - because max-age from Cache-control yet to be expired.

    opened by grayodesa 4
  • page display error in readme.md

    page display error in readme.md

    There's a display error in your readme.md file: image

    The first two examples (static provider; automatic docker discovery) are truncated. May want to correct them. :-)

    Keep up the good work!

    opened by rzv-trsc 0
  • initial support for builtin plugins

    initial support for builtin plugins

    1. Clone plugin repo to folder plugins with folder name as plugin package name git clone https://github.com/negasus/reproxy-brotli-plugin.git plugins/brotli

    2. Remove go module files rm plugins/brotli/go.*

    3. Generate app/proxy/plugins.go go run ./cmd/plugins

    4. Vendoring go mod tidy go mod vendor

    5. Done Run reproxy for test

    opened by negasus 0
  • Feature request: per route timeout and throttle

    Feature request: per route timeout and throttle

    It would be great if I can set (in config file) timeout and throttle for specific routes. For example I use exponential timeouts for login form to protect from brute force and I'd like to increase timeouts for this endpoint.

    opened by alehano 2
  • Listen to IPv6 address

    Listen to IPv6 address

    Hey!

    Tried to set reproxy to listen on [::]:8080 (and :::8080 too) address but it doesn't want to start with some errors logged:

    reproxy  | 2022/03/31 21:17:03.924 [INFO]  activate http proxy server on "[::]:8080"
    reproxy  | 2022/03/31 21:17:03.925 [ERROR] proxy server failed, listen tcp: address "[::]:8080": too many colons in address
    reproxy  | 2022/03/31 21:17:03.925 [ERROR] proxy server failed, listen tcp: address "[::]:8080": too many colons in address
    

    Is IPv6 supported in reproxy? Could we implement if it isn't? Thanks!

    opened by ffix 0
  • An ability to bypass Host header

    An ability to bypass Host header

    Hey!

    Tried to use reproxy for the first time and got some issues with services like Grafana. It looks like Grafana server doesn't handle X-Forwarded-Host header which is non-standard. But there is no ability in reproxy to bypass Host header like you can do with nginx:

    proxy_set_header Host $host;
    

    It may be implemented using command line flag or we can enable it by default. What do you think about it?

    Thanks!

    opened by ffix 7
Releases(v0.11.0)
Tcp-proxy - A dead simple reverse proxy server.

tcp-proxy A proxy that forwords from a host to another. Building go build -ldflags="-X 'main.Version=$(git describe --tags $(git rev-list --tags --max

Injamul Mohammad Mollah 0 Jan 2, 2022
mt-multiserver-proxy is a reverse proxy designed for linking multiple Minetest servers together

mt-multiserver-proxy mt-multiserver-proxy is a reverse proxy designed for linking multiple Minetest servers together. It is the successor to multiserv

null 13 Aug 29, 2022
4chain is a simple、fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet.

4Chain What is 4chain? 4chain is a simple、fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet. Using the ssh

null 8 Sep 7, 2022
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io

中文 MOSN is a network proxy written in Golang. It can be used as a cloud-native network data plane, providing services with the following proxy functio

MOSN 3.9k Sep 22, 2022
A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.

frp README | 中文文档 What is frp? frp is a fast reverse proxy to help you expose a local server behind a NAT or firewall to the Internet. As of now, it s

null 60.2k Sep 21, 2022
Reverse proxy server to filter traffic based on JA3 fingerprint/hash

JA3RP (JA3 Reverse Proxy) Ja3RP is a basic reverse proxy server that filters traffic based on JA3 fingerprints. It can also operate as a regular HTTP

Sleeyax 16 Sep 17, 2022
A standalone Web Server developed with the standard http library, suport reverse proxy & flexible configuration

paddy 简介 paddy是一款单进程的独立运行的web server,基于golang的标准库net/http实现。 paddy提供以下功能: 直接配置http响应 目录文件服务器 proxy_pass代理 http反向代理 支持请求和响应插件 部署 编译 $ go build ./main/p

fangyousong 5 May 2, 2022
A paywall bypassing reverse proxy and DNS server written in go 🔨💵🧱

FreeNews ?? ?? ?? A paywall bypassing reverse proxy and DNS server written in go. This project is still hard work in progress. Expect stuff to just no

fipso 10 Jun 21, 2022
Dead simple reverse proxy for all your containerized needss

Whats this ? Pawxi is yet another reverse proxy designed with simplicity in mind. Born out of a certain users frustration at the complexity of setting

null 14 May 26, 2022
Simple Reverse Proxy Load Balancer

lb - a reverse proxy load-balancing server, It implements the Weighted Round Robin Balancing algorithm

Blessing Pariola 3 Mar 23, 2022
Simple SNI based reverse proxy

SNIProxy SNIProxy is a very simple reverse proxy server that uses TLS SNI to route to hosts (in HTTP mode it just uses the Host header). It is designe

Maartje Eyskens 5 Sep 14, 2022
Simple Minecraft Bedrock reverse proxy

BedProx [WIP] Simple Minecraft Bedrock reverse proxy Features Reverse Proxy HAProxy Protocol Support (NOT TESTED) Webhooks REST API How to use/deploy

Hendrik Schlehlein 7 Sep 7, 2022
A simple small reverse proxy for golang

ssrp simple small reverse proxy bash <(curl -Ls https://raw.githubusercontent.com/go-bai/ssrp/master/install.sh) cn install bash <(curl -Ls https://gh

gobai 4 Apr 5, 2022
Gouter: Simple development reverse proxy

Gouter: Simple development reverse proxy

null 0 Jan 15, 2022
Reverse cwmp proxy

cwmp-proxy Integration of the proxy will provide you the ability to place CPEs and ACS servers in different networks. What about if the devices are pl

Ivan Stefanov 15 Aug 9, 2022
Open Source HTTP Reverse Proxy Cache and Time Series Dashboard Accelerator

Trickster is an HTTP reverse proxy/cache for http applications and a dashboard query accelerator for time series databases. Learn more below, and chec

null 1.8k Sep 27, 2022
An Advanced HTTP Reverse Proxy with Dynamic Sharding Strategies

Weaver - A modern HTTP Proxy with Advanced features Description Features Installation Architecture Configuration Contributing License Description Weav

Gojek 561 Sep 10, 2022
An Advanced HTTP Reverse Proxy with Dynamic Sharding Strategies

Weaver - A modern HTTP Proxy with Advanced features Description Features Installation Architecture Configuration Contributing License Description Weav

Gojek 561 Sep 10, 2022
gobetween - modern & minimalistic load balancer and reverse-proxy for the ☁️ Cloud era.

gobetween - modern & minimalistic load balancer and reverse-proxy for the ☁️ Cloud era. Current status: Maintenance mode, accepting PRs. Currently in

Yaroslav Pogrebnyak 1.8k Sep 24, 2022