HTTP cache server, such as varnish

Overview

pike

Build Status

与varnish类似的HTTP缓存服务器,主要的特性如下:

  • 提供WEB的管理配置界面,简单易上手
  • 支持br与gzip两种压缩方式,根据客户端自动选择。对于可缓存与不可缓存请求使用不同的压缩配置,更佳的时间与空间的平衡
  • 仅基于Cache-Control生成缓存有效期,接口缓存完全由接口开发者决定,准确而高效(开发比运维更清楚接口是否可缓存,可缓存时长)
  • 配置支持文件与etcd两种形式存储,无中断的配置实时更新
  • 支持H2C的转发,提升与后端服务的调用性能(如果是内网转发,不需要启用)
  • 与upstream的调用支持gzipbrotlilz4snappy以及zstd压缩,可根据与upstream的网络线路选择合适的压缩方式
  • 支持upstream检测失败时回调告警,可及时获取异常upstream信息
  • 支持自定义日志,可配置按日期与大小分割日志并压缩
  • LUR与持久化存储(可选)配合使用,可根据内存使用选择更小的LRU缓存并增加持久化存储的方式

启动方式

启动参数主要如下:

  • config 配置保存地址,可以指定为etcd或者本地文件,如:etcd://user:[email protected]:2379/pike,本地文件:/opt/pike/config.yml
  • admin 配置管理后台的访问地址,如:--admin=:9013
  • log 日志文件目录,支持单文件与lumberjack形式,如/var/pike.loglumberjack:///tmp/pike.log?maxSize=100&maxAge=1&compress=true,lumberjack会根据文件内容大小与时间将文件分割

使用文件保存配置

# linux etcd,管理后台使用9013端口访问
./pike --config=etcd://127.0.0.1:2379/pike --admin=:9013
# linux file,配置文件保存在/opt/pike.yml,管理后台使用9013端口访问
./pike --config=/opt/pike.yml --admin=:9013

# docker
docker run -it --rm \
    -p 9013:9013 \
    vicanso/pike:4.0.0-alpha --config=etcd://172.16.183.177:2379/pike --admin=:9013

TODO

  • 缓存查询(如果缓存量较大,有可能导致查询性能较差,暂时未支持)
Comments
  • NFR : group cashing

    NFR : group cashing

    First of all want to admire your project. Its the best and most complex opensource cashing software out there. Keep the good work!

    Second i like to propose several new new feature requests and if possible to discus them (from what i see form the code you have impressive knowledge about http protocol specially in the optimization aspect)

    First proposal is about several instances that use a distributed cache: Ref : https://github.com/golang/groupcache

    Second is related to abstraction of the upstreams. To be able to handle GCS or S3 buckets as upstream

    Third one is for UI : options to display some statistics like miss/hit ratio and ability to purge cache entries.

    And documentation is a bit foggy Took me about a 3 days to run the app with my scenario but at the end results are impressive !

    good first issue feat 
    opened by Menta2L 5
  • 打不开管理主页

    打不开管理主页

    版本 v4.1.0 windows x64

    启动命令 : .\pike.exe --admin=:9013

    访问 127.0.0.1:9013 出现错误:

    {"statusCode":500,"category":"elton-error","message":"open web\index.html: file does not exist","exception":true}

    opened by liaohongxing 4
  • Potential import collision: import path should be

    Potential import collision: import path should be "gopkg.in/yaml.v2", not "github.com/go-yaml/yaml".

    Background

    The go-yaml/yaml has already renamed it’s import path from "github.com/go-yaml/yaml" to "gopkg.in/yaml.v2". As README of go-yaml/yaml v2.1.0 said, downstream repos should use "gopkg.in/yaml.v2" to get or import go-yaml/yaml.

    Example
    Some more examples can be found in the "examples" folder.
    
    package main
    import (
            "fmt"
            "log"
            "gopkg.in/yaml.v2"
    )
    …
    

    But vicanso/pike still used the old path: https://github.com/vicanso/pike/blob/master/go.mod#L10

    github.com/go-yaml/yaml v2.1.0+incompatible 
    

    When you use the old path "github.com/go-yaml/yaml" to import the go-yaml/yaml, it will be very easy to reintroduce go-yaml/yaml through the import statements "import gopkg.in/yaml.v2" in the go source file of go-yaml/yaml. https://github.com/go-yaml/yaml/blob/v2.1.0/decode_test.go#L12

    package yaml_test
    import (
    	…
    	. "gopkg.in/check.v1"
    	"gopkg.in/yaml.v2"
    )
    

    The "gopkg.in/yaml.v2" and "github.com/go-yaml/yaml" are the same repos. This will work in isolation, bring about potential risks and problems.

    Solution

    Replace all the old import paths, change "github.com/go-yaml/yaml" to "gopkg.in/yaml.v2". Where did you import it: https://github.com/vicanso/pike/search?q=github.com%2Fgo-yaml%2Fyaml&unscoped_q=github.com%2Fgo-yaml%2Fyaml

    opened by KateGo520 2
  • chore(deps): bump elliptic from 6.5.2 to 6.5.3 in /web

    chore(deps): bump elliptic from 6.5.2 to 6.5.3 in /web

    Bumps elliptic from 6.5.2 to 6.5.3.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • chore(deps): bump handlebars from 4.1.2 to 4.5.3 in /web

    chore(deps): bump handlebars from 4.1.2 to 4.5.3 in /web

    Bumps handlebars from 4.1.2 to 4.5.3.

    Changelog

    Sourced from handlebars's changelog.

    v4.5.3 - November 18th, 2019

    Bugfixes:

    • fix: add "no-prototype-builtins" eslint-rule and fix all occurences - f7f05d7
    • fix: add more properties required to be enumerable - 1988878

    Chores / Build:

    • fix: use !== 0 instead of != 0 - c02b05f
    • add chai and dirty-chai and sinon, for cleaner test-assertions and spies, deprecate old assertion-methods - 93e284e, 886ba86, 0817dad, 93516a0

    Security:

    • The properties __proto__, __defineGetter__, __defineSetter__ and __lookupGetter__ have been added to the list of "properties that must be enumerable". If a property by that name is found and not enumerable on its parent, it will silently evaluate to undefined. This is done in both the compiled template and the "lookup"-helper. This will prevent new Remote-Code-Execution exploits that have been published recently.

    Compatibility notes:

    • Due to the security-fixes. The semantics of the templates using __proto__, __defineGetter__, __defineSetter__ and __lookupGetter__ in the respect that those expression now return undefined rather than their actual value from the proto.
    • The semantics have not changed in cases where the properties are enumerable, as in:
    {
      __proto__: 'some string'
    }
    
    • The change may be breaking in that respect, but we still only increase the patch-version, because the incompatible use-cases are not intended, undocumented and far less important than fixing Remote-Code-Execution exploits on existing systems.

    Commits

    v4.5.2 - November 13th, 2019

    Bugfixes

    • fix: use String(field) in lookup when checking for "constructor" - d541378
    • test: add fluent API for testing Handlebars - c2ac79c

    Compatibility notes:

    • no incompatibility are to be expected
    ... (truncated)
    Commits
    • c819c8b v4.5.3
    • 827c9d0 Update release notes
    • f7f05d7 fix: add "no-prototype-builtins" eslint-rule and fix all occurences
    • 1988878 fix: add more properties required to be enumerable
    • 886ba86 test/chore: add chai/expect and sinon to "runtime"-environment
    • 0817dad test: add sinon as global variable to eslint in the specs
    • 93516a0 test: add sinon.js for spies, deprecate current assertions
    • 93e284e chore: add chai and dirty-chai for better test assertions
    • c02b05f fix: use !== 0 instead of != 0
    • 8de121d v4.5.2
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot ignore this [patch|minor|major] version will close this PR and stop Dependabot creating any more for this minor/major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • chore(deps): bump http-proxy from 1.18.0 to 1.18.1 in /web

    chore(deps): bump http-proxy from 1.18.0 to 1.18.1 in /web

    Bumps http-proxy from 1.18.0 to 1.18.1.

    Changelog

    Sourced from http-proxy's changelog.

    v1.18.1 - 2020-05-17

    Merged

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • chore(deps): bump websocket-extensions from 0.1.3 to 0.1.4 in /web

    chore(deps): bump websocket-extensions from 0.1.3 to 0.1.4 in /web

    Bumps websocket-extensions from 0.1.3 to 0.1.4.

    Changelog

    Sourced from websocket-extensions's changelog.

    0.1.4 / 2020-06-02

    • Remove a ReDoS vulnerability in the header parser (CVE-2020-7662, reported by Robert McLaughlin)
    • Change license from MIT to Apache 2.0
    Commits
    • 8efd0cd Bump version to 0.1.4
    • 3dad4ad Remove ReDoS vulnerability in the Sec-WebSocket-Extensions header parser
    • 4a76c75 Add Node versions 13 and 14 on Travis
    • 44a677a Formatting change: {...} should have spaces inside the braces
    • f6c50ab Let npm reformat package.json
    • 2d211f3 Change markdown formatting of docs.
    • 0b62083 Update Travis target versions.
    • 729a465 Switch license to Apache 2.0.
    • See full diff in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • chore(deps): bump acorn from 5.7.3 to 5.7.4 in /web

    chore(deps): bump acorn from 5.7.3 to 5.7.4 in /web

    Bumps acorn from 5.7.3 to 5.7.4.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(v4.1.1)
Owner
Tree Xie
Do something or nothing
Tree Xie
A memory-safe SSH server, focused on listening only on VPN networks such as Tailscale

Features Is tested to work with SCP Integrates well with systemd Quickstart Download binary for your architecture. We only support Linux. If you don't

function61.com 2 Jun 10, 2022
Http-server - A HTTP server and can be accessed via TLS and non-TLS mode

Application server.go runs a HTTP/HTTPS server on the port 9090. It gives you 4

Vedant Pareek 0 Feb 3, 2022
Open Source HTTP Reverse Proxy Cache and Time Series Dashboard Accelerator

Trickster is an HTTP reverse proxy/cache for http applications and a dashboard query accelerator for time series databases. Learn more below, and chec

null 1.8k Jan 2, 2023
gRPC LRU-cache server and client with load test

gRPC k-v storage with LRU-cache server & client + load test. Specify LRU-cache capacity: server/cmd/app.go -> StorageCapacity go build ./server/cmd/*

Eldar Mamedov 0 Dec 26, 2021
Proxy server with go context, go cache

proxy-tut 1. 프록시 서버의 기능 클라이언트에게 요청이 오면 캐시되었는지 확인한후, 캐시되어있다면 본서버로 요청을 보내지 않고 데이터를 내려줍니다. 동일한 요청이 여러개 들어올 시(본서버에서 응답을 받기전이라 캐시되지 않은경우) 동일한 요청은 본서버에 한번만

jaeho choi 0 Jan 13, 2022
A LoRaWAN nodes' and network simulator that works with a real LoRaWAN environment (such as Chirpstack) and equipped with a web interface for real-time interaction.

LWN Simulator A LoRaWAN nodes' simulator to simulate a LoRaWAN Network. Table of Contents General Info Requirements Installation General Info LWN Simu

ARSLab 33 Nov 20, 2022
Prometheus exporter for ping metrics such as RTT, packet loss, and jitter to any number of hosts.

ping_exporter Command ping_exporter provides a Prometheus exporter for ping metrics such as RTT, packet loss, and jitter to any number of hosts. Usage

Beard Slayer 7 Sep 24, 2022
Go-http-sleep: Delayed response http server, useful for testing various timeout issue for application running behind proxy

delayed response http server, useful for testing various timeout issue for application running behind proxy

guessi 0 Jan 22, 2022
Echo-server - An HTTP echo server designed for testing applications and proxies

echo-server An HTTP echo server designed for testing applications and proxies. R

Erik Cavalcanti 6 Dec 20, 2022
“Dear Port80” is a zero-config TCP proxy server that hides SSH connection behind a HTTP server!

Dear Port80 About The Project: “Dear Port80” is a zero-config TCP proxy server that hides SSH connection behind a HTTP server! +---------------------

Abbas Gheydi 6 Jun 29, 2022
Fast HTTP package for Go. Tuned for high performance. Zero memory allocations in hot paths. Up to 10x faster than net/http

fasthttp Fast HTTP implementation for Go. Currently fasthttp is successfully used by VertaMedia in a production serving up to 200K rps from more than

Aliaksandr Valialkin 18.9k Jan 5, 2023
Go HTTP tunnel is a reverse tunnel based on HTTP/2.

Go HTTP tunnel is a reverse tunnel based on HTTP/2. It enables you to share your localhost when you don't have a public IP.

Michal Jan Matczuk 3k Dec 28, 2022
Simple GUI to convert Charles headers to golang's default http client (net/http)

Charles-to-Go Simple GUI to convert Charles headers to golang's default http client (net/http) Usage Compile code to a binary, go build -ldflags -H=wi

null 0 Dec 14, 2021
Http-logging-proxy - A HTTP Logging Proxy For Golang

http-logging-proxy HTTP Logging Proxy Description This project builds a simple r

null 3 Aug 1, 2022
Http-recorder - Application for record http response as static files

http-recorder This is a application for record http response as static files. Th

null 1 Mar 21, 2022
httpstream provides HTTP handlers for simultaneous streaming uploads and downloads of objects, as well as persistence and a standalone server.

httpfstream httpfstream provides HTTP handlers for simultaneous streaming uploads and downloads of files, as well as persistence and a standalone serv

Sourcegraph 16 May 1, 2021
Executing shell commands via HTTP server

shell2http HTTP-server to execute shell commands. Designed for development, prototyping or remote control. Settings through two command line arguments

Sergey Mudrik 1k Jan 1, 2023
Lightweight network boot/install server (DHCP, TFTP, HTTP)

netbootd netbootd is a lightweight network boot server, designed for maximum flexibility and with "batteries included" approach in mind, serving as a

Daniel Speichert 173 Dec 22, 2022