Easegress (formerly known as EaseGateway)is an all-rounder traffic orchestration system

Overview

Easegress

What is Easegress

Easegress (formerly known as EaseGateway)is an all-rounder traffic orchestration system, which is designed for:

  • High Availability: Built-in Raft consensus & leader election makes 99.99% availability.
  • Traffic Orchestration: Dynamically orchestrating various filters to a traffic pipeline.
  • High Performance: Lightweight and essential features speed up the performance.
  • Observability: There are many meaningful statistics periodically in a readable way.
  • Extensibility: It's easy to develop your own filter or controller with high-level programming language.
  • Integration: The simple interfaces make it easy to integrate with other systems, such as Kubernetes Ingress, EaseMesh sidecar, Workflow, etc.

The architecture of Easegress:

architecture

Features

  • Service Management
    • Multiple protocols:
      • HTTP/1.1
      • HTTP/2
      • HTTP/3(QUIC)
      • MQTT(coming soon)
    • Rich Routing Rules: exact path, path prefix, regular expression of the path, method, headers.
    • Resilience&Fault Tolerance
      • Circuit breaker: temporarily blocks possible failures.
      • Rate limiter: limits the rate of incoming requests.
      • Retryer: repeats failed executions.
      • Time limiter: limits the duration of execution.
    • Deployment Management
      • Blue-green Strategy: switches traffic at one time.
      • Canary Strategy: schedules traffic slightly.
    • API Management
      • API Aggregation: aggregates results of multiple APIs.
      • API Orchestration: orchestrates the flow of APIs.
    • Security
      • IP Filter: Limits access to IP addresses.
      • Static HTTPS: static certificate files.
      • API Signature: supports HMAC verification.
      • JWT Verification: verifies JWT Token.
      • OAuth2: validates OAuth/2 requests.
      • Let's Encrypt: automatically manage certificate files.
    • Pipeline-Filter Mechanism
      • Chain of Responsibility Pattern: orchestrates filters chain.
      • Filter Management: makes it easy to develop new filters.
    • Service Mesh
      • Mesh Master: is the control plane to manage the lifecycle of mesh services.
      • Mesh Sidecar: is the data plane as the endpoint to do traffic interception and routing.
      • Mesh Ingress Controller: is the mesh-specific ingress controller to route external traffic to mesh services.
    • Third-Part Integration
      • FaaS integrates with the serverless platform Knative.
      • Service Discovery integrates with Eureka, Consul, Etcd, and Zookeeper.
  • High Performance and Availability
    • Adaption: adapts request, response in the handling chain.
    • Validation: headers validation, OAuth2, JWT, and HMAC verification.
    • Load Balance: round-robin, random, weighted random, ip hash, header hash.
    • Cache: for the backend servers.
    • Compression: compresses body for the response.
    • Hot-Update: updates both config and binary of Easegress in place without losing connections.
  • Operation
    • Easy to Integrate: command line(egctl), MegaEase Portal, HTTP clients such as curl, postman, etc.
    • Distributed Tracing
    • Observability
      • Node: role(leader, writer, reader), health or not, last heartbeat time, and so on
      • Traffic: in multi-dimension: server and backend.
        • Throughput: total and error statistics of request count, TPS/m1, m5, m15, and error percent, etc.
        • Latency: p25, p50, p75, p95, 98, p99, p999.
        • Data Size: request and response size.
        • Status Codes: HTTP status codes.
        • TopN: sorted by aggregated APIs(only in server dimension).

Get Started

The basic common usage of Easegress is to quickly set up proxy for the backend servers. We split it into multiple simple steps to illustrate the essential concepts and operations.

Setting up Easegress

We can download the binary from release page. For example we use linux version:

$ wget https://github.com/megaease/easegress/releases/download/v1.0.0/easegress_Linux_x86_64.tar.gz
$ tar zxvf easegress_Linux_x86_64.tar.gz -C easegress && cd easegress

or use source code:

$ git clone https://github.com/megaease/easegress && cd easegress
$ make

Then we can add the binary directory to the PATH and execute the server:

$ export PATH=${PATH}:$(pwd)/bin/
$ easegress-server
2021-05-17T16:45:38.185+08:00	INFO	cluster/config.go:84	etcd config: init-cluster:eg-default-name=http://localhost:2380 cluster-state:new force-new-cluster:false
2021-05-17T16:45:38.185+08:00	INFO	cluster/cluster.go:379	client is ready
2021-05-17T16:45:39.189+08:00	INFO	cluster/cluster.go:590	server is ready
2021-05-17T16:45:39.21+08:00	INFO	cluster/cluster.go:451	lease is ready
2021-05-17T16:45:39.231+08:00	INFO	cluster/cluster.go:187	cluster is ready
2021-05-17T16:45:39.253+08:00	INFO	supervisor/supervisor.go:180	create system controller StatusSyncController
2021-05-17T16:45:39.253+08:00	INFO	cluster/cluster.go:496	session is ready
2021-05-17T16:45:39.253+08:00	INFO	api/api.go:96	api server running in localhost:2381
2021-05-17T16:45:44.235+08:00	INFO	cluster/member.go:210	self ID changed from 0 to 689e371e88f78b6a
2021-05-17T16:45:44.236+08:00	INFO	cluster/member.go:137	store clusterMembers: eg-default-name(689e371e88f78b6a)=http://localhost:2380
2021-05-17T16:45:44.236+08:00	INFO	cluster/member.go:138	store knownMembers  : eg-default-name(689e371e88f78b6a)=http://localhost:2380

The default target of Makefile is to compile two binary into the directory bin/. bin/easegress-server is the server-side binary, bin/egctl is the client-side binary. We could add it to the $PATH for simplifying the following commands.

We could run easegress-server without specifying any arguments, which launch itself by opening default ports 2379, 2380, 2381. Of course, we can change them in the config file or command arguments that are explained well in easegress-server --help.

$ egctl member list
- options:
    name: eg-default-name
    labels: {}
    cluster-name: eg-cluster-default-name
    cluster-role: writer
    cluster-request-timeout: 10s
    cluster-listen-client-urls:
    - http://127.0.0.1:2379
    cluster-listen-peer-urls:
    - http://127.0.0.1:2380
    cluster-advertise-client-urls:
    - http://127.0.0.1:2379
    cluster-initial-advertise-peer-urls:
    - http://127.0.0.1:2380
    cluster-join-urls: []
    api-addr: localhost:2381
    debug: false
    home-dir: ./
    data-dir: data
    wal-dir: ""
    log-dir: log
    member-dir: member
    cpu-profile-file: ""
    memory-profile-file: ""
  lastHeartbeatTime: "2021-05-05T15:43:27+08:00"
  etcd:
    id: a30c34bf7ec77546
    startTime: "2021-05-05T15:42:37+08:00"
    state: Leader

After launched successfully, we could check the status of the one-node cluster. It shows the static options and dynamic status of heartbeat and etcd.

Create an HTTPServer and Pipeline

Now let's create an HTTPServer listening on port 10080 to handle the HTTP traffic.

$ echo '
kind: HTTPServer
name: server-demo
port: 10080
keepAlive: true
https: false
rules:
  - paths:
    - pathPrefix: /pipeline
      backend: pipeline-demo' | egctl object create

The rules of routers above mean that it will lead the traffic with the prefix /pipeline to the pipeline pipeline-demo, which will be created below. If we curl it before created, it will return 503.

$ echo '
name: pipeline-demo
kind: HTTPPipeline
flow:
  - filter: proxy
filters:
  - name: proxy
    kind: Proxy
    mainPool:
      servers:
      - url: http://127.0.0.1:9095
      - url: http://127.0.0.1:9096
      - url: http://127.0.0.1:9097
      loadBalance:
        policy: roundRobin' | egctl object create

The pipeline means it will do proxy for 3 backend endpoints in load balance policy roundRobin.

Test

Now you can use some HTTP clients such as curl to test the feature:

$ curl -v http://127.0.0.1:10080/pipeline

If you are not set up some applications to handle the 9095, 9096, and 9097 in the localhost, it will return 503 too. We prepare a simple service to let us test handily, the example shows:

$ go run test/backend-service/mirror.go & # Running in background
$ curl http://127.0.0.1:10080/pipeline -d 'Hello, Easegress'
Your Request
===============
Method: POST
URL   : /pipeline
Header: map[Accept:[*/*] Accept-Encoding:[gzip] Content-Type:[application/x-www-form-urlencoded] User-Agent:[curl/7.64.1]]
Body  : Hello, Easegress

More Filters

Now we want to add more features to the pipeline, then we could add kinds of filters to the pipeline. For example, we want validation and request adaptation for the pipeline-demo.

$ cat pipeline-demo.yaml
name: pipeline-demo
kind: HTTPPipeline
flow:
  - filter: validator
    jumpIf: { invalid: END }
  - filter: requestAdaptor
  - filter: proxy
filters:
  - name: validator
    kind: Validator
    headers:
      Content-Type:
        values:
        - application/json
  - name: requestAdaptor
    kind: RequestAdaptor
    header:
      set:
        X-Adapt-Key: goodplan
  - name: proxy
    kind: Proxy
    mainPool:
      servers:
      - url: http://127.0.0.1:9095
      - url: http://127.0.0.1:9096
      - url: http://127.0.0.1:9097
      loadBalance:
        policy: roundRobin

$ egctl object update -f pipeline-demo.yaml

After updating the pipeline, the original curl -v http://127.0.0.1:10080/pipeline will get 400 because of the validating. So we changed it to satisfy the limitation:

$ curl http://127.0.0.1:10080/pipeline -H 'Content-Type: application/json' -d '{"message": "Hello, Easegress"}'
Your Request
===============
Method: POST
URL   : /pipeline
Header: map[Accept:[*/*] Accept-Encoding:[gzip] Content-Type:[application/json] User-Agent:[curl/7.64.1] X-Adapt-Key:[goodplan]]
Body  : {"message": "Hello, Easegress"}

We can also see Easegress send one more header X-Adapt-Key: goodplan to the mirror service.

Documentation

See reference and developer guide for more information.

Roadmap

See Easegress Roadmap for details.

License

Easegress is under the Apache 2.0 license. See the LICENSE file for details.

Issues
  • WASM:The new plugin mechanism for the Easegress

    WASM:The new plugin mechanism for the Easegress

    Backgroud

    Our flagship product Easegress(EaseGateway) has many splendid features, it is fascinating, especially our pipeline/plugin mechanism which empower customers to achieve their specific goal with the Easegress customizing way But the current pipeline/plugin mechanism still has too many barriers to use If a user really wants to extend the Easegress he needs to conquer the following issues:

    1. Master the Go language
    2. Master and understand low level pipeline/plugin mechanism
    3. Commit changes to the Easegress repository and rebuild the Easegress server
    4. Deploy Easegress, need to reboot the Easegress server

    I think the last two of these barriers are the biggest obstacles for users to extend the Easegress. So I think we need another pipeline/plugin mechanism for the EG customization.

    Goal

    Compare with other gateway productions, we can found they are all choosing a solution that is embedding a weak language to enhance the power of extensibility. but there are serval cons in these implementations.

    • Weak language (general the embedded language is the Lua), it's difficult to solve complicated business logic in users' scenarios
    • Performance penalty, as the Lua is a lightweight interpreted programming language, although the Lua introduces the JIT mechanism, it can't be high performance.

    If we want to provide a more flexible customization mechanism, we must solve the above disadvantages.

    Proposal

    After several days of study, I found we can leverage WebAssembly to solve the above problems.(被打脸了……), because the WebAssembly has the following feature:

    • Near-native performance.
    • Wasm expands to any language.
    • Wasm runs in an isolated VM and can dynamically update without EG restarts.
    • Eliminate the need to recompile and maintain a build of EG.

    Golang has rich ecology, I found an open-source Golang WebAssembly runtime library at [1].

    PS: I don't want to deprecate the current pipeline/plugin mechanism, but I think we need multiple customized abstraction, the different way to process the different scene. This solution has been adopted by Envoy as its filter's extensibility [2].

    [1] https://github.com/wasmerio/wasmer-go [2] https://www.envoyproxy.io/docs/envoy/latest/start/sandboxes/wasm-cc

    enhancement 
    opened by zhao-kun 26
  • [question]Is there a global filter ?

    [question]Is there a global filter ?

    Our company wants to customize and develop a filter. For business reasons, it is found that all pipelines must use this filter. So I want to ask if there are plans to support this global filter or how to implement this global filter is the best way.

    enhancement question go 
    opened by brzyangg 18
  • BusinessController creation order issue

    BusinessController creation order issue

    I uses EtcdServiceRegistry in my HttpPipeline. When the easegress is started, the BussinessController will be created in turn. If the HttpPipeline is created before the EtcdServiceRegistry, an error will be reported. The details are as follows:

    2021-10-12T16:12:25.143+08:00   ERROR   proxy/server.go:139  get service palfish-service-registry/base/gateway-controller failed: palfish-service-registry not found
    

    Errors are often reported during startup. Although there is no impact on the program, there is an ERROR level log, I always feel that there is something wrong.

    opened by brzyangg 15
  • ResourceExhausted error when creating many HTTPPipelines and HTTPServer paths

    ResourceExhausted error when creating many HTTPPipelines and HTTPServer paths

    Describe the bug Easegress server throws 2021-12-16T03:40:40.8Z ERROR statussynccontroller/statussynccontroller.go:217 sync status failed: rpc error: code = ResourceExhausted desc = trying to send message larger than max (2274303 vs. 2097152) when creating 1000 dummy HTTPPipelines and one HTTPServer with one backend rule for each pipeline.

    To Reproduce Steps to reproduce the behavior:

    1. Generate 1000 identical HTTPPipeline configurations, with unique names name: pipeline-$i. Run the second script provided below: bash generate_pipeline.sh > pipelines.yaml
    2. Generate one HTTPServer configuration with 1000 rules using the first script provided below: bash generate_server.sh > httpserver.yaml
    3. Start Easegress server: bin/easegress-server
    4. Create pipelines bin/easegress-server object create -f pipelines.yaml
    5. Create httpserver bin/easegress-server object create -f httpserver.yaml
    6. Easegress server will fail in the middle of the object creation.

    Expected behavior Easegress-server should not fail when creating (many) objects.

    Version 1.4.0

    Configuration

    • Easegress Configuration Default parameters.

    • HTTP server configuration The following bash script generates the server HTTPServer:

    #!/bin/bash
    echo "
    kind: HTTPServer
    name: server-demo
    port: 10080
    keepAlive: true
    https: false
    maxConnections: 10240
    rules:
      - paths:
    "
    for i in {0..1000..1}
       do
          echo "    - pathPrefix: /pipeline$i
          backend: pipeline-$i"
    done
    
    • Pipeline Configuration The following bash script generates the pipeline:
    #!/bin/bash
    for i in {0..1000..1}
       do
          echo "name: pipeline-$i
    kind: HTTPPipeline
    flow:
      - filter: proxy
    filters:
      - kind: Proxy
        name: proxy
        mainPool:
          loadBalance:
            policy: roundRobin
          servers:
          - url: http://172.20.2.14:9095
          - url: http://172.20.2.160:9095
    ---"
    done
    

    Logs This is the output of easegress-server when the error happens:

    2021-12-16T04:45:22.857Z	INFO	trafficcontroller/trafficcontroller.go:424	create http pipeline default/pipeline-977
    2021-12-16T04:45:22.858Z	INFO	trafficcontroller/trafficcontroller.go:424	create http pipeline default/pipeline-989
    2021-12-16T04:45:22.858Z	INFO	trafficcontroller/trafficcontroller.go:424	create http pipeline default/pipeline-980
    2021-12-16T04:45:22.858Z	INFO	trafficcontroller/trafficcontroller.go:424	create http pipeline default/pipeline-966
    2021-12-16T04:45:22.859Z	INFO	trafficcontroller/trafficcontroller.go:424	create http pipeline default/pipeline-974
    2021-12-16T04:46:05.821Z	ERROR	statussynccontroller/statussynccontroller.go:217	sync status failed: rpc error: code = ResourceExhausted desc = trying to send message larger than max (2274327 vs. 2097152)
    2021-12-16T04:46:10.775Z	ERROR	statussynccontroller/statussynccontroller.go:217	sync status failed: rpc error: code = ResourceExhausted desc = trying to send message larger than max (2274327 vs. 2097152)
    2021-12-16T04:46:15.795Z	ERROR	statussynccontroller/statussynccontroller.go:217
    

    OS and Hardware

    • OS: Ubuntu 20.04
    • CPU: Intel(R) Xeon(R)
    • Memory: 15GB
    opened by samutamm 12
  • 配置http代理完成后,不能访问代理的内容

    配置http代理完成后,不能访问代理的内容

    Describe the bug 正常启动easegress,配置完成httpserver、httppipeline后,启动无异常。

    但是不能通过http://10.10..:10080 访问代理内容。

    To Reproduce Steps to reproduce the behavior:

    1. Execute ' easegress-server --ip-addr xxx.xxx.12.3'
    2. Send a request to 'postman http get'
    3. no response

    Expected behavior A clear and concise description of what you expected to happen.

    Version 1.4.1

    Configuration

    • Easegress Configuration default
    • HTTP server configuration
    autoCert: false
    caCertBase64: ""
    cacheSize: 0
    certBase64: ""
    certs: {}
    http3: false
    https: false
    keepAlive: true
    keepAliveTimeout: 60s
    keyBase64: ""
    keys: {}
    kind: HTTPServer
    maxConnections: 10240
    name: server-demo
    port: 10080
    rules:
    - host: ""
      hostRegexp: ""
      paths:
      - backend: pipeline-demo
        headers: []
        pathPrefix: /pipeline
        rewriteTarget: ""
    tracing: null
    xForwardedFor: false
    
    • Pipeline Configuration
    filters:
    - kind: Proxy
      mainPool:
        loadBalance:
          policy: roundRobin
        servers:
        - url: http://www.baidu.com
        - url: http://www.baidu.com
        - url: http://www.baidu.com
      name: proxy
    flow:
    - filter: proxy
      jumpIf: {}
    kind: HTTPPipeline
    name: pipeline-demo
    

    Logs

    Easegress logs, if applicable.
    

    OS and Hardware

    • OS: centos 7.x
    • CPU: Intel(R) Core(TM) i5-8265U
    • Memory: 32GB

    Additional context Add any other context about the problem here.

    opened by martinou 11
  • support windows

    support windows

    opened by tg123 10
  • can not build easegress cluster

    can not build easegress cluster

    sudo ./server --api-addr 192.168.42.103:38080 \
    --cluster-listen-client-urls  http://192.168.42.103:2379 \
    --cluster-listen-peer-urls http://192.168.42.103:2380 \
    --cluster-advertise-client-urls http://192.168.42.103:2379 \
    --cluster-initial-advertise-peer-urls http://192.168.42.103:2380 \
    --cluster-join-urls http://192.168.42.104:2380,http://192.168.42.103:2380,http://192.168.42.105:2380 \
    --cluster-name gw-cluster --force-new-cluster false --cluster-role writer --name gw1 
    
    sudo ./server --api-addr 192.168.42.104:38080 \
    --cluster-listen-client-urls  http://192.168.42.104:2379 \
    --cluster-listen-peer-urls http://192.168.42.104:2380 \
    --cluster-advertise-client-urls http://192.168.42.104:2379 \
    --cluster-initial-advertise-peer-urls http://192.168.42.104:2380 \
    --cluster-join-urls http://192.168.42.105:2380,http://192.168.42.103:2380,http://192.168.42.104:2380 \
    --force-new-cluster false --cluster-name gw-cluster --cluster-role writer --name gw2 
    
    sudo ./server --api-addr 192.168.42.105:38080 \
    --cluster-listen-client-urls  http://192.168.42.105:2379 \
    --cluster-listen-peer-urls http://192.168.42.105:2380 \
    --cluster-advertise-client-urls http://192.168.42.105:2379 \
    --cluster-initial-advertise-peer-urls http://192.168.42.105:2380 \
    --cluster-join-urls http://192.168.42.103:2380,http://192.168.42.105:2380,http://192.168.42.104:2380 \
    --force-new-cluster false --cluster-name gw-cluster --cluster-role writer --name gw3 
    

    I run above command on three nodes.

    图片

    Above is the easegress running result.

    I have create a http server like this:

    图片

    But easegress just start a http server on node(192.168.42.103), didn't create http server on node(42.104/42.105). T think when i create a http server on node(42.103), the easegree-cluster should create http server on node(42.104/105) by object watcher notify.

    And i find that each node etcd state is leader. I think etcd-cluster can only have one leader.

    图片 图片

    question 
    opened by jxd134 8
  • Filter HttpResponse  return two same headers to client

    Filter HttpResponse return two same headers to client

    Describe the bug In any one filter,as follow example shows setting some response header:

    func (f *XXXFilter) Handle(ctx context.HTTPContext) (result string) {
        ctx.Response().Header().Set("key-xxx", "value-xxx")
        ctx.Response().Header().Set("key-yyy", "value-yyy")
    }
    

    The client(whether using curl or chrome browser) will receive two same headers: key-xxx: value-xxx key-xxx: value-xxx key-yyy: value-yyy key-yyy: value-yyy

    This will cause CORSAdaptor filter not working properly

    Version v1.4.1

    bug 
    opened by jthann 7
  • fix httpserver httppipeline status not show error

    fix httpserver httppipeline status not show error

    Fix #438

    return status is map -> map[member id]status... Since our egctl does not specific namespace now, we return status from all namespace. this part may need change.

    opened by suchen-sci 7
  • Proxy with mirrorPool panics under high QPS

    Proxy with mirrorPool panics under high QPS

    Describe the bug

    I created HTTPPipeline with Proxy filter that has mirrorPool pointing to an echo server and mainPool pointing to another echo server. I can query HTTPServer with curl without any problem, but if I send many queries in parallel with ab tool, there is an panic: close of closed channel error after few seconds.

    To Reproduce Steps to reproduce the behavior:

    1. Start Easegress server bin/easegress-server -f eg-server.yaml.
    2. Add HTTPServer with egctl object create -f httpserver.yaml
    3. Add HTTPPipeline with egctl object create -f mirrorpool-pipeline.yaml
    4. Start echo server at HOST2 and HOST3.
    5. Send queries in parallel using ab:
    ab -k -c 450 -n 1000000 -H "X-Mirror: mirror"  http://{HOST1}:10080/pipeline
    

    After few second Easegress server fails with panic: close of closed channel.

    Expected behavior A clear and concise description of what you expected to happen.

    Version 1.4.0

    Configuration

    • Easegress Configuration
    # eg-server.yaml
    name: machine-1
    cluster-name: cluster-stress-test
    cluster-role: primary
    api-addr: localhost:2381
    data-dir: ./data
    wal-dir: ""
    cpu-profile-file: "./cpu-profile.txt"
    memory-profile-file: "./memory-profile.txt"
    disable-access-log: true
    log-dir: ./log
    debug: false
    cluster:
      listen-peer-urls:
       - http://{HOST1}:2380
      listen-client-urls:
       - http://{HOST1}:2379
      advertise-client-urls:
       - http://{HOST1}:2379
      initial-advertise-peer-urls:
       - http://{HOST1}:2380
      initial-cluster:
       - machine-1: http://{HOST1}:2380
    
    • HTTP server configuration
    # httpserver.yaml
    kind: HTTPServer
    name: stress-server
    port: 10080
    keepAlive: true
    https: false
    maxConnections: 10240
    rules:
      - paths:
        - pathPrefix: /pipeline
          backend: mirrorpool-pipeline
    
    • Pipeline Configuration
    name: mirrorpool-pipeline
    kind: HTTPPipeline
    flow:
      - filter: proxy
    filters:
      - kind: Proxy
        name: proxy
        mainPool:
          loadBalance:
            policy: roundRobin
          servers:
          - url: http://{HOST2}:9095
        mirrorPool:
          filter:
            headers:
              "X-Mirror":
                exact: mirror
          loadBalance:
            policy: roundRobin
          servers:
          - url: http://{HOST3}:9095
    

    Logs

    panic: close of closed channel
    
    goroutine 4160 [running]:
    github.com/megaease/easegress/pkg/filter/proxy.(*masterReader).Read(0xc005800390, {0xc00582a000, 0x8000, 0x8000})
    	github.com/megaease/easegress/pkg/filter/proxy/masterslavereader.go:69 +0x17d
    github.com/megaease/easegress/pkg/util/callbackreader.(*CallbackReader).Read(0xc005806b90, {0xc00582a000, 0xc00007c070, 0xc005449000})
    	github.com/megaease/easegress/pkg/util/callbackreader/callbackreader.go:57 +0xa5
    net/http.(*readTrackingBody).Read(0x0, {0xc00582a000, 0xc002afdb00, 0x44ced2})
    	net/http/transport.go:634 +0x2a
    io.(*multiReader).Read(0xc0056cba28, {0xc00582a000, 0x8000, 0x8000})
    	io/multi.go:26 +0x9b
    io.copyBuffer({0x7f3e41f08db0, 0xc0048b1890}, {0x2e430c0, 0xc0056cba28}, {0x0, 0x0, 0x0})
    	io/io.go:423 +0x1b2
    io.Copy(...)
    	io/io.go:382
    net/http.(*transferWriter).doBodyCopy(0xc0058041e0, {0x7f3e41f08db0, 0xc0048b1890}, {0x2e430c0, 0xc0056cba28})
    	net/http/transfer.go:410 +0x4d
    net/http.(*transferWriter).writeBody(0xc0058041e0, {0x2e3d5e0, 0xc002797080})
    	net/http/transfer.go:357 +0x225
    net/http.(*Request).write(0xc00577ac00, {0x2e3d5e0, 0xc002797080}, 0x0, 0xc005775d70, 0x0)
    	net/http/request.go:698 +0xb4e
    net/http.(*persistConn).writeLoop(0xc00153ad80)
    	net/http/transport.go:2389 +0x189
    created by net/http.(*Transport).dialConn
    	net/http/transport.go:1748 +0x1e65
    

    OS and Hardware

    • OS: Ubuntu 20.04
    • CPU: Intel(R) Xeon(R)
    • Memory: 15GB
    • Host 1,2 and 3 are equivalent.
    bug 
    opened by samutamm 7
  • How to make cluster arch architect

    How to make cluster arch architect

    Just confused, easegress has a Leader node?Is traffic all through the leader and forward to other node? I mean ,how to load balance the traffic? Is nginx/vip or something at front of easegress? thank you.

    question 
    opened by Liubey 7
  • Implement a simple data store

    Implement a simple data store

    Components like filters and controllers often need persistent data and share the data inside the cluster, we already have the Custom Data for this but it is too complex to use in most cases. Another way is to use the functions of the cluster package directly, but it is dangerous because it can access any data of the cluster, a wrong implemented component can easily break the whole cluster.

    So we'd like to implement a simple data store that meets the below requirements:

    1. the store has its own fixed (hard-coded) key prefix
    2. when creating an instance of the store, the user needs to provide a second-level prefix.
    3. the store instances provide functions like the cluster package, for example: Get/Set/GetPrefix/SetPrefix and etc.
    4. the actual key used to access data is the two levels of prefixes and the provided key.
    5. client APIs to manage(create/update/delete/list) the data, and the egctl command need to be updated accordingly.
    enhancement 
    opened by localvar 0
  • It will hapen endless loop when a http request through  a mirror pool of pipeline

    It will hapen endless loop when a http request through a mirror pool of pipeline

    Describe the bug when I config the pipeline with mainPool and a mirrorPool,and request with http header to through the mirrorPool(like example http-pipeline-example.yaml in source, set header X-Filter=mirror ), it will response nothing

    To Reproduce Steps to reproduce the behavior:

    1. update http-pipeline-example.yaml in example/config,keep mainPool and mirrorPool,just remain http-pipeline-example.yaml and http-server-example.yaml
    2. start cluster and create objects
    3. send request with header X-Filter=mirror
    4. cannot get any response

    Expected behavior response correct data

    Version all version

    Configuration

    • HTTP server configuration
    kind: HTTPServer
    name: http-server-example
    port: 10080
    https: false
    http3: false
    keepAlive: true
    keepAliveTimeout: 75s
    maxConnection: 10240
    cacheSize: 0
    rules:
      - paths:
        - pathPrefix: /pipeline
          backend: http-pipeline-example
    
    • Pipeline Configuration
    name: http-pipeline-example
    kind: HTTPPipeline
    
    flow:
      - filter: proxy
        jumpIf: { fallback: END }
      - filter: responseAdaptor
    
    filters:
      - name: proxy
        kind: Proxy
        failureCodes: [500, 501]
        fallback:
          forCodes: true
          forCircuitBreaker: false
          mockCode: 200
          mockHeader: {X-Fallback: yes}
          mockBody: '{"message": "Backend failed, try it later."}'
        mirrorPool:
          servers:
          - url: http://127.0.0.1:9092
          loadBalance:
            policy: roundRobin
            headerHashKey: ""
          filter:
            headers:
              X-Filter:
                exact: mirror
        mainPool:
          servers:
          - url: http://127.0.0.1:9091
            tags: ["blue", "v2"]
          loadBalance:
            policy: roundRobin
            headerHashKey: X-User-Id
        compression:
          minLength: 1024
      - name: responseAdaptor
        kind: ResponseAdaptor
        header:
          del: []
          set:
            Server: Easegress v1.0.0
          add:
            X-Proxy-Name: http-proxy-exmaple
    
    bug 
    opened by thewangzl 6
  • Add more mqtt info to kafka headers when mqttproxy publish message to kafka

    Add more mqtt info to kafka headers when mqttproxy publish message to kafka

    Is your feature request related to a problem? Please describe. I want to resend some mqtt topics to the client, I need to know more information, such as current mqtt topic, clientId, username, etc. These info should be put in kafka headers


    Thanks for contributing 🎉!

    opened by codering 0
  • Simplify CORSAdaptor

    Simplify CORSAdaptor

    Is your feature request related to a problem? Please describe. https://github.com/megaease/easegress/pull/498 enabled both CORS pre-flight and CORS requests for CORSAdaptor while originally only CORS pre-flight requests were possible. In many cases both CORS pre-flight and CORS requests are necessary and should be enabled by default.

    Describe the solution you'd like Either

    • change default value of supportCORSRequest to true
    • remove supportCORSRequest and support both CORS pre-flight and CORS requests by default.

    Describe alternatives you've considered

    Additional context


    Thanks for contributing 🎉!

    opened by samutamm 1
Releases(v1.5.3)
Owner
MegaEase
MegaEase
Kagetaka : Decentralized Microservice Orchestration for Saito

Caravela : Decentralized Microservice Orchestration Shamelessly cloned and repurposed for Go 1.17 from André Pires: https://github.com/strabox/caravel

Thomas Pluck 0 Oct 15, 2021
The Consul API Gateway is a dedicated ingress solution for intelligently routing traffic to applications running on a Consul Service Mesh.

The Consul API Gateway is a dedicated ingress solution for intelligently routing traffic to applications running on a Consul Service Mesh.

HashiCorp 65 Jun 15, 2022
A controller(CES) for controlling container egress traffic. Working with F5 AFM.

Container Egress Services (CES) Kubernetes is piloting projects transition to enterprise-wide application rollouts, companies must be able to extend t

F5 DevCentral 10 Jun 1, 2022
Dubbo2istio watches Dubbo ZooKeeper registry and synchronize all the dubbo services to Istio.

Dubbo2Istio Dubbo2istio 将 Dubbo ZooKeeper 服务注册表中的 Dubbo 服务自动同步到 Istio 服务网格中。 Aeraki 根据 Dubbo 服务信息和用户设置的路由规则生成数据面相关的配置,通过 Istio 下发给数据面 Envoy 中的 Dubbo p

Aeraki 25 May 8, 2022
Pulls ARO RP versions and their corresponding upgrade streams for all regions

aro-rp-versions Description Pulls ARO RP versions and their corresponding upgrade streams for all regions in a table format by default or json if requ

Peter Kostyukov 4 Mar 11, 2022
Application tracing system for Go, based on Google's Dapper.

appdash (view on Sourcegraph) Appdash is an application tracing system for Go, based on Google's Dapper and Twitter's Zipkin. Appdash allows you to tr

Sourcegraph 1.7k Jun 22, 2022
a microservice framework for rapid development of micro services in Go with rich eco-system

中文版README Go-Chassis is a microservice framework for rapid development of microservices in Go. it focus on helping developer to deliver cloud native a

null 2.5k Jun 17, 2022
Demo Fully Isolated System Architecture

Fully Isolated System Architecture (Microservices) Arsitektur Request | | | Api Gateway --- Auth Provider |\________________________

Muhamad Surya Iksanudin 27 Jun 1, 2022
This example showcases an event-sourced CQRS system based on github.com/romshark/eventlog

Eventlog Example This example is showcasing an eventually consistent, fault-tolerant, event sourced system following the CQRS (Command-Query-Responsib

Roman Sharkov 3 Mar 13, 2022
HTTP load generator, ApacheBench (ab) replacement, formerly known as rakyll/boom

hey is a tiny program that sends some load to a web application. hey was originally called boom and was influenced from Tarek Ziade's tool at tarekzia

Jaana Dogan 13.8k Jun 29, 2022
HTTP load generator, ApacheBench (ab) replacement, formerly known as rakyll/boom

hey is a tiny program that sends some load to a web application. hey was originally called boom and was influenced from Tarek Ziade's tool at tarekzia

Jaana Dogan 13.8k Jun 24, 2022
SAP (formerly sybase) ASE/RS/IQ driver written in pure go

tds import "github.com/thda/tds" Package tds is a pure Go Sybase ASE/IQ/RS driver for the database/sql package. Status This is a beta release. This dr

Thomas 50 Apr 24, 2022
Package zaperations provides a Google Cloud operations suite (formerly Stackdriver) compatible config for the uber-go/zap logger.

Package zaperations provides a Google Cloud Operations (formerly Stackdriver) compatible config for the excellent uber-go/zap logger. Example This exa

Wayne 0 Nov 6, 2021
A programmable, observable and distributed job orchestration system.

?? Overview Odin is a programmable, observable and distributed job orchestration system which allows for the scheduling, management and unattended bac

James McDermott 439 Jun 23, 2022
Catalyst is an incident response platform / SOAR (Security Orchestration, Automation and Response) system.

Catalyst Speed up your reactions Website - The Catalyst Handbook (Documentation) - Try online (user: bob, password: bob) Catalyst is an incident respo

Security Brewery 94 Jun 23, 2022
Mizu - API traffic viewer for Kubernetes enabling you to view all API communication between microservices

The API Traffic Viewer for Kubernetes A simple-yet-powerful API traffic viewer f

UP9 3.7k Jun 22, 2022
A collection of well-known string hash functions, implemented in Go

This library is a collection of "well-known" 32-bit string hashes, implemented in Go. It includes: Java string hash ELF-32 Jenkins' One-A

Damian Gryski 64 Mar 3, 2022
Fast division, modulus and divisibility checks in Go for divisors known only at runtime.

fastdiv Fast division, modulus and divisibility checks for divisors known only at runtime via the method of: "Faster Remainder by Direct Computation:

null 99 May 17, 2022
A scanner/exploitation tool written in GO, which leverages Prototype Pollution to XSS by exploiting known gadgets.

ppmap A simple scanner/exploitation tool written in GO which automatically exploits known and existing gadgets (checks for specific variables in the g

kleiton0x00 324 Jun 20, 2022
asciigrid is a Go package that implements decoder and encoder for the Esri ASCII grid format, also known as ARC/INFO ASCII GRID.

asciigrid asciigrid is a Go package that implements decoder and encoder for the Esri ASCII grid format, also known as ARC/INFO ASCII GRID. Install go

Ahmet Artu Yildirim 0 Oct 16, 2021
Hybridnet is an open source container networking solution, integrated with Kubernetes and used officially by following well-known PaaS platforms

Hybridnet What is Hybridnet? Hybridnet is an open source container networking solution, integrated with Kubernetes and used officially by following we

Alibaba 130 Jun 16, 2022
A passive reconnaissance tool for known URLs discovery

A passive reconnaissance tool for known URLs discovery - it gathers a list of URLs passively using various online sources.

Signed Security 177 Jun 24, 2022
Build a local copy of Known Exploited Vulnerabilities Catalog by CISA. Server mode for easy querying.

go-kev go-kev build a local copy of Known Exploited Vulnerabilities Catalog by CISA. Usage $ go-kev help Go Known Exploited Vulnerabilities Usage:

MaineK00n 11 Apr 18, 2022
Go module that allows you to authenticate to Azure with a well known client ID using interactive logon and grab the token

azureimposter Go module that pretends to be any clientID and grabs an authentication token from Azure using interactive login (w/mfa if enabled) and r

Lars Karlslund 22 Jun 23, 2022
Build a local copy of Known Exploited Vulnerabilities Catalog by CISA. Server mode for easy querying.

go-kev go-kev build a local copy of Known Exploited Vulnerabilities Catalog by CISA. Usage $ go-kev help Go Known Exploited Vulnerabilities Usage:

null 11 Apr 18, 2022
Package filippo.io/intermediates embeds a bundle of known unexpired, unrevoked intermediate certificates chaining to roots in the Mozilla Root Program

filippo.io/intermediates Package intermediates embeds a list of known unexpired, unrevoked intermediate certificates chaining to roots with Websites t

Filippo Valsorda 45 Jan 22, 2022
A very simple utility that allows you to run the desired command or script as soon as a certain process with a known PID completes correctly or with an error.

go-monkill A very simple utility that allows you to run the desired command or script as soon as a certain process with a known PID completes correctl

Michael Savin 6 Mar 31, 2022