Ultra performant API Gateway with middlewares

Overview

Krakend logo

The KrakenD framework

Go Report Card Coverage Status GoDoc CII Best Practices Docker Pulls Slack Widget

An open framework to assemble ultra performance API Gateways with middlewares; core service of the KrakenD API Gateway.

Looking for the API gateway ready to use?

Site | Download | Build | Documentation | Blog

Motivation

Consumers of REST API content (specially in microservices) often query backend services that weren't coded for the UI implementation. This is of course a good practice, but the UI consumers need to do implementations that suffer a lot of complexity and burden with the sizes of their microservices responses.

KrakenD is an API Gateway builder and proxy generator that sits between the client and all the source servers, adding a new layer that removes all the complexity to the clients, providing them only the information that the UI needs. KrakenD acts as an aggregator of many sources into single endpoints and allows you to group, wrap, transform and shrink responses. Additionally it supports a myriad of middlewares and plugins that allow you to extend the functionality, such as adding Oauth authorization or security layers.

KrakenD not only supports HTTP(S), but because it is a set of generic libraries you can build all type of API Gateways and proxies, including for instance, a RPC gateway.

Practical Example

A mobile developer needs to construct a single front page that requires data from 4 different calls to their backend services, e.g:

1) api.store.server/products
2) api.store.server/marketing-promos
3) api.users.server/users/{id_user}
4) api.users.server/shopping-cart/{id_user}

The screen is very simple and the mobile client only needs to retrieve data from 4 different sources, wait for the round trip and then hand pick only a few fields from the response.

What if the mobile could call a single endpoint?

1) krakend.server/frontpage/{id_user}

That's something KrakenD can do for you. And this is how it would look like:

Gateway

KrakenD would merge all the data and return only the fields you need (the difference in size in the graph).

Visit the KrakenD website for more information.

What's in this repository?

The source code on which the KrakenD service core is built on. It is designed to work with your own middleware and extend the functionality by using small, independent, reusable components following the Unix philosophy.

Use this repository if want to build from source your API Gateway or if you want to reuse the components in another application.

If you need the KrakenD API Gateway download the binary for your architecture or build it yourself.

Library Usage

KrakenD is presented as a go library that you can include in your own go application to build a powerful proxy or API gateway. In order to get you started several examples of implementations are included in the examples folder.

Of course you will need golang installed in your system to compile the code.

A ready to use example:

    package main

    import (
        "flag"
        "log"
        "os"

        "github.com/devopsfaith/krakend/config"
        "github.com/devopsfaith/krakend/logging"
        "github.com/devopsfaith/krakend/proxy"
        "github.com/devopsfaith/krakend/router/gin"
    )

    func main() {
        port := flag.Int("p", 0, "Port of the service")
        logLevel := flag.String("l", "ERROR", "Logging level")
        debug := flag.Bool("d", false, "Enable the debug")
        configFile := flag.String("c", "/etc/krakend/configuration.json", "Path to the configuration filename")
        flag.Parse()

        parser := config.NewParser()
        serviceConfig, err := parser.Parse(*configFile)
        if err != nil {
            log.Fatal("ERROR:", err.Error())
        }
        serviceConfig.Debug = serviceConfig.Debug || *debug
        if *port != 0 {
            serviceConfig.Port = *port
        }

        logger, _ := logging.NewLogger(*logLevel, os.Stdout, "[KRAKEND]")

        routerFactory := gin.DefaultFactory(proxy.DefaultFactory(logger), logger)

        routerFactory.New().Run(serviceConfig)
    }

Visit the framework overview for more details about the components of the KrakenD.

Examples

The project KrakenD examples

  1. gin router
  2. mux router
  3. gorilla router
  4. negroni middlewares
  5. dns srv service discovery
  6. jwt middlewares
  7. httpcache based proxies

Configuration file

KrakenD config file

Benchmarks

Check out the benchmark results of several KrakenD components

Contributing

We are always happy to receive contributions. If you have questions, suggestions, bugs please open an issue. If you want to submit the code, create the issue and send us a pull request for review.

If you want to contribute on the KrakenD API Gateway binary see the builder

Want more?

Enjoy the KrakenD!

Issues
  • fatal error: concurrent map writes when multiple backends

    fatal error: concurrent map writes when multiple backends

    krakend[4734]: goroutine 1053 [running]:
    krakend[4734]: runtime.throw(0xc8a990, 0x15)
    krakend[4734]: /var/opt/go/1.10/src/runtime/panic.go:619 +0x81 fp=0xc42061d920 sp=0xc42061d900 pc=0x42c921
    krakend[4734]: runtime.mapassign_faststr(0xb74440, 0xc4205adda0, 0xc84b14, 0xd, 0xc4201eacb0)
    krakend[4734]: /var/opt/go/1.10/src/runtime/hashmap_fast.go:703 +0x3e9 fp=0xc42061d990 sp=0xc42061d920 pc=0x40d529
    krakend[4734]: ioki.pl/krakend-ce/oauth1.newMiddleware.func1.1(0xd225c0, 0xc4201a0840, 0xc4205ca140, 0xc4205f85a6, 0xc4205f85c0, 0xc42064ac78)
    krakend[4734]: /home/bartoszgolek/go/src/ioki.pl/krakend-ce/oauth1/backend.go:35 +0x12c fp=0xc42061dbc0 sp=0xc42061d990 pc=0x96f96c
    krakend[4734]: ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend-metrics.NewProxyMiddleware.func1.1(0xd225c0, 0xc4201a0840, 0xc4205ca140, 0x0, 0x0, 0x50)
    krakend[4734]: /home/bartoszgolek/go/src/ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend-metrics/proxy.go:56 +0xd9 fp=0xc42061dc88 sp=0xc42061dbc0 pc=0x95de49
    krakend[4734]: ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend/proxy.newLoadBalancedMiddleware.func1.1(0xd225c0, 0xc4201a0840, 0xc4205ca0f0, 0x412468, 0x10, 0xba0580)
    krakend[4734]: /home/bartoszgolek/go/src/ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend/proxy/balancing.go:59 +0x304 fp=0xc42061ddc0 sp=0xc42061dc88 pc=0x6dba54
    krakend[4734]: ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend/proxy.NewRequestBuilderMiddleware.func1.1(0xd225c0, 0xc4201a0840, 0xc42049ca50, 0xc4201a0840, 0xc4203620d0, 0xc4201282d0)
    krakend[4734]: /home/bartoszgolek/go/src/ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend/proxy/http.go:87 +0x1df fp=0xc42061deb8 sp=0xc42061ddc0 pc=0x6dcd3f
    krakend[4734]: ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend/proxy.requestPart(0xd22640, 0xc420020f60, 0xc4202dd770, 0xc42049ca50, 0xc420020fc0, 0xc420021020)
    krakend[4734]: /home/bartoszgolek/go/src/ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend/proxy/merging.go:61 +0x84 fp=0xc42061dfb0 sp=0xc42061deb8 pc=0x6dad14
    krakend[4734]: runtime.goexit()
    krakend[4734]: /var/opt/go/1.10/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc42061dfb8 sp=0xc42061dfb0 pc=0x459921
    krakend[4734]: created by ioki.pl/krakend-ce/vendor/github.com/devopsfaith/krakend/proxy.NewMergeDataMiddleware.func1.1
    

    I think it is connected to: https://github.com/devopsfaith/krakend/pull/98/commits/4fbd601e5f5f7799cd0e693f7649d7f0ee0db076

    I've found that better solutino would be: https://github.com/devopsfaith/krakend/commit/079376adbc655473d3eabd12e6c91581bdb77597

    help wanted locked 
    opened by bartoszgolek 17
  • How to cache backend responses?

    How to cache backend responses?

    Describe the bug I was interested in caching, so I tested it, but the result was a little confusing

    To Reproduce Steps to reproduce the behavior:

    1. Configuration used
    // port 8000
    .....
    .....
    {
         "endpoint": "/page",
         "backend": [
            {
              "url_pattern": "/page",
              "host": [
                 "http://127.0.0.1:8001"
              ],
              "extra_config": {
                "github.com/devopsfaith/krakend-httpcache": {}
              }
            }
          ]
    }
    
    1. Then i just run krakend-ce , and call endpoint multiple times
    curl -i "http://127.0.0.1:8000/page"
    
    1. However, every time the backend has a requested log, it is not cached after the first request
    2. I noticed that krakend uses sony's go-breaker and gets cache time from Cache-Control. my response header also has Cache-Control

    Expected behavior Only the first request has a log in the backend, and the rest should use the response in the cache

    Logs Krakend-ce: [GIN] 2020/03/13 - 12:20:13 | 200 | 2.541701ms | 127.0.0.1 | GET /page [GIN] 2020/03/13 - 12:20:13 | 200 | 1.019577ms | 127.0.0.1 | GET /page [GIN] 2020/03/13 - 12:20:14 | 200 | 683.903µs | 127.0.0.1 | GET /page

    Servers: [GIN] 2020/03/13 - 12:20:13 | 200 | 1.396364ms | 127.0.0.1 | GET /page [GIN] 2020/03/13 - 12:20:13 | 200 | 342.851µs | 127.0.0.1 | GET /page [GIN] 2020/03/13 - 12:20:14 | 200 | 202.322µs | 127.0.0.1 | GET /page Additional context Add any other context about the problem here.

    bug question locked 
    opened by granty1 16
  • 500 Internal Server Error /  no hosts available

    500 Internal Server Error / no hosts available

    I'm setting up KrakenD for the first time. I'm trying to use SD DNS to work in tandem with Consul. If I specify a static IP everything works fine, but SD DNS isn't working.

    For example, this simple curl example returns a 500: >curl http://10.30.54.185:8000/consul --verbose

    *   Trying 10.30.54.185...
    * TCP_NODELAY set
    * Connected to 10.30.54.185 (10.30.54.185) port 8000 (#0)
    > GET /consul HTTP/1.1
    > Host: 10.30.54.185:8000
    > User-Agent: curl/7.58.0
    > Accept: */*
    >
    < HTTP/1.1 500 Internal Server Error
    < X-Krakend: Version 0.9.0
    < X-Krakend-Completed: false
    < Date: Mon, 15 Jul 2019 19:42:36 GMT
    < Content-Length: 0
    <
    * Connection #0 to host 10.30.54.185 left intact
    

    The configuration file, cat /etc/krakend/krakend.json:

    {
      "version": 2,
      "name": "CAMP API Gateway",
      "timeout": "3000ms",
      "cache_ttl": "300s",
      "port": 8000,
      "output_encoding": "json",
      "extra_config": {
        "github_com/devopsfaith/krakend-gologging": {
          "level": "DEBUG",
          "prefix": "[KRAKEND]",
          "syslog": true,
          "stdout": true,
          "format": "custom",
          "custom_format": "%{message}"
        },
        "github_com/devopsfaith/krakend-cors": {
          "allow_origins": [ "*" ],
          "allow_methods": [ "POST", "GET", "PUT", "DELETE" ],
          "allow_headers": [
            "Origin",
            "Authorization",
            "Content-Type",
            "Accept",
            "X-Auth-Token"
          ],
          "expose_headers": [ "Content-Length" ],
          "max_age": "12h"
        }
      },
      "endpoints": [
        {
          "endpoint": "/consul",
          "timeout": "15s",
          "method": "GET",
          "headers_to_pass": ["*", "Cookie"],
          "backend": [
            {
              "url_pattern": "/v1/status/leader",
              "sd": "dns",
              "host": [
                "http://consul.service.consul:8500"
              ],
              "disable_host_sanitize": true
            }
          ]
        }
      ]
    }
    

    Note: For 'host', I also tried "consul.service.consul:8500" but it made no difference. Log file (notice logging is set to DEBUG level):

    krakend.service - Krakend API Gateway
       Loaded: loaded (/lib/systemd/system/krakend.service; enabled; vendor preset: enabled)
       Active: active (running) since Mon 2019-07-15 19:39:27 UTC; 6min ago
         Docs: http://krakend.io
     Main PID: 5661 (krakend)
        Tasks: 9 (limit: 4599)
       CGroup: /system.slice/krakend.service
               └─5661 /usr/bin/krakend run -c /etc/krakend/krakend.json
    
    Jul 15 19:39:27 krakend1-vm [KRAKEND][5661]: AMQP: http://consul.service.consul:8500: no amqp consumer defined
    Jul 15 19:39:27 krakend1-vm [KRAKEND][5661]: AMQP: http://consul.service.consul:8500: no amqp producer defined
    Jul 15 19:39:27 krakend1-vm [KRAKEND][5661]: CEL: no extra config detected for backend /v1/status/leader
    Jul 15 19:39:27 krakend1-vm [KRAKEND][5661]: CEL: no extra config detected for pipe /consul
    Jul 15 19:39:27 krakend1-vm [KRAKEND][5661]: JOSE: singer disabled for the endpoint /consul
    Jul 15 19:39:27 krakend1-vm [KRAKEND][5661]: JOSE: validator disabled for the endpoint /consul
    Jul 15 19:40:04 krakend1-vm krakend[5661]: [GIN] 2019/07/15 - 19:40:04 | 500 |        67.9µs |    10.30.54.185 | GET      /consul
    Jul 15 19:40:04 krakend1-vm krakend[5661]: Error #01: no hosts available
    Jul 15 19:42:36 krakend1-vm krakend[5661]: [GIN] 2019/07/15 - 19:42:36 | 500 |        13.5µs |    10.30.54.185 | GET      /consul
    Jul 15 19:42:36 krakend1-vm krakend[5661]: Error #01: no hosts available
    

    Results of dig on consul.service.consul:

    ; <<>> DiG 9.11.3-1ubuntu1.8-Ubuntu <<>> consul.service.consul
    ;; global options: +cmd
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37758
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
    
    ;; OPT PSEUDOSECTION:
    ; EDNS: version: 0, flags:; udp: 65494
    ;; QUESTION SECTION:
    ;consul.service.consul.         IN      A
    
    ;; ANSWER SECTION:
    consul.service.consul.  0       IN      A       10.30.54.161
    consul.service.consul.  0       IN      A       10.30.54.171
    
    ;; Query time: 4 msec
    ;; SERVER: 127.0.0.53#53(127.0.0.53)
    ;; WHEN: Mon Jul 15 19:47:27 UTC 2019
    ;; MSG SIZE  rcvd: 82
    

    Those IP addresses are correct - there are two consul servers.

    Also: dig consul.service.consul SRV +short

    1 1 8300 consul2-vm.node.stage-vm.consul.
    1 1 8300 consul1-vm.node.stage-vm.consul.
    

    I can ping those two addresses. This is running on Ubuntu Server 18.04.2 LTS I'm not sure what I'm missing. I've been banging away at this all day. Any help is much appreciated.

    locked 
    opened by mlamothe 16
  • plugin was built with a different version of package github.com/luraproject/lura/v2/register

    plugin was built with a different version of package github.com/luraproject/lura/v2/register

    Describe what are you trying to do Using the github.com/luraproject/lura/v2/proxy for request/response modifier plugin following below KrakenD suggested document.

    Modifier plugin is not getting registered with the below errors. Looks like some of the below libraries mentioned in go.mod as below are not built with the GO version our project is using. Not sure how to overcome this. If I remove proxy.ResponseMetadataWrapper from the ResponseWrapper interface all working fine. I am not sure how to build the proxy package provided by lura to make it compatible with Go 1.17.8.

    Tried with both v2 and 1.4.1 but giving same errors

    V2 with below go.mod giving below errors

    go 1.17

    require ( plugin was built with a different version of package github.com/luraproject/lura/v2/register )

    2022/04/11 08:53:56 KRAKEND DEBUG: [SERVICE: Executor Plugin] plugin #1 (/etc/krakend/plugins/modifierPlugin.so): plugin.Open("/etc/krakend/plugins/modifierPlugin"): plugin was built with a different version of package github.com/luraproject/lura/v2/register

    V1.4.1 with below go.mod giving below errors

    go 1.17

    require ( github.com/devopsfaith/flatmap v0.0.0-20200601181759-8521186182fc // indirect github.com/luraproject/lura v1.4.1 // indirect github.com/valyala/fastrand v1.0.0 // indirect )

    https://www.krakend.io/docs/extending/plugin-modifiers/

    Your configuration file

    Go Lang version tried with 1.17.8 Used this below command to build .so plugin

    go build -buildmode=plugin -o modifierPlugin.so modifierPlugin.go

    Logs 2022/04/07 16:49:51 KRAKEND DEBUG: [SERVICE: Executor Plugin] plugin https://github.com/devopsfaith/krakend-ce/pull/1 (/etc/krakend/plugins/modifierPlugin.so): plugin.Open("/etc/krakend/plugins/modifierPlugin"): plugin was built with a different version of package github.com/valyala/fastrand 2022/04/07 16:49:51 KRAKEND DEBUG: [SERVICE: Handler Plugin] plugin https://github.com/devopsfaith/krakend-ce/pull/1 (/etc/krakend/plugins/modifierPlugin.so): plugin.Open("/etc/krakend/plugins/modifierPlugin.so"): plugin was built with a different version of package github.com/valyala/fastrand (previous failure) 2022/04/07 16:49:51 KRAKEND DEBUG: [SERVICE: Modifier Plugin] plugin #0 (/etc/krakend/plugins/httpHandlerPlugin.so): plugin: symbol ModifierRegisterer not found in plugin plugin/unnamed-be415b995a80d55a5fde08e3201bb850429c9287 2022/04/07 16:49:51 KRAKEND DEBUG: [SERVICE: Modifier Plugin] plugin https://github.com/devopsfaith/krakend-ce/pull/1 (/etc/krakend/plugins/modifierPlugin.so): plugin.Open("/etc/krakend/plugins/modifierPlugin.so"): plugin was built with a different version of package github.com/valyala/fastrand (previous failure)

    opened by krishna-kunderu 13
  • Why am I getting Error #01: Invalid status code?

    Why am I getting Error #01: Invalid status code?

    Hi, it me again, still new to Krakend ! 💃

    When I do a request throug the gateway, I am getting a status code 0 (?) in my response when using the "gin example" of the repository.

    api_gateway_1  | [GIN] 2018/03/15 - 07:49:49 |   0 |   85.720918ms |      172.20.0.1 | POST     /api/v1/users?create_profile=true
    api_gateway_1  | Error #01: Invalid status code
    

    (I still get the response body I expected)

    Then doing the request again:

    api_gateway_1  | [GIN] 2018/03/15 - 07:49:49 |   200 |   85.720918ms |      172.20.0.1 | POST     /api/v1/users?create_profile=true
    

    The request is returning 200 while I am waiting for a 409, nothing is called and nothing is returned.

    I then tried to use the example directly in the README.md (since it is an even simpler version using gin too) and I get better results:

    api_user_1     | [GIN] 2018/03/15 - 07:57:21 | 201 |   86.655951ms |      172.20.0.1 | POST     /api/v1/users?create_profile=true
    api_gateway_1  | [GIN] 2018/03/15 - 07:57:21 | 200 |   89.901626ms |      172.20.0.1 | POST     /api/v1/users?create_profile=true
    

    But I get a 200 while I was expecting a 201 (?) Then I tried to do the request again, it fails as expected but I get a 500 instead of the expected 409 with the message again:

    api_user_1     | [GIN] 2018/03/15 - 07:57:28 | 409 |   51.846764ms |      172.20.0.1 | POST     /api/v1/users?create_profile=true
    api_gateway_1  | [GIN] 2018/03/15 - 07:57:28 | 500 |   52.963005ms |      172.20.0.1 | POST     /api/v1/users?create_profile=true
    api_gateway_1  | Error #01: Invalid status code
    

    go code:

    func main() {
    	serviceConfig, err := config.NewParser().Parse("/etc/krakend/configuration.json")
    	if err != nil {
    		log.Fatal("ERROR:", err.Error())
    	}
    	serviceConfig.Debug = true
    	serviceConfig.Port = 4200
    
    	logger, err := logging.NewLogger("ERROR", os.Stdout, "[KRAKEND]")
    	if err != nil {
    		log.Fatal("ERROR:", err.Error())
    	}
    
    	routerFactory := gin.DefaultFactory(proxy.DefaultFactory(logger), logger)
    
    	routerFactory.New().Run(serviceConfig)
    }
    

    Why are the response I am getting inconsistent? How is it possible and what am I doing wrong here?

    Thank you, I really value your time and I appreciate your help very much! 👍

    question good first issue locked 
    opened by adriendomoison 13
  • Support middleware on specific endpoint

    Support middleware on specific endpoint

    Considering authentication, some endpoint may require to be authenticated (account update is an exemple) and some other may be publicly available without authentication (reaching public profile information).

    To implement such a feature, several options are available including:

    • Add a global middleware matching itself the authenticated endpoint
    • Add an authentication middleware only on endpoints that require authentication

    In the first option, the middleware should have to perform operations similar to what the http router already does and thus would add some latency to each request.

    The second option has the advantage of adding the computation time at startup, when creating middlewares, and leaving middleware to do their goal: authentication (or any other middleware core logic), leaving the route selection to the router, already implemented

    opened by tjamet 12
  • Collection merging is not working.

    Collection merging is not working.

    Describe the bug Collection merging is not working. It returns one of collection.

    To Reproduce Steps to reproduce the behavior:

    1. Add two backends that return collections without mapping or grouping.
    2. Trigger endpoint.

    Expected behavior Returning two merged collections.

    Actual behavior Returns one of collection.

    Additional context Grouping or mapping working although.

    question locked 
    opened by apuzyrevsky 11
  • Krakend returns 500 status code with backend returning 201

    Krakend returns 500 status code with backend returning 201

    Describe the bug The krakend is returning the 500 status code with backend returning 201, i think that can be a timeout issue, but i didn't find in docs if there is a default timeout.

    locked 
    opened by riibeirogabriel 10
  • Help with defining krakend.json for a backend service

    Help with defining krakend.json for a backend service

    Hi,

    I've the following curl command to query a backend service. How can I define it in krakend.json?

    curl  -H "Authorization: xxxxxxxxxxxx"  -G  --data-urlencode 'query={"a":"b"}'  http://localhost:4000/api/v1/load
    

    Here's my krakend.json:

    {
      "version": 2,
      "extra_config": {},
      "timeout": "3000ms",
      "cache_ttl": "300s",
      "output_encoding": "json",
      "name": "test",
      "port": 8080,
      "endpoints": [
        {
          "endpoint": "/test",
          "method": "GET",
          "extra_config": {},
          "output_encoding": "json",
          "concurrent_calls": 1,
          "backend": [
            {
              "url_pattern": "/api/v1/load",
              "encoding": "json",
              "sd": "static",
              "extra_config": {},
              "method": "GET",
              "disable_host_sanitize": true,
              "host": [
                "localhost:4000"
              ]
            }
          ]
        }
      ]
    }
    
    locked 
    opened by devopsprosiva 10
  • How to prevent

    How to prevent "Transfer-Encoding: chunked"

    Hi, I am implementing an application where KrakenD is used as a gateway. I am trying to make it work to integrate with a .NET Web API service, but the POST body received at the service is empty. After thorough investigation I came to the conclusion that it is related to the way POST body is being forwarded by krakenD - krakenD sends it with "Transfer-Encoding: chunked", which .NET has problems with handling (could be a bug in .NET).

    I have tried different strategies to prevent krakenD from sending POST body with "Transfer-Encoding: chunked", which is e.g.:

    • forwarding all the headers
    • different combinations between "no-op" encoding and "json",
    • disabling compression.

    Do you know how we could achieve that krakenD sends the body not in a "chunked" way?

    My latest configuration file: configuration.zip

    Thanks in advance!

    locked 
    opened by madzia912 10
  • Backend does not receive any body

    Backend does not receive any body

    I'm running KrakenD 0.8.0 in a Kubernetes Cluster and trying to setup all endpoints. The problem i'm facing is that no body gets send to the backend. This is the config part of the krakend config file:

    {
          "method": "POST",
          "endpoint": "/jobs/",
          "output_encoding": "json",
          "concurrent_calls": 1,
          "headers_to_pass":[
            "Content-Type"
          ],
          "backend": [
            {
              "method": "POST",
              "url_pattern": "/jobs/",
              "encoding": "json",
              "disable_host_sanitize": false,
              "host": ["hostname.tld"]
            }
          ]
    }
    
    
    locked 
    opened by tspycher 10
  • examples folder missing?

    examples folder missing?

    Describe the bug The README under "Library Usage" states that there is an examples folder. However this seems to be missing.

    To Reproduce Steps to reproduce the behavior: Check for existence of examples folder

    Expected behavior examples folder exists or documentation is updated

    Logs NA

    Additional context NA.

    opened by shivahnshankar 0
  • strings.Title is deprecated in go 1.18

    strings.Title is deprecated in go 1.18

    Describe the bug

    The Go 1.18 release has deprecated strings.Title due to issues in how it handles Unicode. The recommended replacement for it is cases.Title.

    https://tip.golang.org/doc/go1.18#minor_library_changes

    opened by sumit-tembe 0
  • Regression introduced in Gin HandlerFactory by 9b38da3

    Regression introduced in Gin HandlerFactory by 9b38da3

    Describe the bug The error returned by an endpoint is not included into the Gin context anymore.

    To Reproduce Steps to reproduce the behavior:

    1. Configuration used: gin-based routing
    2. Steps to run the software: call an endpoint that fails (timeout, ...)

    Expected behavior We can access the error message in a custom Gin log formatter passed to NewEngine, from gin.LogFormatterParams.

    Logs N/A.

    Additional context

    It seems to be this line exactly that removes the error from the Gin context. It's really annoying when using a custom access log formatter that includes info about the request: we're supposed to have the reason of the failure in gin.LogFormatterParams.ErrorMessage to correlate it. Instead, we're forced to use the logging.Logger passed to CustomErrorEndpointHandler which does not include info about the request.

    opened by mzanibelli 0
  • Routing by header or request params more generally

    Routing by header or request params more generally

    Hi,

    I have been reading through the documentation for Krakend whilst evaluating it as part of a move from an existing API gateway. One of the things that we are looking to do is to route by header - the scenario looks something like as follows:

    The proxy checks if the header X-API-Version is supplied, which itself is of the format 2020-11-19, and routes according to the backend that has the most recent API version that existed before the defined date. If no header is supplied, the user details are queried and cached, and the X-API-Version header is populated with the creation date of the user's account.

    Is this something that is possible with Krakend? Routing by method appears to be well documented, but routing by header contents does not appear to be documented anywhere. The closest I have come to finding information on it is in #308 where it states:

    Routing by header is not a recommended strategy but can be implemented using a custom martian modifier or http client plugin
    

    However, further exploration does not provide any hints as to how one can influence which backend handles a request based on the header.

    Edit: I realise the above behaviour of logic comparisons may be a bit beyond the scope of Krakend. We do not have that many different versions, and these could be provided as a static map, so the above question could be distilled, if necessary, into: 'Is it possible to use a header value to map a request to a specific backend?'

    opened by sashahilton00 0
Releases(v2.0.5)
  • v2.0.5(Jun 8, 2022)

    What's Changed

    • Fix unused method receiver by @deepsource-autofix in https://github.com/luraproject/lura/pull/576
    • plugin tests fixed by @kpacha in https://github.com/luraproject/lura/pull/575
    • Replace use of HTTP codes with constants by @deepsource-autofix in https://github.com/luraproject/lura/pull/577
    • proxy: allow user to inject their own NewRequestBuilder Middleware by @sumit-tembe in https://github.com/luraproject/lura/pull/582
    • Change package path flatmap by @alombarte in https://github.com/luraproject/lura/pull/583
    • fix the response interface so plugins can replace the response by @kpacha in https://github.com/luraproject/lura/pull/589. A bug discovered and deeply detailed by @arcticShadow. Thanks for that!

    New Contributors

    • @sumit-tembe made their first contribution in https://github.com/luraproject/lura/pull/582
    • @arcticShadow made their first contribution in https://github.com/luraproject/lura/pull/589 helping to debug the issue

    Full Changelog: https://github.com/luraproject/lura/compare/v2.0.4...v2.0.5

    Source code(tar.gz)
    Source code(zip)
  • v2.0.4(May 3, 2022)

    What's Changed

    • Set the metadata in the response modifier wrapper using the new metho… by @taik0 in https://github.com/luraproject/lura/pull/574

    Full Changelog: https://github.com/luraproject/lura/compare/v2.0.3...v2.0.4

    Source code(tar.gz)
    Source code(zip)
  • v2.0.3(Apr 28, 2022)

  • v2.0.2(Apr 22, 2022)

    What's Changed

    • Function call can be replaced with helper function by @deepsource-autofix in https://github.com/luraproject/lura/pull/550
    • Fix unused method receiver by @deepsource-autofix in https://github.com/luraproject/lura/pull/551
    • Unused parameter should be replaced by underscore by @deepsource-autofix in https://github.com/luraproject/lura/pull/552
    • Deepsource quick fix by @kpacha in https://github.com/luraproject/lura/pull/558
    • Simplify slice expression to sliced value itself by @deepsource-autofix in https://github.com/luraproject/lura/pull/559
    • Remove unnecessary wrapping of function call by @deepsource-autofix in https://github.com/luraproject/lura/pull/560
    • Simplify make call by @deepsource-autofix in https://github.com/luraproject/lura/pull/561
    • Fix unnecessary typecasting on (*bytes.Buffer) by @deepsource-autofix in https://github.com/luraproject/lura/pull/564
    • Use plain channel send or receive by @deepsource-autofix in https://github.com/luraproject/lura/pull/563
    • Types of function parameters can be combined by @deepsource-autofix in https://github.com/luraproject/lura/pull/565
    • Code improvements by @kpacha in https://github.com/luraproject/lura/pull/568
    • param validator middleware by @kpacha in https://github.com/luraproject/lura/pull/571
    • always declare the endpoints sequentially by @kpacha in https://github.com/luraproject/lura/pull/572

    New Contributors

    • @deepsource-autofix made their first contribution in https://github.com/luraproject/lura/pull/550

    Full Changelog: https://github.com/luraproject/lura/compare/v2.0.1...v2.0.2

    Source code(tar.gz)
    Source code(zip)
  • v2.0.1(Mar 7, 2022)

    • remove default transport
    • omit flaky integration test
    • endpoint level flatmap filter required at multi backend endpoints
    • Update documentation example
    • typo fixed
    • query string and header params renamed
    • support injecting custom jwt claims into the backend url pattern #537
    • Add GlibcVersion variable to the core package to set the GLIBC used by CGO at build time so plugins can build using the same version
    • integration tests removed
    • health endpoint test removed
    • avoid race conditions when tracking the auto-options
    • init and normalize are two different stages
    • go version var added to the core pkg
    • async pkg added
    • async agents added
    • backoff pkg for basic strategies added
    • Create SECURITY.md
    • accept customized log formatters for the gin engine
    • plugins log improved
    • config version upgraded
    • documentation improved
    • minor refactor
    • deprecated functions removed
    • use the records with lowest prio and normalize the result by weight
    • log the mw setup when enabled
    • health endpoint can be disabled o renamed
    • gin upgraded to v1.7.7
    • return nil responses when all the partials failed
    • integration tests improved
    • return the pipe error as the response body if required by configuration
    • better multi error logging
    • avoid logging expected errors on closing
    • support for namespace alias added
    • ut fixed
    • avoid mixing variadic and constant arguments
    • avoid dumping the array of arguments
    • use the release mode by default
    • Remove references to old examples repository
    • Improve some endpoint related error messages
    • Include prefix in endpoint related errors
    • Add log prefix with context
    • load graphql queries from dedicated files
    • auto options endpoint added
    • moving the v2 pkg to the root level
    • v2 added to the github workflow
    • v2 added to the github workflow
    • return status code as error details
    • integration tests added to v2
    • binding and running logic splitted
    • better gin engine initialization and customization
    • support for customizing the gin router added
    • spdx header added
    • basic graphql support added
    • initial commit of the v2
    • check if the data field is null before adding the static values
    • The headers to pass map should contain the header names already canonicalized
    • race and regular tests splited in 2 stages
    • 404 responses do not include the incomplete header
    • tls 1.3 added
    • replacement of the initial request with a clone fixed
    • Update .gitignore
    • go report badge updated
    • merge completed
    • Merge branch 'master' into plugin_logger
    • logger added to the test plugin
    • test the plugins
    • avoid checking everytime if the logger is set
    • spdx header added
    • spdx headers added
    • logger register added to the req/resp modifier plugins
    • logger register added to the server plugins
    • client plugins can accept loggers
    • upgrade config to version 2
    • Fix typo
    Source code(tar.gz)
    Source code(zip)
  • v1.4.0(May 21, 2021)

    • pkg doc fixed
    • fix project references and copyright
    • krakend references removed
    • small fixes to the README
    • added a minor comment regarding the name being ignored
    • ignore the cfg name when extracting the hash
    • Remove some old resources from docs folder
    • Update references to KrakenD, pointing to Lura Project.
    • donating the repo to the Linux Foundation
    • spdx headers added
    • do not set a default noroute handler
    • fix type definition
    • minimal documentation added
    • json-collection added as a valid output encoding for collections
    • support for modifier plugins added
    • Fix typo in slides
    • Used make to create array instead of a slice when capacity is certainly known.
    Source code(tar.gz)
    Source code(zip)
  • v1.3.0(Feb 27, 2021)

    • [feature] create a group for the declared enpoints so mw are injected just to the group
    • [bugfix] Moved CI from Travis to Github Actions
    • [feature] envar GODEBUG=x509ignoreCN=0 added to the tests
    • [feature] build and test workflow added to the repo actions
    • [feature] avoid using the default logger from the stdlib
    • [feature] Use filepath.Join to avoid problems with paths with or without a trailing slash
    • [bugfix] Fix integration tests
    • [feature] Increase map size
    • [bugfix] Fix tests
    • [feature] pass original request host in X-Forwarded-Host header
    • [feature] flatmap del operation accepts more than one argument
    Source code(tar.gz)
    Source code(zip)
  • v1.2.0(Oct 5, 2020)

    • [feature] Flatmap (array manipulation) now also at endpoint level
    • [bugfix] Removed wording with racist connotations whitelist and blacklist. Now using allow and deny instead.
    • [feature] Upgrade go to 1.15.
    • [feature] support for the append operation added
    • [feature] Added a new supported encoding safejson
    • [feature] support for nested targets added
    • [bugfix] Client plugin example fixed
    • [bugfix] Do not copy nil readers on no-op
    • [bugfix] Nested sequential params
    • [feature] Clone also the request body in the CloneRequest method
    • [feature] /__debug/ endpoint accepts now any method
    • [feature] Use the weight of the SRV record to generate the list of hosts when resolving a service name
    • [feature] Decompress gzipped responses before parsing them
    • [feature] Added a /__health endpoint
    • [feature] Ability to use collections in sequential proxy as input (collection filters)
    • [feature] Support using JWT claims as backend url params
    • [feature] Mutual TLS between KrakenD and clients added
    Source code(tar.gz)
    Source code(zip)
  • 1.0.0(Sep 30, 2019)

    Summary of changes

    • [bugfix] http server plugin example fixed
    • [feature] go mod tidy
    • [upgrade] switch to go modules
    • [feature] godoc for http plugin packages added
    • [feature] Return all headers from proxy response
    • [feature] add to mux router funk clientIP
    • [upgrade] log improved
    • [bugfix] handler namespace fixed
    • [feature] http handler injection via plugin added
    • [feature] noop param extractor exported
    • [feature] code line of sight improved
    • [feature] use latest golang version 1.13.1
    • [feature] multilevel selection of parameters for sequential pipes
    • [feature] Push KrakenD User-Agent only if not overriding forwarded Header
    Source code(tar.gz)
    Source code(zip)
  • 0.9.0(Apr 7, 2019)

    Summary of changes

    • [bugfix] Abort a sequential merge after the first error or incomplete response
    • [bugfix] Propagate the content length when present
    • [bugfix] Avoid counting output params referred to a response value
    • [feature] Flatmap filter for collections
    • [other] Old go versions 1.8 and 1.9 are now discontinued
    • [performance] Load balancing URL creation improved
    • [performance] URL creation for sequential merger improved
    Source code(tar.gz)
    Source code(zip)
  • 0.8.0(Mar 9, 2019)

    Summary of changes

    • New routers Chi and httpmux
    • Mux based routers add multiple methods in same URL pattern
    • Integration tests
    • Mixing mandatory and optional query strings parameters is now possible
    • Error interface
    • Shadow backends / Traffic mirroring
    • Improved response headers
    • A fix in sequential backends
    • Mux based features now are like the Gin's ones
    • A detailed http executor
    Source code(tar.gz)
    Source code(zip)
  • 0.7.0(Dec 8, 2018)

    Summary of changes

    • Default configuration parser accepts now custom file readers. File formats accepted:
      • .json
      • .toml
      • .yaml
      • .yml
      • .properties
      • .props
      • .prop
      • .hcl
    • Replace the http package by transport. Getting ready for gRPC? (oops, spoiler)
    • Do not close a nil body (on the proxy layer)
    • Expose the HTTP server construction
    • Delay endpoint method sanitization until required
    • Sequential proxy merger
    • Add error details into the HTTP response
    • Bugfix: HTTP client timeout was set to 15 maximum seconds (no limit now)
    • Updated brand, new logo!

    Between 0.6 and 0.7, we have published an intermediate release 0.6.1 with:

    • Support for HTTP2/HTTPS
    • FIX: query string parameters works now with arrays and repeated variables (e.g.: ?a[]=a1&a[]=a2&b=b1&b=b2)
    • Added Go 1.11 to the build matrix
    Source code(tar.gz)
    Source code(zip)
  • 0.6.1(Oct 4, 2018)

    Summary of changes

    • NEW: Added support for HTTP2/HTTPS
    • FIX: query string parameters works now with arrays and repeated variables (e.g.: ?a[]=a1&a[]=a2&b=b1&b=b2)
    • UPDATE: Added Go 1.11 to the build matrix
    • Other minor fixes, compare to 0.6.0
    Source code(tar.gz)
    Source code(zip)
  • 0.6.0(Sep 7, 2018)

    Summary of changes

    • Decoupled the http server from the router packages
    • Add the Service name in the configuration
    • BUGFIX: Merge with incomplete responses
    Source code(tar.gz)
    Source code(zip)
  • 0.5(Jun 10, 2018)

    Summary of changes

    This release adds 89 commits to 0.4.2 and integrates the 0.5 Milestone. Most relevant items are:

    • Custom combiners for merging the backend responses.
    • Static partial responses middleware added
    • NoOp logger
    • Add an explicit header about response completeness
    • Avoid fancy strategies when balancing a FixedSubscriber with a single host
    • Expose all the http settings values
    • Request deep copy helpers added
    Source code(tar.gz)
    Source code(zip)
  • 0.4.2(Mar 9, 2018)

  • 0.4.1(Mar 2, 2018)

  • 0.4(Jan 20, 2018)

    Summary of changes

    • Removed ALL external dependencies and created independent components for each of them
    • Decoder and Service Discovery register system
    • Response metadata
    • Response decoder and formatter exposed
    • Custom status code handler
    • Minor handler optimizations
    • Config version upgraded
    • Server configuration extended (with timeouts and limits)
    • Now non strict REST urls are allowed
    • Request and response manipulation optimized
    • func version for all the interfaces
    Source code(tar.gz)
    Source code(zip)
  • 0.3(Sep 8, 2017)

    Summary of changes:

    • Support for go 1.9
    • etcd
    • Support for the latest versions of gin
    • Extended configuration for custom proxy factories
    • Improved support for custom transport layers for the backend communication
    • More data collected from the backend responses
    Source code(tar.gz)
    Source code(zip)
  • 0.2(May 26, 2017)

    Summary of changes:

    • Support for go 1.8
    • Improved injection for middlewares
    • Tons of new practical examples for middleware injection:
      • Gorilla
      • Mux
      • Negroni
      • Gin
      • JSON Web Token (JWT)
    • DNS SRV Service Discovery
    • Accept collections in the backend responses (as opposed to objects)
    • Router extended to allow injection of contexts
    • Integration tests on Travis
    • Custom HTTPClientFactory implementations
    • RSS decoder added
    Source code(tar.gz)
    Source code(zip)
Owner
Devops Faith - Open source for DevOps
KrakenD API Gateway, API2HTML, Go tools.
Devops Faith - Open source for DevOps
protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript clients that connect the web frontend and golang backend fronted by grpc-gateway.

protoc-gen-grpc-gateway-ts protoc-gen-grpc-gateway-ts is a Typescript client generator for the grpc-gateway project. It generates idiomatic Typescript

gRPC Ecosystem 69 Jun 26, 2022
HTTP Authentication middlewares

goji/httpauth httpauth currently provides HTTP Basic Authentication middleware for Go. It is compatible with Go's own net/http, goji, Gin & anything t

Goji 214 Jun 6, 2022
Useful HTTP middlewares

This project contains middlewares that I often found myself reimplementing in new projects. In addition, it includes a middleware that logs in a forma

Kevin Burke 19 Apr 16, 2022
Golang gRPC Middlewares: interceptor chaining, auth, logging, retries and more.

Golang gRPC Middlewares: interceptor chaining, auth, logging, retries and more.

gRPC Ecosystem 4.7k Jun 29, 2022
gin middlewares, just like nginx try-file function

Gin Middleware TryFile This project is to solve the problem that the gin framework processes the dynamic routing file in the front-end compilation fil

null 0 Jan 10, 2022
The Durudex gateway combines all durudex services so that it can be used through a single gateway.

The Durudex gateway combines all durudex services so that it can be used through a single gateway.

null 12 Jun 18, 2022
Ruuvi-go-gateway - Software replica of the Ruuvi Gateway

ruuvi-go-gateway ruuvi-go-gateway is a software that tries to replicate Ruuvi Ga

Scrin 6 May 10, 2022
Grpc-gateway-map-null - gRPC Gateway test using nullable values in map

Demonstrate gRPC gateway behavior with nullable values in maps Using grpc-gatewa

null 1 Jan 6, 2022
CrankDB is an ultra fast and very lightweight Key Value based Document Store.

CrankDB is a ultra fast, extreme lightweight Key Value based Document Store.

Shrey Batra 30 Apr 12, 2022
Ultra simple project scaffolding

?? You can help the author become a full-time open-source maintainer by sponsoring him on GitHub. aho ultra simple project scaffolding Install curl -f

EGOIST 62 Jun 6, 2022
Rpcx-framework - An RPC microservices framework based on rpcx, simple and easy to use, ultra fast and efficient, powerful, service discovery, service governance, service layering, version control, routing label registration.

RPCX Framework An RPC microservices framework based on rpcx. Features: simple and easy to use, ultra fast and efficient, powerful, service discovery,

ZYallers 1 Jan 5, 2022
BlobStore is a highly reliable,highly available and ultra-large scale distributed storage system

BlobStore Overview Documents Build BlobStore Deploy BlobStore Manage BlobStore License Overview BlobStore is a highly reliable,highly available and ul

CubeFS 14 Apr 28, 2022
A dead simple, highly performant, highly customizable sessions middleware for go http servers.

If you're interested in jwt's, see my jwt library! Sessions A dead simple, highly performant, highly customizable sessions service for go http servers

Adam Hanna 65 May 4, 2022
Composable, observable and performant config handling for Go for the distributed processing era

Konfig Composable, observable and performant config handling for Go. Written for larger distributed systems where you may have plenty of configuration

Lalamove 628 Jun 12, 2022
A collection of useful, performant, and threadsafe Go datastructures.

go-datastructures Go-datastructures is a collection of useful, performant, and threadsafe Go datastructures. NOTE: only tested with Go 1.3+. Augmented

Workiva 6.5k Jun 20, 2022
🐝 A Highly Performant and easy to use goroutine pool for Go

gohive Package gohive implements a simple and easy to use goroutine pool for Go Features Pool can be created with a specific size as per the requireme

Lovelesh 36 Jun 21, 2022
Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications

Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.

HashiCorp 11.2k Jun 22, 2022
Simple and performant ORM for sql.DB

Simple and performant ORM for sql.DB Main features are: Works with PostgreSQL, MySQL, SQLite. Selecting into a map, struct, slice of maps/structs/vars

Uptrace 1.1k Jun 23, 2022
Mildly performant age public key bruteforcer

vanity-age Mildly performant age vanity public key brute-forcer.

Mars 9 Aug 30, 2021
A highly-performant command runtime server written in Go for Flame 🔥

Flame Command Runtime A highly-performant command runtime server written in Go for Flame ?? Flame Command Runtime(FCR) is powerful, highly customizabl

Flame 3 Aug 29, 2021
Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality management.

Optimus Optimus is an easy-to-use, reliable, and performant workflow orchestrator for data transformation, data modeling, pipelines, and data quality

Open Data Platform 620 Jun 20, 2022
Sand is the next, versatile, high-level compiled or interpreted language that's easy to learn and performant to run.

Sand is the newest, dynamically typed, interpreted programming language. Table of Contents History Project Stats History Sand was created as part of @

Neuron AI 4 Mar 13, 2022
Go-p9p - A modern, performant 9P library for Go.

p9p A modern, performant 9P library for Go. For information on usage, please see the GoDoc. Refer to 9P's documentation for more details on the protoc

null 0 Jan 4, 2022
VMAGI - Write an performant emulator in 24 hours!

VMAGI Welcome! VMAGI is a small emulator/interpreter my friend Matthew and I challenged each other to build in 24 hours. This includes both the implem

Sandy 3 Feb 27, 2022
Marshmallow provides a flexible and performant JSON unmarshalling in Go. It specializes in dealing with unstructured struct - when some fields are known and some aren't, with zero performance overhead nor extra coding needed.

Marshmallow Marshmallow package provides a simple API to perform flexible and performant JSON unmarshalling in Go. Marshmallow specializes in dealing

PerimeterX 14 Jun 23, 2022