Fast and light-weight API proxy firewall for request and response validation by OpenAPI specs.

Overview

Open Source API Firewall

API Firewall is a high-performance proxy with API request and response validation based on OpenAPI/Swagger schema. It is designed to protect REST API endpoints in cloud-native environments. API Firewall provides API hardening with the use of a positive security model allowing calls that match a predefined API specification for requests and responses, while rejecting everything else.

The key features of API Firewall are:

  • Secure REST API endpoints by blocking malicious requests
  • Stop API data breaches by blocking malformed API responses
  • Discover Shadow API endpoints
  • Validate JWT access tokens for OAuth 2.0 protocol-based authentication
  • (NEW) Denylist compromised API tokens, keys, and Cookies

The product is open source, available at DockerHub and already got 1 billion (!!!) pulls. To support this project, you can star the repository.

Use cases

Running in blocking mode

  • Block malicious requests that do not match the OpenAPI 3.0 specification
  • Block malformed API responses to stop data breaches and sensitive information exposure

Running in monitoring mode

  • Discover Shadow APIs and undocumented API endpoints
  • Log malformed requests and responses that do not match the OpenAPI 3.0 specification

API schema validation and positive security model

When starting API Firewall, you should provide the OpenAPI 3.0 specification of the application to be protected with API Firewall. The started API Firewall will operate as a reverse proxy and validate whether requests and responses match the schema defined in the specification.

The traffic that does not match the schema will be logged using the STDOUT and STDERR Docker services or blocked (depending on the configured API Firewall operation mode). When operating in the logging mode, API Firewall also logs so-called shadow API endpoints, those that are not covered in API specification but respond to requests (except for endpoints returning the code 404).

API Firewall scheme

OpenAPI 3.0 specification is supported and should be provided as a YAML or JSON file (.yaml, .yml, .json file extensions).

By allowing you to set the traffic requirements with the OpenAPI 3.0 specification, API Firewall relies on a positive security model.

Technical data

API Firewall works as a reverse proxy with a built-in OpenAPI 3.0 request and response validator. It's written in Golang and using fasthttp proxy. The project is optimized for extreme performance and near-zero added latency.

Starting API Firewall

To download, install, and start API Firewall on Docker, see the instructions.

Demos

You can try API Firewall by running the demo environment that deploys an example application protected with API Firewall. There are two available demo environments:

Wallarm's blog articles related to API Firewall

Performance

When creating API Firewall, we prioritized speed and efficiency to ensure that our customers would have the fastest APIs possible. Our latest tests demonstrate that the average time required for API Firewall to process one request is 1.339 ms which is 66% faster than Nginx:

API Firewall 0.6.2 with JSON validation

$ ab -c 200 -n 10000 -p ./large.json -T application/json http://127.0.0.1:8282/test/signup

Requests per second:    13005.81 [#/sec] (mean)
Time per request:       15.378 [ms] (mean)
Time per request:       0.077 [ms] (mean, across all concurrent requests)

NGINX 1.18.0 without JSON validation

$ ab -c 200 -n 10000 -p ./large.json -T application/json http://127.0.0.1/test/signup

Requests per second:    7887.76 [#/sec] (mean)
Time per request:       25.356 [ms] (mean)
Time per request:       0.127 [ms] (mean, across all concurrent requests)

These performance results are not the only ones we have got during API Firewall testing. Other results along with the methods used to improve API Firewall performance are described in this Wallarm's blog article.

Comments
  • Question about settings

    Question about settings

    Hello, team Wallarm. I have studied your documentation, i have a question. Tell me if it is possible to change the http response code in the blocking mode through variables. By default, as I understand it, it is 403. But sometimes it is more convenient to be able to set your own response code, for accurate diagnostics by logs, or for introducing a custom page when blocking the api by a firewall.

    opened by shrv 6
  • Feature Suggestion - x509 key pair authentication

    Feature Suggestion - x509 key pair authentication

    This application is a really nice tool and I can imagine numerous applications for this model. What I would like to suggest is an additional module that would allow for zero trust via x509 key pair. This would allow users to deploy unconfigured versions on their applications and the said application could just auth with a key pair, request what it needs to request for an initial configuration download.

    Private key verification could be integrated into cloud-based key storage like Google KMS, an intermediate CA, or even stored locally on the host machine's file system or k8s secrets.

    opened by kbrewer92 4
  • Bypass Upstream Certificate Checking

    Bypass Upstream Certificate Checking

    Is there a configuration parameter to bypass the certificate validity checking for upstream / proxied APIs?

    This is the error I am getting {"level":"error","msg":"#0000000100000001 : error while proxying request: x509: certificate is valid for abc.xyz.net not abc","time":"2021-11-17T10:44:07Z"}

    This would be helpful for self signed certificates and networks that are not directly internet accessible.

    opened by swarmee 4
  • OpenAPI Base Path

    OpenAPI Base Path

    Hi, in OpenAPI 2 there was a top-level property called basePath which (if set) defined the base path before all API endpoints, like /api/v1. In OpenAPI 3 this was removed, but the Swagger OpenAPI 3.0 docs and the spec itself suggest putting the base path in the server URL like this:

    servers:
    - url: "https://api.my-service.com/api/v1"
    - url: "https://staging-api.my-service.com/api/v1"
    

    Without support for a base path, all of my API endpoints are rejected because they are missing the /api/v1 portion (my base path).

    Four ways to solve this problem come to mind:

    1. Change all my endpoint paths to include the base path (this doesn't seem ideal since the docs suggest I use a base path in the server URL).
    2. Run my OpenAPI spec through a preprocessor that prepends the base path to each endpoint before you read it.
    3. You can infer the base path from the servers by capturing the path of the first server URL (if present). This is what the AWS API Gateway does.
    4. You can provide an environment variable / CLI argument to set the base path like APIFW_API_BASE_PATH.

    I would be happy to contribute this enhancement but I don't see the source code anywhere. Also, you might want to clarify the license of this product.

    opened by kamermans 4
  • File upload api being blocked

    File upload api being blocked

    File upload api is being blocked when it should be allowed. Can you please suggest what can be changed in order to allow this upload? The spec has it defined as { "/document/upload": { "post": { "tags": [ "document-controller" ], "operationId": "Document_1", "parameters": [ { "name": "id", "in": "query", "required": true, "schema": { "type": "string" } } ], "requestBody": { "content": { "multipart/form-data": { "schema": { "required": [ "file" ], "type": "object", "properties": { "file": { "type": "string", "format": "binary" } } } } } }, "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "type": "string" } } } } } } } }

    The error when uploading a docx is api-firewall | time=2021-12-04T17:32:18Z level=error msg=#0000000900000001 : request validation error: request body has an error: failed to decode request body: path file: unsupported content type "application/vnd.openxmlformats-officedocument.wordprocessingml.document"

    The content type passed in request header is Content-Type multipart/form-data; boundary=---------------------------162567064334244515552744535554 Request itself is `-----------------------------162567064334244515552744535554 Content-Disposition: form-data; name="file"; filename="mydoc.docx" Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document

    TEST-doc

    -----------------------------162567064334244515552744535554-- `

    opened by mbhagdev 3
  • Enforce OAuth Access Token Scope requirements

    Enforce OAuth Access Token Scope requirements

    Ideally, when I design an endpoint, I should be able to say how each method is authorized. In OAuth, we use scopes to determine the extent of access granted for an access_token ("AT"). The AT may be a value token: a JWT requiring signature validation (and auto-fetching of the current keys). The AT may also be a reference token: a guess-resistent identifier requiring OAuth token introspection. In either case, the AT should contain the scope claim, which is a space-delimited list of scopes (normally in URI format).

    In the example below, the swagger doc is saying that to do a GET on the persistence endpoint, you need to present a token with either the properties.read OR properties.write scope

    /jans-config-api/api/v1/jans-auth-server/config/persistence:
        get:
          summary: Returns persistence type configured for Jans authorization server.
          description: Returns persistence type configured for Jans authorization server.
          operationId: get-properties-persistence
          security:
            - oauth2: [properties.read, properties.write]
    

    For extra credit, you can consider how the scopes are combined. Can you use AND instead of OR for a given endpoint? For more complex booleans, you could support JSONLogic syntax.

    opened by nynymike 3
  • Integer field in JSON request body is validated as null instead of its value

    Integer field in JSON request body is validated as null instead of its value

    When configuring request body schema with an integer type field and sending a valid request, it produces a validation error with the value considered as null.

    Minimal example built by adjusting the docker-compose demo in this repo: int-example.json instead of httpbin.json:

    {
      "openapi": "3.0.1",
      "info": {
        "title": "Minimal integer field example",
        "version": "0.0.1"
      },
      "paths": {
        "/integer-field": {
          "post": {
            "requestBody": {
              "content": {
                "application/json": {
                  "schema": {
                    "type": "object",
                    "properties": {
                      "count": {
                        "type": "integer"
                      }
                    }
                  }
                }
              }
            },
            "responses": {
              "200": {
                "description": "Success"
              }
            }
          }
        }
      }
    }
    

    Add these env vars to the original docker-compose demo:

    APIFW_API_SPECS: "/opt/resources/int-example.json"
    APIFW_LOG_LEVEL: "DEBUG"
    APIFW_ADD_VALIDATION_STATUS_HEADER: "true"
    

    Request (curl):

    curl -X POST http://localhost:8080/integer-field \
       -H 'Content-Type: application/json' \
       -d '{"count":1}'
    

    Error in log:

    time=2022-12-12T11:51:11Z level=error msg=request validation error error=request body has an error: doesn't match the schema: Error at "/count": Value is not nullable
    Schema:
      {
        "type": "integer"
      }
    
    Value:
      null
     request_id=#0000000400000001
    
    opened by yonatane 1
  • Published 386 musl binary appears to be amd64

    Published 386 musl binary appears to be amd64

    Hi! Building https://github.com/wallarm/api-firewall-docker/blob/5cd07f115f16d10095a385a73568a4e60caf278b/0.6.7/Dockerfile on i386 (docker build --platform linux/386) fails with /bin/sh: api-firewall: not found.

    When downloading https://github.com/wallarm/api-firewall/releases/download/v0.6.7/api-firewall-386-musl.tar.gz in an amd64 container instead, I can successfully ldd (and run!) the binary:

    $ docker run -it --rm alpine:3.15
    / # wget https://github.com/wallarm/api-firewall/releases/download/v0.6.7/api-firewall-386-musl.tar.gz
    Connecting to github.com (192.30.255.113:443)
    Connecting to objects.githubusercontent.com (185.199.111.133:443)
    saving to 'api-firewall-386-musl.tar.gz'
    api-firewall-386-mus 100% |********************************| 3589k  0:00:00 ETA
    'api-firewall-386-musl.tar.gz' saved
    / # tar -xvf api-firewall-386-musl.tar.gz
    ./
    ./api-firewall
    ./LICENSE
    / # ldd api-firewall
    	/lib/ld-musl-x86_64.so.1 (0x7f2461ac7000)
    	libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f2461ac7000)
    / # ./api-firewall -v
    Version: 0.6.7
    Wallarm API-Firewall
    

    I'm not 100% certain I'm looking in the right place, but it seems like https://github.com/wallarm/api-firewall/blob/9a2d7f2bd3c9c68360c54e01cb3c749a6ee11e7b/.github/workflows/binaries.yml#L156 is maybe missing the same case block to set GOARCH=386 as build-x86: has? https://github.com/wallarm/api-firewall/blob/9a2d7f2bd3c9c68360c54e01cb3c749a6ee11e7b/.github/workflows/binaries.yml#L95-L102

    Honestly, it should be pretty safe to just set GOARCH unilaterally -- if you're on amd64 and explicitly set GOARCH=amd64, it doesn't do anything (since that's the default detected value for that case). :sweat_smile:

    (Also, setting GOHOSTARCH and GOHOSTOS explicitly like that shouldn't be necessary.)

    opened by tianon 1
  • nginx configuration for tests

    nginx configuration for tests

    Does fasthttp accidentally keep connected connections, if so, it would be more appropriate to use this configuration:

    worker_processes 8; # equal to number of CPU cores on our machine
    ...
    upstream backend {
        server 127.0.0.1:9090;
        keepalive 8;
    }
    ...
    client_body_buffer_size 128K; # greater than our JSON file size
    ... 
           location / {
                    proxy_pass http://backend;
                    proxy_http_version 1.1;
                    proxy_set_header Host $host;
                    proxy_set_header X-Real-IP $remote_addr;
                    proxy_set_header Connection "";
            }
    
    opened by tomglaza 1
  • OpenAPI Spec loading via URL

    OpenAPI Spec loading via URL

    Updates:

    • Add support of the OpenAPI spec loading via URL. From now on it's possible to set openapi specification URL to env var APIFW_API_SPECS.
    • Add custom Content-Type of the request to introspection endpoint. Custom content-type could be set to the env var APIFW_SERVER_OAUTH_INTROSPECTION_CONTENT_TYPE.
    opened by afr1ka 0
  • move method next to msg id to keep same as the response log

    move method next to msg id to keep same as the response log

    response sequence for log

                        logger.Infof("(%d) : #%016X : %s %s -> %s (%s)",
                                ctx.Response.StatusCode(),
                                ctx.ID(),
                                ctx.Request.Header.Method(), ctx.Path(),
                                ctx.RemoteAddr(), time.Since(start),
                        )
    
    opened by zoulinz 0
Releases(v0.6.10)
Owner
Wallarm
Wallarm delivers automated cloud-native application and API security throughout application development and deployment lifecycle
Wallarm
Openapi - OpenAPI 3.x parser. Written in Go

OpenAPI OpenAPI specification object model Features Easy to integrate. Installat

null 10 Nov 23, 2022
Add request id to a request's context

RequestID ?? This is a very simple piece of middleware for adding request/correlation IDs to the context of a http request. By default, this module wi

Jamie Aitken 0 Dec 4, 2021
OpenAPI for Todolist RESTful API

Go API client for swagger OpenAPI for Todolist RESTful API Overview This API cli

Iko Afianando 1 Dec 18, 2021
Light JSON API for storing user ratings of NASA's Astronomy Picture of the Day (APOD).

nasa-apod-api-go Light JSON API for storing user ratings of NASA's Astronomy Picture of the Day (APOD). To run this server you must have access to a N

null 1 Oct 26, 2021
OpenAPI specification and related artifacts for HashiCorp Nomad

Overview This repository contains the HashiCorp Nomad OpenAPI specification and related artifacts. The OpenAPI specification defines a machine-readabl

HashiCorp 48 Dec 14, 2022
Create a new OpenAPI based blink-plugin

OpenAPI plugin template Create a new OpenAPI based blink-plugin. READ FIRST Getting the Repo Option 1: From Github Press Use this template Naming conv

Blink 0 Oct 14, 2021
A fully-featured OpenAPI generated HubSpot client for Go

This is a HubSpot Go client generated using openapi-generator. It includes packages for every OpenAPI spec listed in HubSpot's API directory. Installi

Clark McCauley 17 Dec 31, 2022
An experimental OpenAPI -> Terraform Provider generator that does not yet function

tfpgen An experimental OpenAPI -> Terraform Provider generator that does not yet function. The goal is to allow developers to incrementally generate a

Brandon Croft 0 Feb 19, 2022
Serverless SOAR (Security Orchestration, Automation and Response) framework for automatic inspection and evaluation of security alert

DeepAlert DeepAlert is a serverless framework for automatic response of security alert. Overview DeepAlert receives a security alert that is event of

null 37 Jan 3, 2023
A YANG-centric Go toolkit - Go/Protobuf Code Generation; Validation; Marshaling/Unmarshaling

Introduction ygot (YANG Go Tools) is a collection of Go utilities that can be used to: Generate a set of Go structures and enumerated values for a set

OpenConfig 230 Jan 8, 2023
lambda-go-api-proxy makes it easy to port APIs written with Go frameworks such as Gin to AWS Lambda and Amazon API Gateway.

aws-lambda-go-api-proxy makes it easy to run Golang APIs written with frameworks such as Gin with AWS Lambda and Amazon API Gateway.

Amazon Web Services - Labs 755 Jan 6, 2023
A cloud native Identity & Access Proxy / API (IAP) and Access Control Decision API

Heimdall Heimdall is inspired by Ory's OAthkeeper, tries however to resolve the functional limitations of that product by also building on a more mode

Dimitrij Drus 19 Jan 6, 2023
Simple-Weather-API - Simple weather api app created using golang and Open Weather API key

Simple Weather API Simple weather api app created using golang and Open Weather

Siva Prakash 3 Feb 6, 2022
A small, fast, reliable pastemyst API wrapper written in Golang

A small, fast, reliable pastemyst API wrapper written in Golang. Official pastemyst API docs found here.

null 10 Dec 12, 2022
A API scanner written in GOLANG to scan files recursively and look for API keys and IDs.

GO FIND APIS _____ ____ ______ _____ _ _ _____ _____ _____ _____ / ____|/ __ \ | ____|_ _| \ | | __ \ /\ | __ \_

Sreekanth Sasi 3 Oct 25, 2021
💾 Wolke API is the API behind Wolke image storage and processing aswell as user management

?? Wolke API Wolke API is the API behind Wolke image storage and processing aswell as user management Deploying To deploy Wolke Bot you'll need podman

wolke.casa 1 Dec 21, 2021
Simple and fast webp library for golang

go-webp Golang Webp library for encoding and decoding, using C binding for Google libwebp Requirements libwebp Benchmarks % go test -bench "^Benchmark

Kolesa Group 101 Dec 28, 2022
Go library for accessing the MyAnimeList API: http://myanimelist.net/modules.php?go=api

go-myanimelist go-myanimelist is a Go client library for accessing the MyAnimeList API. Project Status The MyAnimeList API has been stable for years a

Stratos Neiros 33 Sep 28, 2022