A tiny http middleware for Golang with added handlers for common needs.

Overview

LICENSE Golang Godocs Go Report Card Travis Build Status codecov

rye

A simple library to support http services. Currently, rye provides a middleware handler which can be used to chain http handlers together while providing statsd metrics for use with DataDog or other logging aggregators. In addition, rye comes with various pre-built middleware handlers for enabling functionality such as CORS and rate/CIDR limiting.

Setup

In order to use rye, you should vendor it and the statsd client within your project.

govendor fetch github.com/InVisionApp/rye
govendor fetch github.com/cactus/go-statsd-client/statsd

Why another middleware lib?

  • rye is tiny - the core lib is ~143 lines of code (including comments)!
  • Each middleware gets statsd metrics tracking for free including an overall error counter
  • We wanted to have an easy way to say “run these two middlewares on this endpoint, but only one middleware on this endpoint”
    • Of course, this is doable with negroni and gorilla-mux, but you’d have to use a subrouter with gorilla, which tends to end up in more code
  • Bundled helper methods for standardising JSON response messages
  • Unified way for handlers and middlewares to return more detailed responses via the rye.Response struct (if they chose to do so).
  • Pre-built middlewares for things like CORS support

Example

You can run an example locally to give it a try. The code for the example is here!

cd example
go run rye_example.go

Writing custom middleware handlers

Begin by importing the required libraries:

import (
    "github.com/cactus/go-statsd-client/statsd"
    "github.com/InVisionApp/rye"
)

Create a statsd client (if desired) and create a rye Config in order to pass in optional dependencies:

config := &rye.Config{
    Statter:  statsdClient,
    StatRate: DEFAULT_STATSD_RATE,
}

Create a middleware handler. The purpose of the Handler is to keep Config and to provide an interface for chaining http handlers.

middlewareHandler := rye.NewMWHandler(config)

Set up any global handlers by using the Use() method. Global handlers get pre-pended to the list of your handlers for EVERY endpoint. They are bound to the MWHandler struct. Therefore, you could set up multiple MWHandler structs if you want to have different collections of global handlers.

middlewareHandler.Use(middleware_routelogger)

Build your http handlers using the Handler type from the rye package.

type Handler func(w http.ResponseWriter, r *http.Request) *rye.Response

Here are some example (custom) handlers:

func homeHandler(rw http.ResponseWriter, r *http.Request) *rye.Response {
    fmt.Fprint(rw, "Refer to README.md for auth-api API usage")
    return nil
}

func middlewareFirstHandler(rw http.ResponseWriter, r *http.Request) *rye.Response {
    fmt.Fprint(rw, "This handler fires first.")
    return nil
}

func errorHandler(rw http.ResponseWriter, r *http.Request) *rye.Response {
    return &rye.Response{
        StatusCode: http.StatusInternalServerError,
        Err:        errors.New(message),
    }
}

Finally, to setup your handlers in your API (Example shown using Gorilla):

routes := mux.NewRouter().StrictSlash(true)

routes.Handle("/", middlewareHandler.Handle([]rye.Handler{
    a.middlewareFirstHandler,
    a.homeHandler,
})).Methods("GET")

log.Infof("API server listening on %v", ListenAddress)

srv := &http.Server{
    Addr:    ListenAddress,
    Handler: routes,
}

srv.ListenAndServe()

Statsd Generated by Rye

Rye comes with built-in configurable statsd statistics that you could record to your favorite monitoring system. To configure that, you'll need to set up a Statter based on the github.com/cactus/go-statsd-client and set it in your instantiation of MWHandler through the rye.Config.

When a middleware is called, it's timing is recorded and a counter is recorded associated directly with the http status code returned during the call. Additionally, an errors counter is also sent to the statter which allows you to count any errors that occur with a code equaling or above 500.

Example: If you have a middleware handler you've created with a method named loginHandler, successful calls to that will be recorded to handlers.loginHandler.2xx. Additionally you'll receive stats such as handlers.loginHandler.400 or handlers.loginHandler.500. You also will receive an increase in the errors count.

If you're sending your logs into a system such as DataDog, be aware that your stats from Rye can have prefixes such as statsd.my-service.my-k8s-cluster.handlers.loginHandler.2xx or even statsd.my-service.my-k8s-cluster.errors. Just keep in mind your stats could end up in the destination sink system with prefixes.

Using with Golang 1.7 Context

With Golang 1.7, a new feature has been added that supports a request specific context. This is a great feature that Rye supports out-of-the-box. The tricky part of this is how the context is modified on the request. In Golang, the Context is always available on a Request through http.Request.Context(). Great! However, if you want to add key/value pairs to the context, you will have to add the context to the request before it gets passed to the next Middleware. To support this, the rye.Response has a property called Context. This property takes a properly created context (pulled from the request.Context() function. When you return a rye.Response which has Context, the rye library will craft a new Request and make sure that the next middleware receives that request.

Here's the details of creating a middleware with a proper Context. You must first pull from the current request Context. In the example below, you see ctx := r.Context(). That pulls the current context. Then, you create a NEW context with your additional context key/value. Finally, you return &rye.Response{Context:ctx}

func addContextVar(rw http.ResponseWriter, r *http.Request) *rye.Response {
    // Retrieve the request's context
    ctx := r.Context()

    // Create a NEW context
    ctx = context.WithValue(ctx,"CONTEXT_KEY","my context value")

    // Return that in the Rye response 
    // Rye will add it to the Request to 
    // pass to the next middleware
    return &rye.Response{Context:ctx}
}

Now in a later middleware, you can easily retrieve the value you set!

func getContextVar(rw http.ResponseWriter, r *http.Request) *rye.Response {
    // Retrieving the value is easy!
    myVal := r.Context().Value("CONTEXT_KEY")

    // Log it to the server log?
    log.Infof("Context Value: %v", myVal)

    return nil
}

For another simple example, look in the JWT middleware - it adds the JWT into the context for use by other middlewares. It uses the CONTEXT_JWT key to push the JWT token into the Context.

Using built-in middleware handlers

Rye comes with various pre-built middleware handlers. Pre-built middlewares source (and docs) can be found in the package dir following the pattern middleware_*.go.

To use them, specify the constructor of the middleware as one of the middleware handlers when you define your routes:

// example
routes.Handle("/", middlewareHandler.Handle([]rye.Handler{
    rye.MiddlewareCORS(), // to use the CORS middleware (with defaults)
    a.homeHandler,
})).Methods("GET")

OR 

routes.Handle("/", middlewareHandler.Handle([]rye.Handler{
    rye.NewMiddlewareCORS("*", "GET, POST", "X-Access-Token"), // to use specific config when instantiating the middleware handler
    a.homeHandler,
})).Methods("GET")

Serving Static Files

Rye has the ability to add serving static files in the chain. Two handlers have been provided: StaticFilesystem and StaticFile. These middlewares should always be used at the end of the chain. Their configuration is simply based on an absolute path on the server and possibly a skipped path prefix.

The use case here could be a powerful one. Rye allows you to serve a filesystem just as a whole or a single file. Used together you could facilitate an application which does both -> fulfilling the capability to provide a single page application. For example, if you had a webpack application which served static resources and artifacts, you would use the StaticFilesystem to serve those. Then you'd use StaticFile to serve the single page which refers to the single-page application through index.html.

A full sample is provided in the static-examples folder. Here's a snippet from the example using Gorilla:

    pwd, err := os.Getwd()
    if err != nil {
        log.Fatalf("NewStaticFile: Could not get working directory.")
    }

    routes.PathPrefix("/dist/").Handler(middlewareHandler.Handle([]rye.Handler{
        rye.MiddlewareRouteLogger(),
        rye.NewStaticFilesystem(pwd+"/dist/", "/dist/"),
    }))

    routes.PathPrefix("/ui/").Handler(middlewareHandler.Handle([]rye.Handler{
        rye.MiddlewareRouteLogger(),
        rye.NewStaticFile(pwd + "/dist/index.html"),
    }))

Middleware list

Name Description
Access Token Provide Access Token validation
CIDR Provide request IP whitelisting
CORS Provide CORS functionality for routes
Auth Provide Authorization header validation (basic auth, JWT)
Route Logger Provide basic logging for a specific route
Static File Provides serving a single file
Static Filesystem Provides serving a single file

A Note on the JWT Middleware

The JWT Middleware pushes the JWT token onto the Context for use by other middlewares in the chain. This is a convenience that allows any part of your middleware chain quick access to the JWT. Example usage might include a middleware that needs access to your user id or email address stored in the JWT. To access this Context variable, the code is very simple:

func getJWTfromContext(rw http.ResponseWriter, r *http.Request) *rye.Response {
    // Retrieving the value is easy!
    // Just reference the rye.CONTEXT_JWT const as a key
    myVal := r.Context().Value(rye.CONTEXT_JWT)

    // Log it to the server log?
    log.Infof("Context Value: %v", myVal)

    return nil
}

API

Config

This struct is configuration for the MWHandler. It holds references and config to dependencies such as the statsdClient.

type Config struct {
    Statter  statsd.Statter
    StatRate float32
}

MWHandler

This struct is the primary handler container. It holds references to the statsd client.

type MWHandler struct {
    Config Config
}

Constructor

func NewMWHandler(statter statsd.Statter, statrate float32) *MWHandler

Use

This method prepends a global handler for every Handle method you call. Use this multiple times to setup global handlers for every endpoint. Call Use() for each global handler before setting up additional routes.

func (m *MWHandler) Use(handlers Handler)

Handle

This method chains middleware handlers in order and returns a complete http.Handler.

func (m *MWHandler) Handle(handlers []Handler) http.Handler

rye.Response

This struct is utilized by middlewares as a way to share state; ie. a middleware can return a *rye.Response as a way to indicate that further middleware execution should stop (without an error) or return a hard error by setting Err + StatusCode or add to the request Context by returning a non-nil Context.

type Response struct {
    Err           error
    StatusCode    int
    StopExecution bool
    Context       context.Context
}

Handler

This type is used to define an http handler that can be chained using the MWHandler.Handle method. The rye.Response is from the rye package and has facilities to emit StatusCode, bubble up errors and/or stop further middleware execution chain.

type Handler func(w http.ResponseWriter, r *http.Request) *rye.Response

Test stuff

All interfacing with the project is done via make. Targets exist for all primary tasks such as:

  • Testing: make test or make testv (for verbosity)
  • Generate: make generate - this generates based on vendored libraries (from $GOPATH)
  • All (test, build): make all
  • .. and a few others. Run make help to see all available targets.
  • You can also test the project in Docker (and Codeship) by running jet steps

Contributing

Fork the repository, write a PR and we'll consider it!

Special Thanks

Thanks go out to Justin Reyna (InVisionApp.com) for the awesome logo!

Comments
  • adding CORS support in rye

    adding CORS support in rye

    • CORS support is necessary if we expect browser JS to communicate with auth-api (such as the case with Labs working on Surface)
    • Tests updated
    • This also adds support for preflight requests (OPTIONS) that will bypass middleware execution
    • Side-effect: having this functionality allows folks to NOT have to use something like gorilla/handlers for CORS support
    • updated readme

    @jescochu @caledhwa @talpert

    fullstop:rye dselans$ make test
    go test -cover -tags integration github.com/InVisionApp/rye
    ok      github.com/InVisionApp/rye  0.012s  coverage: 100.0% of statements
    
    opened by dselans 4
  • Adding custom statter option for clients to optionally use

    Adding custom statter option for clients to optionally use

    The custom statter option will allow the client to receive all the metrics that Rye calculates around a request. The custom statter must contain a ReportStats method which will then receive information around the request, response, as well as the duration of the request handler. This is ideal for clients that would prefer to use the datadog-go library or any other library. Additionally, this is a potential option to move the Rye middleware away from any kind of statsD reporting.

    opened by bstanley0811 3
  • [WIP] Add OpenTracing support

    [WIP] Add OpenTracing support

    [WIP] This PR adds optional OpenTracing support to Rye. It follows a similar pattern to what was done for for statsd support. By default it will name the span using the request's method and URI (ie. GET /foo/bar).

    Using Middlewares for OpenTracing was discarded because there isn't a way to enforce an "after Handler" to finalize the span.

    opened by jfernandez 3
  • .Use duplicates its middleware each time a route is called

    .Use duplicates its middleware each time a route is called

    When .Use is specified with middleware, the middleware ends up being invoked X times for X amount of request to a particular route.

    Example:

    /healthcheck receives first request: .Use specified middleware runs 1 time. /healthcheck receives second request: .Use specified middleware runs 2 times. /healthcheck receives n request: .Use specified middleware runs n times.

    opened by joshmatz 2
  • Fix handle closure

    Fix handle closure

    So, with the addition of the Use() functionality, a bug was discovered where the closure around handlers was being frozen at the last execution. The issue is about when the Handle function is called and initialized.

    The issue is described here (in number 5 in the blog post): http://keshavabharadwaj.com/2016/03/31/closure_golang/

    After a few different approaches, the only solution here seems to be to add the handlers in the function. Basically, since these are function pointers, this is a very small and quick operation that shouldn't have material impact on performance. However, if performance is impacted and noticeable, we can take a different approach potentially regarding the adding of global handlers.

    The added test here should execute before handlers and multiple Handles should manage their closure correctly -> Confirms the problem. When run with the old code, both executions of h and h2 end in only the success2handler() being called for both (so a total of twice). YUCK. With the

    handlers = append(m.beforeHandlers, handlers...)
    

    being INSIDE the return of the anonymous function, both successHandler() and success2handler() are called correctly.

    opened by caledhwa 2
  • Add context

    Add context

    This PR adds Golang Context support.

    • Context added to rye.Response to support adding to requests.
    • Examples updated to include samples
    • Godoc updated
    • README updated with straightforward examples of how to use
    • JWT middleware now adds a context with the token under CONTEXT_JWT key.
    • So easy to use.
    • Still 100%

    Enjoy!

    opened by caledhwa 2
  • Handle StatusCode with StopExecution

    Handle StatusCode with StopExecution

    This allows a rye.Response of this form to increment the correct counters in statsd:

    &rye.Response{
      StopExecution: true,
      StatusCode: 404,
    }
    

    The status code will be reported to statsd as 404 whereas previously it would report as 2xx.

    opened by mheisig 1
  • Prepended Global Handlers

    Prepended Global Handlers

    This PR adds the ability to set up a bunch of handlers that get pre-pended to every Handler that you set up.

    • Use() func bound to MWHandler
    • MWHandler basically holds an ordered slice of global handlers
    • On SETUP of a Handler, the two structs are joined together and then supplied to the closure
    • README updated
    • Tests updated and checked
    • Examples updated and checked that they run
    • Only 5 lines of code added to base library
    opened by caledhwa 1
  • Godocs

    Godocs

    Ok, I reformatted all the godocs. Too bad we have no way to see the changes (unless you pull and look at them locally with godoc --http=":6060" or something.

    • Added new image
    • Added LICENSE file (MIT)
    • Added some more stinkin' badges (I'll add the GoDoc one once we go public)
    • Added GoDocs (added example and code through docs.go & example_overall_test.go)
    • Tested example and added information on how to run it out of the box.
    opened by caledhwa 1
  • Rename Sirupsen to sirupsen

    Rename Sirupsen to sirupsen

    Go won't tolerate two different capitalizations of that name. Latest new relic uses the lower case one (which is what the main repo is also using). Conforming to the new style.

    Note: This will break all repos depending on it if they did not vendor.

    opened by johannesinvision 0
Releases(v1.0.8)
Owner
InVision
InVision
Minimalist net/http middleware for golang

interpose Interpose is a minimalist net/http middleware framework for golang. It uses http.Handler as its core unit of functionality, minimizing compl

James Pirruccello 296 Sep 27, 2022
Idiomatic HTTP Middleware for Golang

Negroni Notice: This is the library formerly known as github.com/codegangsta/negroni -- Github will automatically redirect requests to this repository

null 7.3k Sep 27, 2022
Go http.Hander based middleware stack with context sharing

wrap Package wrap creates a fast and flexible middleware stack for http.Handlers. Features small; core is only 13 LOC based on http.Handler interface;

go-on - web toolkit in Go 59 Apr 5, 2022
Lightweight Middleware for net/http

MuxChain MuxChain is a small package designed to complement net/http for specifying chains of handlers. With it, you can succinctly compose layers of

Stephen Searles 209 Apr 5, 2022
A collection of useful middleware for Go HTTP services & web applications 🛃

gorilla/handlers Package handlers is a collection of handlers (aka "HTTP middleware") for use with Go's net/http package (or any framework supporting

Gorilla Web Toolkit 1.5k Sep 18, 2022
Simple middleware to rate-limit HTTP requests.

Tollbooth This is a generic middleware to rate-limit HTTP requests. NOTE 1: This library is considered finished. NOTE 2: Major version changes are bac

Didip Kerabat 2.3k Sep 27, 2022
OpenID Connect (OIDC) http middleware for Go

Go OpenID Connect (OIDC) HTTP Middleware Introduction This is a middleware for http to make it easy to use OpenID Connect. Currently Supported framewo

Xenit AB 63 Aug 22, 2022
Go HTTP middleware to filter clients by IP

Go HTTP middleware to filter clients by IP

cristaltech 3 May 29, 2022
Chi ip banner is a chi middleware that bans some ips from your Chi http server.

Chi Ip Banner Chi ip banner is a chi middleware that bans some ips from your Chi http server. It reads a .txt file in your project's root, called bani

null 1 Jan 4, 2022
Painless middleware chaining for Go

Alice Alice provides a convenient way to chain your HTTP middleware functions and the app handler. In short, it transforms Middleware1(Middleware2(Mid

Justinas Stankevičius 2.7k Oct 1, 2022
A Go middleware that stores various information about your web application (response time, status code count, etc.)

Go stats handler stats is a net/http handler in golang reporting various metrics about your web application. This middleware has been developed and re

Florent Messa 586 Jul 27, 2022
gorilla/csrf provides Cross Site Request Forgery (CSRF) prevention middleware for Go web applications & services 🔒

gorilla/csrf gorilla/csrf is a HTTP middleware library that provides cross-site request forgery (CSRF) protection. It includes: The csrf.Protect middl

Gorilla Web Toolkit 831 Sep 14, 2022
URL Rewrite middleware for gin

Url Rewrite middleware for gin Example In this exable these urls use the same route http://localhost:1234/test-me http://localhost:1234/index.php/test

Lucian I. Last 3 Sep 15, 2022
A customized middleware of DAPR.

A customized middleware of DAPR.

Gatty 1 Dec 24, 2021
Gin middleware for session.

wsession Gin middleware for session management with multi-backend support: cookie-based Redis memstore Usage Start using it Download and install it: g

null 0 Jan 9, 2022
Fiber middleware for server-timing

Server Timing This is a Fiber middleware for the [W3C Server-Timing API] based on mitchellh/go-server-timing

Vlad Fratila 0 Feb 6, 2022
echo-http - Echo http service

echo-http - Echo http service Responds with json-formatted echo of the incoming request and with a predefined message. Can be install directly (go get

Umputun 13 Sep 13, 2022
A Concurrent HTTP Static file server using golang .

A Concurrent HTTP static server using Golang. Serve Static files like HTML,CSS,Js,Images,Videos ,ect. using HTTP. It is Concurrent and Highly Scalable.Try now!

sambath kumar B 4 Dec 19, 2021
Composable chains of nested http.Handler instances.

chain go get github.com/codemodus/chain Package chain aids the composition of nested http.Handler instances. Nesting functions is a simple concept. I

Code Modus 65 Sep 27, 2022