xmux is a httprouter fork on top of xhandler (net/context aware)

Related tags

Routers xmux
Overview

Xmux

godoc license Build Status Coverage

Xmux is a lightweight high performance HTTP request muxer on top xhandler. Xmux gets its speed from the fork of the amazing httprouter. Route parameters are stored in context instead of being passed as an additional parameter.

In contrast to the default mux of Go's net/http package, this muxer supports variables in the routing pattern and matches against the request method. It also scales better.

The muxer is optimized for high performance and a small memory footprint. It scales well even with very long paths and a large number of routes. A compressing dynamic trie (radix tree) structure is used for efficient matching.

Features

Only explicit matches: With other muxers, like http.ServeMux, a requested URL path could match multiple patterns. Therefore they have some awkward pattern priority rules, like longest match or first registered, first matched. By design of this router, a request can only match exactly one or no route. As a result, there are also no unintended matches, which makes it great for SEO and improves the user experience.

Stop caring about trailing slashes: Choose the URL style you like, the muxer automatically redirects the client if a trailing slash is missing or if there is one extra. Of course it only does so, if the new path has a handler. If you don't like it, you can turn off this behavior.

Path auto-correction: Besides detecting the missing or additional trailing slash at no extra cost, the muxer can also fix wrong cases and remove superfluous path elements (like ../ or //). Is CAPTAIN CAPS LOCK one of your users? Xmux can help him by making a case-insensitive look-up and redirecting him to the correct URL.

Parameters in your routing pattern: Stop parsing the requested URL path, just give the path segment a name and the router delivers the dynamic value to you. Because of the design of the router, path parameters are very cheap.

RouteGroups: A way to create groups of routes without incurring any per-request overhead.

Zero Garbage: The matching and dispatching process generates zero bytes of garbage. In fact, the only heap allocations that are made, is by building the slice of the key-value pairs for path parameters and the context instance to store them in the context. If the request path contains no parameters, not a single heap allocation is necessary.

No more server crashes: You can set a Panic handler to deal with panics occurring during handling a HTTP request. The router then recovers and lets the PanicHandler log what happened and deliver a nice error page.

Of course you can also set custom NotFound and MethodNotAllowed handlers.

Usage

This is just a quick introduction, view the GoDoc for details.

Let's start with a trivial example:

package main

import (
	"fmt"
	"log"
	"net/http"
	"context"

	"github.com/rs/xhandler"
	"github.com/rs/xmux"
)

func Index(ctx context.Context, w http.ResponseWriter, r *http.Request) {
	fmt.Fprint(w, "Welcome!\n")
}

func Hello(ctx context.Context, w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "hello, %s!\n", xmux.Param(ctx, "name"))
}

func main() {
	mux := xmux.New()
	mux.GET("/", xhandler.HandlerFuncC(Index))
	mux.GET("/hello/:name", xhandler.HandlerFuncC(Hello))

	log.Fatal(http.ListenAndServe(":8080", xhandler.New(context.Background(), mux)))
}

You may also chain middleware using xhandler.Chain:

package main

import (
	"context"
	"fmt"
	"log"
	"net/http"
	"time"

	"github.com/rs/xhandler"
	"github.com/rs/xmux"
)

func main() {
	c := xhandler.Chain{}

	// Append a context-aware middleware handler
	c.UseC(xhandler.CloseHandler)

	// Another context-aware middleware handler
	c.UseC(xhandler.TimeoutHandler(2 * time.Second))

	mux := xmux.New()

	// Use c.Handler to terminate the chain with your final handler
	mux.GET("/welcome/:name", xhandler.HandlerFuncC(func(ctx context.Context, w http.ResponseWriter, req *http.Request) {
		fmt.Fprintf(w, "Welcome %s!", xmux.Param(ctx, "name"))
	}))

	if err := http.ListenAndServe(":8080", c.Handler(mux)); err != nil {
		log.Fatal(err)
	}
}

Named parameters

As you can see, :name is a named parameter. The values are accessible via xmux.Params(ctx), which returns xmux.ParamHolder. You can get the value of a parameter by its name using Get(name) method:

user := xmux.Params(ctx).Get("user")

or using xmux.Param(ctx, name) shortcut:

user := xmux.Param(ctx, "user")

Named parameters only match a single path segment:

Pattern: /user/:user

 /user/gordon              match
 /user/you                 match
 /user/gordon/profile      no match
 /user/                    no match

Note: Since this muxer has only explicit matches, you can not register static routes and parameters for the same path segment. For example you can not register the patterns /user/new and /user/:user for the same request method at the same time. The routing of different request methods is independent from each other.

Catch-All parameters

The second type are catch-all parameters and have the form *name. Like the name suggests, they match everything. Therefore they must always be at the end of the pattern:

Pattern: /src/*filepath

 /src/                     match
 /src/somefile.go          match
 /src/subdir/somefile.go   match

Benchmarks

Thanks to Julien Schmidt excellent HTTP routing benchmark, we can see that xhandler's muxer is pretty close to httprouter as it is a fork of it. The small overhead is due to the context allocation used to store route parameters. It still outperform other routers, thanks to amazing httprouter's radix tree based matcher.

BenchmarkXhandler_APIStatic-8   	50000000	        39.6 ns/op	       0 B/op	       0 allocs/op
BenchmarkChi_APIStatic-8        	 3000000	       439 ns/op	     144 B/op	       5 allocs/op
BenchmarkGoji_APIStatic-8       	 5000000	       272 ns/op	       0 B/op	       0 allocs/op
BenchmarkHTTPRouter_APIStatic-8 	50000000	        37.3 ns/op	       0 B/op	       0 allocs/op

BenchmarkXhandler_APIParam-8    	 5000000	       328 ns/op	     160 B/op	       4 allocs/op
BenchmarkChi_APIParam-8         	 2000000	       675 ns/op	     432 B/op	       6 allocs/op
BenchmarkGoji_APIParam-8        	 2000000	       692 ns/op	     336 B/op	       2 allocs/op
BenchmarkHTTPRouter_APIParam-8  	10000000	       166 ns/op	      64 B/op	       1 allocs/op

BenchmarkXhandler_API2Params-8  	 5000000	       362 ns/op	     160 B/op	       4 allocs/op
BenchmarkChi_API2Params-8       	 2000000	       814 ns/op	     432 B/op	       6 allocs/op
BenchmarkGoji_API2Params-8      	 2000000	       680 ns/op	     336 B/op	       2 allocs/op
BenchmarkHTTPRouter_API2Params-8	10000000	       183 ns/op	      64 B/op	       1 allocs/op

BenchmarkXhandler_APIAll-8      	  200000	      6473 ns/op	    2176 B/op	      64 allocs/op
BenchmarkChi_APIAll-8           	  100000	     17261 ns/op	    8352 B/op	     146 allocs/op
BenchmarkGoji_APIAll-8          	  100000	     15052 ns/op	    5377 B/op	      32 allocs/op
BenchmarkHTTPRouter_APIAll-8    	  500000	      3716 ns/op	     640 B/op	      16 allocs/op

BenchmarkXhandler_Param1-8      	 5000000	       271 ns/op	     128 B/op	       4 allocs/op
BenchmarkChi_Param1-8           	 2000000	       620 ns/op	     432 B/op	       6 allocs/op
BenchmarkGoji_Param1-8          	 3000000	       522 ns/op	     336 B/op	       2 allocs/op
BenchmarkHTTPRouter_Param1-8    	20000000	       112 ns/op	      32 B/op	       1 allocs/op

BenchmarkXhandler_Param5-8      	 3000000	       414 ns/op	     256 B/op	       4 allocs/op
BenchmarkChi_Param5-8           	 1000000	      1204 ns/op	     432 B/op	       6 allocs/op
BenchmarkGoji_Param5-8          	 2000000	       847 ns/op	     336 B/op	       2 allocs/op
BenchmarkHTTPRouter_Param5-8    	 5000000	       247 ns/op	     160 B/op	       1 allocs/op

BenchmarkXhandler_Param20-8     	 2000000	       747 ns/op	     736 B/op	       4 allocs/op
BenchmarkChi_Param20-8          	 2000000	       746 ns/op	     736 B/op	       4 allocs/op
BenchmarkGoji_Param20-8         	  500000	      2439 ns/op	    1247 B/op	       2 allocs/op
BenchmarkHTTPRouter_Param20-8   	 3000000	       585 ns/op	     640 B/op	       1 allocs/op

BenchmarkXhandler_ParamWrite-8  	 5000000	       404 ns/op	     144 B/op	       5 allocs/op
BenchmarkChi_ParamWrite-8       	 3000000	       407 ns/op	     144 B/op	       5 allocs/op
BenchmarkGoji_ParamWrite-8      	 2000000	       594 ns/op	     336 B/op	       2 allocs/op
BenchmarkHTTPRouter_ParamWrite-8	10000000	       166 ns/op	      32 B/op	       1 allocs/op

You can run this benchmark by executing the following commands at the root of xmux repository:

go get -t ./bench/routers
go test ./bench/routers -bench .

How does it work?

The muxer relies on a tree structure which makes heavy use of common prefixes, it is basically a compact prefix tree (or just Radix tree). Nodes with a common prefix also share a common parent. Here is a short example what the routing tree for the GET request method could look like:

Priority   Path             Handle
9          \                *<1>
3          ├s               nil
2          |├earch\         *<2>
1          |└upport\        *<3>
2          ├blog\           *<4>
1          |    └:post      nil
1          |         └\     *<5>
2          ├about-us\       *<6>
1          |        └team\  *<7>
1          └contact\        *<8>

Every *<num> represents the memory address of a handler function (a pointer). If you follow a path trough the tree from the root to the leaf, you get the complete route path, e.g \blog\:post\, where :post is just a placeholder (parameter) for an actual post name. Unlike hash-maps, a tree structure also allows us to use dynamic parts like the :post parameter, since we actually match against the routing patterns instead of just comparing hashes. As benchmarks show, this works very well and efficient.

Since URL paths have a hierarchical structure and make use only of a limited set of characters (byte values), it is very likely that there are a lot of common prefixes. This allows us to easily reduce the routing into ever smaller problems. Moreover the router manages a separate tree for every request method. For one thing it is more space efficient than holding a method->handle map in every single node, for another thing is also allows us to greatly reduce the routing problem before even starting the look-up in the prefix-tree.

For even better scalability, the child nodes on each tree level are ordered by priority, where the priority is just the number of handles registered in sub nodes (children, grandchildren, and so on..). This helps in two ways:

  1. Nodes which are part of the most routing paths are evaluated first. This helps to make as much routes as possible to be reachable as fast as possible.
  2. It is some sort of cost compensation. The longest reachable path (highest cost) can always be evaluated first. The following scheme visualizes the tree structure. Nodes are evaluated from top to bottom and from left to right.
├------------
├---------
├-----
├----
├--
├--
└-

Why doesn't this work with http.Handler?

It does! The router itself implements the http.Handler interface. Moreover the router provides convenient adapters for http.Handlers and http.HandlerFuncs which allows them to be used as a xhandler.HandlerC when registering a route. The only disadvantage is, that no context and thus no parameter values can be retrieved when a http.Handler or http.HandlerFunc is used.

Where can I find Middleware X?

This package just provides a very efficient request muxer with a few extra features. The muxer is just a xhandler.HandlerC, you can chain any http.Handler or xhandler.HandlerC compatible middleware before the router, for example the Gorilla handlers. Or you could just write your own, it's very easy!

Multi-domain / Sub-domains

Here is a quick example: Does your server serve multiple domains / hosts? You want to use sub-domains? Define a router per host!

// We need an object that implements the xhandler.HandlerC interface.
// Therefore we need a type for which we implement the ServeHTTP method.
// We just use a map here, in which we map host names (with port) to xhandler.HandlerC
type HostSwitch map[string]xhandler.HandlerC

// Implement the ServerHTTP method on our new type
func (hs HostSwitch) ServeHTTPC(ctx context.Context, w http.ResponseWriter, r *http.Request) {
	// Check if a xhandler.HandlerC is registered for the given host.
	// If yes, use it to handle the request.
	if handler := hs[r.Host]; handler != nil {
		handler.ServeHTTPC(ctx, w, r)
	} else {
		// Handle host names for wich no handler is registered
		http.Error(w, "Forbidden", 403) // Or Redirect?
	}
}

func main() {
	c := xhandler.Chain{}

	// Initialize a muxer as usual
	mux := xmux.New()
	mux.GET("/", Index)
	mux.GET("/hello/:name", Hello)

	// Make a new HostSwitch and insert the muxer (our http handler)
	// for example.com and port 12345
	hs := make(HostSwitch)
	hs["example.com:12345"] = mux

	// Use the HostSwitch to listen and serve on port 12345
	if err := http.ListenAndServe(":12345", c.Handler(hs)); err != nil {
		log.Fatal(err)
	}
}

Basic Authentication

Another quick example: Basic Authentication (RFC 2617) for handles:

package main

import (
	"bytes"
	"context"
	"encoding/base64"
	"fmt"
	"log"
	"net/http"
	"strings"

	"github.com/rs/xhandler"
	"github.com/rs/xmux"
)

func BasicAuth(user, pass []byte, next xhandler.HandlerC) xhandler.HandlerC {
	return xhandler.HandlerFuncC(func(ctx context.Context, w http.ResponseWriter, r *http.Request) {
		const basicAuthPrefix string = "Basic "

		// Get the Basic Authentication credentials
		auth := r.Header.Get("Authorization")
		if strings.HasPrefix(auth, basicAuthPrefix) {
			// Check credentials
			payload, err := base64.StdEncoding.DecodeString(auth[len(basicAuthPrefix):])
			if err == nil {
				pair := bytes.SplitN(payload, []byte(":"), 2)
				if len(pair) == 2 &&
					bytes.Equal(pair[0], user) &&
					bytes.Equal(pair[1], pass) {

					// Delegate request to the next handler
					next.ServeHTTPC(ctx, w, r)
					return
				}
			}
		}

		// Request Basic Authentication otherwise
		w.Header().Set("WWW-Authenticate", "Basic realm=Restricted")
		http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
	})
}

func Index(ctx context.Context, w http.ResponseWriter, r *http.Request) {
	fmt.Fprint(w, "Not protected!\n")
}

func Protected(ctx context.Context, w http.ResponseWriter, r *http.Request) {
	fmt.Fprint(w, "Protected!\n")
}

func main() {
	user := []byte("gordon")
	pass := []byte("secret!")

	c := xhandler.Chain{}
	mux := xmux.New()
	mux.GET("/", xhandler.HandlerFuncC(Index))
	mux.GET("/protected/", BasicAuth(user, pass, xhandler.HandlerFuncC(Protected)))

	log.Fatal(http.ListenAndServe(":8080", c.Handler(mux)))
}

Licenses

All source code is licensed under the BSD License.

Xmux is forked from httprouter with BSD License.

Comments
  • Setting URL parameters from test code

    Setting URL parameters from test code

    I was wondering what is the cleanest way to set url params when we test handlers, and thought it might be useful to make xmux.newParamContext public, or create another xmux.TestNewParamContext just for testing. I'll describe the concrete example below, and would appreciate if you can give me some feedback.

    Suppose we have the following handler and server.

    func helloWithParam(ctx context.Context, w http.ResponseWriter, r *http.Request) {
        n := xmux.Param(ctx, "name")
        fmt.Fprintf(w, "Hello, %s!", n)
        return
    }
    
    func main() {
        m := xmux.New()
        m.GET("/hello/:name", xhandler.HandlerFuncC(helloWithParam))
        if err := http.ListenAndServe(":8888", xhandler.New(context.Background(), m)); err != nil {
            log.Fatal(err)
        }
    }
    

    I want to test like the below so that I can test only a simple function, keeping routing/middleware aside.

    func TestHelloWithParam(t *testing.T) {
        n := "dummyName"
        ps := xmux.ParamHolder{xmux.Parameter{Name: "name", Value: n}}
        ctx := xmux.NewParamContext(context.TODO(), ps) // This line could be xmux.TestNewParamContext()
        req, _ := http.NewRequest("GET", "/hello/"+n, nil)
        w := httptest.NewRecorder()
        helloWithParam(ctx, w, req)
        t.Log(w.Body)
    }
    

    The idea of test helper public API is borrowed from this slide.

    • Advanced Testing with Go
      • https://speakerdeck.com/mitchellh/advanced-testing-with-go?slide=52
    opened by achiku 2
  • Wrapping group with middleware

    Wrapping group with middleware

    Hi,

    Firstly, thanks for the awesome package!

    I'm currently evaluating xhandler/xmux, and I can't seem to find a way to wrap a Group instance with middleware.

    My use case would be something like:

    m := xmux.New()
    g := m.Group("/admin")
    g.GET("/users", AuthMiddleware(adminUserHandler))
    g.GET("/items", AuthMiddleware(adminItemHandler))
    

    Is there a way for me to remove the redundant AuthMiddleware on each handler and associate it with g so it is evaluated for every route on g?

    Thanks!

    opened by Taik 1
  • License issue

    License issue

    I just discovered your fork. Great work so far!

    Unfortunately there is a license issue. Like the Go source, HttpRouter is licensed under the new BSD (aka BSD 3-Clause) license. While it is also a permissive license, it can NOT be re-licensed to MIT, as it requires to preserve the redistribution conditions.

    compatibility chart

    opened by julienschmidt 1
  • Add TestSetParamContext to sets ParamHolder for testing

    Add TestSetParamContext to sets ParamHolder for testing

    As discussed in https://github.com/rs/xmux/issues/7, this function is purely for testing, and set ParamHolder in context.Context.

    func helloWithParam(ctx context.Context, w http.ResponseWriter, r *http.Request) {
        n := xmux.Param(ctx, "name")
        fmt.Fprintf(w, "Hello, %s!", n)
        return
    }
    
    func main() {
        m := xmux.New()
        m.GET("/hello/:name", xhandler.HandlerFuncC(helloWithParam))
        if err := http.ListenAndServe(":8888", xhandler.New(context.Background(), m)); err != nil {
            log.Fatal(err)
        }
    }
    
    func TestHelloWithParam(t *testing.T) {
        n := "dummyName"
        ps := xmux.ParamHolder{xmux.Parameter{Name: "name", Value: n}}
        ctx := xmux.TestSetParamContext(context.TODO(), ps)
        req, _ := http.NewRequest("GET", "/hello/"+n, nil)
        w := httptest.NewRecorder()
        helloWithParam(ctx, w, req)
        t.Log(w.Body)
    }
    
    opened by achiku 0
  • Go 1.8 context

    Go 1.8 context

    The context package is now part of the standard library. there is a fix tool for rewriting this import path, I already done it in our build flow, but I think it's hard to decide to break the old builds, so I prefer to open an issue instead of a PR.

    Is this any plan to move forward? another branch maybe, or not? this apply to xhandler and cors repo as well.

    opened by fzerorubigd 4
  • Provide matched route as part of context

    Provide matched route as part of context

    It would be useful if the matched route is embedded as a context value. For example, you could build a middleware that computed and logged response time metrics over time on a per-route basis:

    func Instrumenter(next xhandler.HandlerC) xhandler.HandlerC {
      return xhandler.HandlerFuncC(func(ctx context.Context, w http.ResponseWriter, r *http.Request) {
        route := RouteFromContext(ctx)  // Hypothetical new function
        now = time.Now()
        next.ServeHTTPC(ctx, w, r)
        elapsed := float64(time.Since(now)) / float64(time.Microsecond)
        logRequest(r, route, elapsed)
      })
    }
    
    func logRequest(r *http.Request, route string, elasped float64) {
      // ... update internal table of response time histograms ...
    }
    
    ...
    chain := xhandler.Chain{}
    chain.UseC(Instrumentater)
    mux.POST("/api/v1/query", chain.HandlerC(xhandler.HandlerFuncC(someHandler)))
    

    I can do a PR if you agree that this is useful.

    opened by atombender 0
Owner
Olivier Poitrey
Director of Engineering at Netflix Co-Founder & ex-CTO of Dailymotion Co-Founder of NextDNS
Olivier Poitrey
Mux is a simple and efficient route distributor that supports the net/http interface of the standard library.

Mux Mux is a simple and efficient route distributor that supports the net/http interface of the standard library. Routing data is stored in the prefix

null 137 Nov 22, 2022
Simple router build on `net/http` supports custom middleWare.

XMUS-ROUTER Fast lightweight router build on net/http supports delegate and in url params. usage : Create new router using NewRouter() which need rout

amupxm [amir hossein mokarrami far] 5 Dec 27, 2021
xlog is a logger for net/context aware HTTP applications

⚠️ Check zerolog, the successor of xlog. HTTP Handler Logger xlog is a logger for net/context aware HTTP applications. Unlike most loggers, xlog will

Olivier Poitrey 137 Sep 26, 2022
gpool - a generic context-aware resizable goroutines pool to bound concurrency based on semaphore.

gpool - a generic context-aware resizable goroutines pool to bound concurrency. Installation $ go get github.com/sherifabdlnaby/gpool import "github.c

Sherif Abdel-Naby 86 Oct 31, 2022
Go library that makes it easy to add automatic retries to your projects, including support for context.Context.

go-retry Go library that makes it easy to add automatic retries to your projects, including support for context.Context. Example with context.Context

Conner Douglass 6 Aug 15, 2022
containedctx detects is a linter that detects struct contained context.Context field

containedctx containedctx detects is a linter that detects struct contained context.Context field Instruction go install github.com/sivchari/contained

sivchari 12 Oct 22, 2022
Example code to demonstrate how to mock external clients via context.Context

Mocking external client libraries using context.Context This code is paired with a blog post: Mocking external client libraries using context.Context

incident.io 3 Nov 6, 2022
Morecontext - Context.Context helpers to make your life slightly easier

morecontext context.Context helpers to make your life slightly easier. -- import

Gopuff 3 Mar 5, 2022
top in container - Running the original top command in a container

Running the original top command in a container will not get information of the container, many metrics like uptime, users, load average, tasks, cpu, memory, are about the host in fact. topic(top in container) will retrieve those metrics from container instead, and shows the status of the container, not the host.

silenceshell 123 Nov 16, 2022
Lightweight Router for Golang using net/http standard library with custom route parsing, handler and context.

Go-Lightweight-Router Lightweight Router for Golang using net/http standard library with custom route parsing, handler and context. Further developmen

null 0 Nov 3, 2021
Rocinante is a gin inspired web framework built on top of net/http.

Rocinante Rocinante is a gin inspired web framework built on top of net/http. ⚙️ Installation $ go get -u github.com/fskanokano/rocinante-go ⚡️ Quicks

null 3 Jul 27, 2021
fhttp is a fork of net/http that provides an array of features pertaining to the fingerprint of the golang http client.

fhttp The f stands for flex. fhttp is a fork of net/http that provides an array of features pertaining to the fingerprint of the golang http client. T

Flexagon 62 Nov 11, 2022
Fork of Go stdlib's net/http that works with alternative TLS libraries like refraction-networking/utls.

github.com/ooni/oohttp This repository contains a fork of Go's standard library net/http package including patches to allow using this HTTP code with

Open Observatory of Network Interference (OONI) 30 Sep 29, 2022
Go package to simulate bandwidth, latency and packet loss for net.PacketConn and net.Conn interfaces

lossy Go package to simulate bandwidth, latency and packet loss for net.PacketConn and net.Conn interfaces. Its main usage is to test robustness of ap

Cevat Barış Yılmaz 313 Oct 14, 2022
A fast data generator that's multi-table aware and supports multi-row DML.

If you need to generate a lot of random data for your database tables but don't want to spend hours configuring a custom tool for the job, then datage

Coding Concepts 51 Nov 21, 2022
Pomerium is an identity-aware access proxy.

Pomerium is an identity-aware proxy that enables secure access to internal applications. Pomerium provides a standardized interface to add access cont

null 3.3k Nov 20, 2022
A collection of (ANSI-sequence aware) text reflow operations & algorithms

reflow A collection of ANSI-aware methods and io.Writers helping you to transform blocks of text. This means you can still style your terminal output

Christian Muehlhaeuser 401 Nov 21, 2022
depaware makes you aware of your Go dependencies

depaware depaware makes you aware of your Go dependencies. It generates a list of your dependencies which you check in to your repo: https://github.co

Tailscale 414 Nov 19, 2022
Content aware image resize library

Caire is a content aware image resize library based on Seam Carving for Content-Aware Image Resizing paper. How does it work An energy map (edge detec

Endre Simo 10.1k Nov 21, 2022
self-aware Golang profile dumper[beta]

holmes WARNING : holmes is under heavy development now, so API will make breaking change during dev. If you want to use it in production, please wait

MOSN 822 Nov 22, 2022