Secure-by-default HTTP servers in Go.

Overview

go-safeweb

DISCLAIMER: This is not an officially supported Google product.

go-safeweb is a collection of libraries for writing secure-by-default HTTP servers in Go.

Contributing

This project is in an early stage. We are currently not accepting any contributions.

Overview

The flexibility of Go’s net/http package allows users to quickly implement HTTP servers.

Responses are then written simply as slices of bytes, headers can be arbitrarily manipulated and so on. This approach offers much needed flexibility for these who really need it.

Unfortunately, this approach leaves great space for introducing security vulnerabilities and even experienced developers tend to do so.

This document aims to design an HTTP API that eliminates whole classes of bugs, like Cross-Site Scripting (XSS) or Cross-Site Request Forgery (XSRF). This can be achieved by an approach known at Google as safe coding. Learn more at Securing the Tangled Web (Chistoph Kern, 2014) or Preventing Security Bugs through Software Design (Christoph Kern, 2016).

Goals and Non-Goals

Goals

G1: Secure-by-default

Security mechanisms are applied by default (opt-out, not opt-in).

G2: Unsafe Usage is Easy to Review, Track and Restrict

All opt-outs from security mechanisms are explicit. Wherever possible, they’re contained inside a package or an option that’s easy to restrict.

G3: Designed for Evolving Security Requirements

Enforcing new security measures is feasible through AST manipulation. Existing users can be migrated using static analysis and/or runtime monitoring. Read more here.

G4: High Compatibility with Go’s Standard Library and Existing Open-Source Frameworks

Whenever possible, keep existing layouts, function signatures and other API parts the same as the Go’s standard library. High compatibility enables wide adoption.

Non Goals

NG1: Safe API Completeness

Creating safe APIs for all the corner cases might result in a bloated codebase. Our experience shows that this isn’t necessary.

NG2: Full Compatibility with Go’s Standard Library and Existing Open-Source Frameworks

Existing open-source frameworks or the Go standard library need to support each developer scenario. This would have left us with limited options of creating safe-by-default HTTP servers.

NG3: Features That Are Not Security Critical

Go Safe Web aims to help you create a secure-by-default Go HTTP server and nothing more. Features that are not security critical will not be added. Focusing solely on security allows us to maintain high compatibility with the standard library and makes adoption easier.

Security Vulnerabilities and Mitigations

On a high level, we plan to address, or provide the needed infrastructure to address, following issues (not an exhaustive list):

  • XSS (cross-site scripting) and XSSI (cross-site script inclusion) - e.g. by controlling how responses are generated
  • XSRF (cross-site request forgery) - e.g. by using Fetch Metadata policies, supporting token-based XSRF protection
  • CORS (cross-origin resource sharing) - e.g. by taking control of CORS response headers and handling CORS preflight requests
  • CSP (content security policy) - e.g. by automatically adding script nonces to HTML responses, adding relevant security headers
  • Transport Security - e.g. by enforcing HSTS support
  • IFraming - e.g. by setting relevant HTTP headers to restrict framing or providing server-side support for origin selection
  • Auth (access control) - e.g. by providing infrastructure for plugging in access control logic in an uniform, auditable way
  • HTTP Request Parsing Bugs - e.g. by implementing strict and well documented parsing behavior
  • Error responses - e.g. by providing infrastructure for uniform error handling (e.g. to prevent accidental leaks or XSS from error responses)
  • Enforcement of other security specific HTTP headers - here

Appendix

Evolving Security Requirements (example)

Imagine an API for configuring access control. It features three types of rules:

  • ALLOW(user) - allows a given user
  • DENY(user) - denies a given user (has priority over ALLOW)
  • REPORT(user) - reports that it has seen a request from a given user

Imagine now that at some point, security standards need to be increased and user = "frombulator" has been determined to not meet the desired bar.

How do we, for all the services running in our company, address this?

  1. For existing services, we add a LegacyFrombulatorAccess option like so: security.AccessControl(rules, unsafe.LegacyFrombulatorAccess()).
  2. We change the security.AccessControl() call to add by default a DENY("frombulator") rule. This rule is not added if unsafe.LegacyFrombulatorAccess is applied.
  3. Instead, unsafe.LegacyFrombulatorAccess adds a REPORT("frombulator") rule.

This way, we have:

  • Ensured that all new callers of security.AccessControl use the safe setting by default.
  • Can monitor existing services dependence on calls from the frombulator. After a period of observation (let’s say, 30 days):
    • If the service doesn’t receive requests from the frombulator: prune the unsafe.LegacyFrombulatorAccess option.
    • If the service does receive requests from the frombulator: inform the service owners and plan a fix.

Crucially, only the last case (dependence on unsafe configuration) requires engineering work per service. The rest can be automated.

This approach is possible due to careful API design. A missing DENY or REPORT rule, or a single sink in the form of security.AccessControl would make this infeasible.

Source Code Headers

Every file containing source code must include copyright and license information. This includes any JS/CSS files that you might be serving out to browsers. (This is to help well-intentioned people avoid accidental copying that doesn't comply with the license.)

Apache header:

Copyright 2020 Google LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Issues
  • If it turns out that a response will be an error (4xx, 5xx) we want to be able to remove headers set by plugins.

    If it turns out that a response will be an error (4xx, 5xx) we want to be able to remove headers set by plugins.

    Imagine that we have two plugins installed. The first one claims and sets a header in the Before phase. The other plugins Before phase runs and an error occurs, so the plugin writes an error response (4xx, 5xx). Now, what happens to the header that the first plugin set? Depending on the plugin this header should probably not be present on error responses. So how do we remove it?

    One option would be that all error responses, 4xx and 5xx, automatically remove all headers and set their own ones. But this becomes a problem since we might want CSP on error responses. CSP is handled by a plugin.

    Another solution is that all plugins are responsible for removing their headers in the commit phase if it sees that the response is an error.

    This needs to be discussed further.

    enhancement core proposal needs decision 
    opened by mattiasgrenfeldt 10
  • Make sure ServeMux is used in the right order

    Make sure ServeMux is used in the right order

    The ServeMux construction goes through phases:

    1. Plugin registration
    2. Handlers registration and configuration
    3. Execution

    Nothing that is done in one of the phases should be done in one of the others.

    I think we should enforce this somehow, by either providing builders for the phases or panicking if something is done out of order at startup, and then ignoring what happens at runtime (e.g. registering a plugin or a handler after a request has been served will log an error and do nothing).

    enhancement core v0.1 needs decision 
    opened by empijei 9
  • Extended the Dispatcher interface to support writing JSON and retrieving the response Content-Type and created a safe default Dispatcher.

    Extended the Dispatcher interface to support writing JSON and retrieving the response Content-Type and created a safe default Dispatcher.

    Fixes #60, #133

    Extended the Dispatcher interface to support writing JSON safe responses and retrieving the Content-Type of safe response and added a . This way the ResponseWriter can query the Dispatcher to retrieve and set the Content-Type of a safe response before writing the response. Implemented and write tests for a default safe dispatcher.

    enhancement core v0.1 
    opened by maramihali 8
  • Extended the MultipartForm type to allow accessing file uploads

    Extended the MultipartForm type to allow accessing file uploads

    Fixes #32, #38

    Extended the MultipartForm to support retrieving file uploads from incoming requests. This will prevent the user from directly accessing the multipart form but will still allow access to the accessor methods in multipart used for opening files.

    Fixed the issue that was causing a multipart form to be capped at 32 MB (my misunderstanding of documentation).

    This branch is rebased on top of grenfeldt-splitting-form-tests branch (#125 ).

    enhancement 
    opened by maramihali 7
  • Add an example of a server which presents DOM XSS protection with Trusted Types

    Add an example of a server which presents DOM XSS protection with Trusted Types

    Add an example of a server which presents DOM XSS protection with Trusted Types.

    • [ ] Tests pass
    • [ ] Appropriate changes to README are included in PR
    v0.1 
    opened by mtrzos 6
  • Added a report collector plugin which can be used to generate a collector handler.

    Added a report collector plugin which can be used to generate a collector handler.

    It can be used to create a reports collector handler for CSP reports. The user can provide a collector object which will be called with each incoming report.

    Fixes #75

    enhancement plugin 
    opened by mattiasgrenfeldt 6
  • Added the Commit phase

    Added the Commit phase

    Fixes #145

    Added the Commit phase to the Interceptor interface. This will be called by the safehttp.ResponseWriter before a response is written to the underlying http.ResponseWriter, for all interceptors that were installed on the handler. If a response is written in any of the Commit phases, then the Commit phases of the remaining interceptor won't execute.

    As the Commit phase takes a safehttp.Response as argument (e.g. for injecting nonces), we also created a safehttp.TemplateResponse interface that encapsulates a safe template and the data that is applied to it.

    Co-authored-by: Mihali Mara [email protected]

    enhancement core v0.1 
    opened by mattiasgrenfeldt 5
  • Splitting form_test.go so that it only contains unit tests for the Form type and adding unit tests for the IncomingRequest Type separately.

    Splitting form_test.go so that it only contains unit tests for the Form type and adding unit tests for the IncomingRequest Type separately.

    Fixes #57

    Test Coverage

    • form.go: 73% -> 100%
    • incoming_request.go: 69.8% -> 95.3%

    Lines

    • Before: form_test.go + incoming_request_test.go = 1147 + 196 = 1343
    • After: form_test.go + incoming_request_test.go = 488 + 360 = 848
    • Diff: -495
    internal cleanup testing 
    opened by mattiasgrenfeldt 5
  • Clear headers map in safehttp.ResponseWriter.WriteError

    Clear headers map in safehttp.ResponseWriter.WriteError

    In order to prevent leaks when an error occurs, we need to ensure the headers map is cleared before calling http.Error in the safehttp.ResponseWriter.WriteError funciton.

    enhancement needs information proposal 
    opened by maramihali 4
  • Added a safehttp.NotWritten result so that safehttp.Result{} can stop being used.

    Added a safehttp.NotWritten result so that safehttp.Result{} can stop being used.

    Fixes #109

    I added a safehttp.NotWritten variable which we can use everywhere instead of safehttp.Result{}. If a Handler returns NotWritten then a 204 No Content response is automatically set by the ServeMux.

    enhancement core 
    opened by mattiasgrenfeldt 4
  • Make sure that ResponseWriter is only written to once, otherwise panic.

    Make sure that ResponseWriter is only written to once, otherwise panic.

    Fixes #97

    To be able to store the state whether or not the ResponseWriter was written to or not the signature of Handler and Interceptor.Before had to change to accept a *ResponseWriter instead of a ResponseWriter. This needed to be changed in a lot of places.

    enhancement core 
    opened by mattiasgrenfeldt 4
  • Muxes are not composable

    Muxes are not composable

    A nice feature of http.Mux is that it can be used as a handler, so users can set up separate parts of an application on separate muxes that share a common configuration (e.g. a set of interceptors) and then install them on the root mux.

    This is currently not possible with GSW, and we should seriously consider doing it.

    enhancement good first issue core v0.2 
    opened by empijei 2
  • gofmt GitHub action seems broken

    gofmt GitHub action seems broken

    On this PR: https://github.com/google/go-safeweb/runs/4009332391?check_suite_focus=true

    I get:

    Run [ "$(gofmt -l $(find . -name '*.go') 2>&1)" = "" ]
      [ "$(gofmt -l $(find . -name '*.go') 2>&1)" = "" ]
      shell: /bin/bash -e {0}
      env:
        GOROOT: /opt/hostedtoolcache/go/1.17.2/x64
    Error: Process completed with exit code 
    

    There is little debugging information here, too. I think this should be changed.

    bug 
    opened by kele 1
  • Add an After stage to the interceptor flow

    Add an After stage to the interceptor flow

    Currently, interceptors can only know what happened before the Dispatcher has run. In particular, any kind of time measurements or status code logging is impossible.

    core 
    opened by kele 0
Owner
Google
Google ❤️ Open Source
Google
Python's webbrowser in Go. Simple API for opening web pages on your default browser.

webbrowser webbrowser provides a simple API for opening web pages on your default browser. It's inspired on Python's webbrowser package but lacks some

Carlos Cobo 175 Jul 20, 2022
IRC bot for launch ddos attack, Mainly of scan target are IoT device that run linux and open default SSH port

IRC bot for launch ddos attack, Mainly of scan target are IoT device that run linux and open default SSH port

R4bin 3 Nov 10, 2021
Zero downtime restarts for go servers (Drop in replacement for http.ListenAndServe)

endless Zero downtime restarts for golang HTTP and HTTPS servers. (for golang 1.3+) Inspiration & Credits Well... it's what you want right - no need t

Florian von Bock 3.6k Aug 9, 2022
Lobby - A Nox game lobby which exposes a simple HTTP API for both listing and registering Nox game servers

Nox lobby server This project provides a Nox game lobby which exposes a simple H

NoX World 3 Mar 6, 2022
Simple, secure and modern Go HTTP server to serve static sites, single-page applications or a file with ease

srv srv is a simple, secure and modern HTTP server, written in Go, to serve static sites, single-page applications or a file with ease. You can use it

Kevin Pollet 55 May 23, 2022
🚀Gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily build high-performance servers.

gev 中文 | English gev is a lightweight, fast non-blocking TCP network library based on Reactor mode. Support custom protocols to quickly and easily bui

徐旭 1.5k Aug 8, 2022
llb - It's a very simple but quick backend for proxy servers. Can be useful for fast redirection to predefined domain with zero memory allocation and fast response.

llb What the f--k it is? It's a very simple but quick backend for proxy servers. You can setup redirect to your main domain or just show HTTP/1.1 404

Kirill Danshin 12 Jan 23, 2022
Easy SSH servers in Golang

gliderlabs/ssh The Glider Labs SSH server package is dope. —@bradfitz, Go team member This Go package wraps the crypto/ssh package with a higher-level

Glider Labs 2.7k Aug 12, 2022
mdmb is a tool for simulating Apple devices interacting with Apple MDM servers.

mdmb mdmb — short for MDM Benchmark, à la ab — is a tool for simulating Apple devices interacting with Apple MDM servers. mdmb creates sets of fake Ap

Jesse Peterson 37 Jul 2, 2022
The fastest way to create self-hosted exit-servers

inletsctl - the fastest way to create self-hosted exit-servers inletsctl automates the task of creating an exit-server (tunnel server) on public cloud

inlets 421 Aug 4, 2022
DNS Ping: to check packet loss and latency issues with DNS servers

DNSping DNS Ping checks packet loss and latency issues with DNS servers Installation If you have golang, easiest install is go get -u fortio.org/dnspi

Fortio (Φορτίο) 60 Aug 5, 2022
A vote botting wrapper for GoLang designed for Minecraft: Pocket Servers.

libvote A vote botting wrapper for GoLang designed for Minecraft: Pocket Servers by Jviguy and JustTal. Disclaimer Usage of libvote requires your own

Tal 2 Apr 17, 2022
List running processes that are acting as DCE/RPC servers or clients

rpcls This project was made to assist in a larger research project. It pulls from a running process' PEB to enumerate the loaded DLLs. If a process im

Alex Flores 4 Apr 21, 2022
mt-multiserver-proxy is a reverse proxy designed for linking multiple Minetest servers together

mt-multiserver-proxy mt-multiserver-proxy is a reverse proxy designed for linking multiple Minetest servers together. It is the successor to multiserv

null 12 Aug 16, 2022
WebRTC media servers stress testing tool (currently only Janus)

GHODRAT WebRTC media servers stress testing tool (currently only Janus) Architecture Janus media-server under load Deployment ghodrat # update or crea

Snapp Cab Incubators 32 Jul 25, 2022
Create inlets servers on fly.io

inlets-on-fly inlets-on-fly automates the task of creating an inlets-pro exit-server (tunnel server) on the fly.io platform. This automation started a

Johan Siebens 8 Jan 8, 2022
Powered by Matterbridge, MatterAMXX is a plugin for AMXX that allows simple bridging between your game servers, Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, and more.

Powered by Matterbridge, MatterAMXX is a plugin for AMXX that allows simple bridging between your game servers, Mattermost, IRC, XMPP, Gitter, Slack, Discord, Telegram, and more.

Gabriel Iggy N. 9 May 21, 2022
Package manager for minecraft servers

KoperManager Package manager for minecraft servers Install minecraft server software and plugins in 1 click command Setup server ./koper_manager setup

null 2 Dec 23, 2021