Golang io.Reader and io.Writer but with limits

Related tags

Utilities go golang io
Overview

LimitIO

GoDoc Build status codecov Go Report Card

io.Reader and io.Writer with limit.

go get github.com/nanmu42/limitio

Rationale and Usage

There are times when a limited reader or writer comes in handy.

  1. wrap upstream so that reading is metered and limited:
// request is an incoming http.Request
request.Body = limitio.NewReadCloser(c.Request.Body, maxRequestBodySize, false)

// deal with the body now with easy mind. It's maximum size is assured.

Yes, io.LimitReader works the same way, but throws EOF on exceeding limit, which is confusing.

LimitIO provides error that can be identified.

decoder := json.NewDecoder(request.Body)
err := decoder.Decode(&myStruct)
if err != nil {
    if errors.Is(err, limitio.ErrThresholdExceeded) {
        // oops, we reached the limit
    }

    err = fmt.Errorf("other error happened: %w", err)
    return
}
  1. wrap downstream so that writing is metered and limited(or instead, just pretending writing):
// request is an incoming http.Request.
// Say, we want to record its body somewhere in the middleware,
// but feeling uneasy since its body might be HUGE, which may
// result in OOM and a successful DDOS...

var reqBuf bytes.buffer

// a limited writer comes to rescue!
// `true` means after reaching `RequestBodyMaxLength`,
// `limitedReqBuf` will start pretending writing so that
// io.TeeReader continues working while reqBuf stays unmodified.
limitedReqBuf := limitio.NewWriter(&reqBuf, RequestBodyMaxLength, true)

request.Body = &readCloser{
    Reader: io.TeeReader(request.Body, limitedReqBuf), 
    Closer: c.Request.Body,
}

LimitIO provides Reader, Writer and their Closer versions, for details, see docs.

Status: Beta

LimitIO is battling under production environment.

APIs are subjected to change in backward compatible way.

License

MIT License Copyright (c) 2021 LI Zhennan

Issues
  • Feature Request: Higher Limits

    Feature Request: Higher Limits

    Hey, could you implement your read limit as a int64 or uint64 instead of int32? I'd like to limit my streams because they get very big, so i would appreciate a high limit. Thanks in advance!

    documentation 
    opened by nettnikl 1
Releases(v1.0.0)
Owner
LI Zhennan
To build beautiful things beautifully.
LI Zhennan
Wrap byte read options with uniform interface for io.Reader and byte slice

nibbler Nibble chunks from Reader streams and slice in a common way Overview This is a golang module that provides an interface for treating a Reader

null 0 Dec 23, 2021
go-linereader: A small library for streaming lines from an io.Reader.

go-linereader: A small library for streaming lines from an io.Reader.

Conner Douglass 1 Oct 25, 2021
A program to create assembly 8086 strings to print without using any printing/strings related function but only mov-xchg-int and loops

Assembly String builder tool A program to create assembly 8086 strings to print without using any printing/strings related function but only mov-xchg-

Reg 2 Feb 1, 2022
An API that provides a small but well-thought service converting Euro to US Dollar and vice-versa

Currency Converter ###Problem An API that provides a small but well-thought service converting Euro to US Dollar and vice-versa. That API should only

Akshay Nambiar 0 Jan 30, 2022
Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package.

Goridge is high performance PHP-to-Golang codec library which works over native PHP sockets and Golang net/rpc package. The library allows you to call Go service methods from PHP with a minimal footprint, structures and []byte support.

Spiral Scout 1.1k Jun 24, 2022
A Go (golang) library for parsing and verifying versions and version constraints.

go-version is a library for parsing versions and version constraints, and verifying versions against a set of constraints. go-version can sort a collection of versions properly, handles prerelease/beta versions, can increment versions, etc.

HashiCorp 1.2k Jun 25, 2022
Golang: unify nil and empty slices and maps

unifynil, unify nil and empty slices and maps in Golang Empty slices and maps can be nil or not nil in Go. It may become a nightmare in tests and JSON

Boris Nagaev 0 Jan 16, 2022
memresolver is an in-memory golang resolver that allows to override current golang Lookup func literals

mem-resolver memresolver is an in-memory golang resolver that allows to override current golang Lookup func literals How to use it Create your custom

Antonio Ojea 3 Nov 25, 2021
Code Generation for Functional Programming, Concurrency and Generics in Golang

goderive goderive derives mundane golang functions that you do not want to maintain and keeps them up to date. It does this by parsing your go code fo

Walter Schulze 984 Jun 20, 2022
Copier for golang, copy value from struct to struct and more

Copier I am a copier, I copy everything from one to another Features Copy from field to field with same name Copy from method to field with same name

Jinzhu 3.2k Jun 25, 2022
Extremely flexible golang deep comparison, extends the go testing package, tests HTTP APIs and provides tests suite

go-testdeep Extremely flexible golang deep comparison, extends the go testing package. Latest news Synopsis Description Installation Functions Availab

Maxime Soulé 301 Jun 17, 2022
A well tested and comprehensive Golang statistics library package with no dependencies.

Stats - Golang Statistics Package A well tested and comprehensive Golang statistics library / package / module with no dependencies. If you have any s

Montana Flynn 2.5k Jun 26, 2022
Cogger is a standalone binary and a golang library that reads an internally tiled geotiff

Cogger is a standalone binary and a golang library that reads an internally tiled geotiff (optionally with overviews and masks) and rewrites it

Airbus DS GEO S.A. 70 Jun 8, 2022
A Golang and Python solution for Queue-it's Proof-of-Work challenge.

Queue-it Proof-of-Work A Golang and Python solution for Queue-it's Proof-of-Work challenge (https://queue-it.com/blog/proof-of-work-block-bad-bots/).

eta 22 Jun 3, 2022
⚙️ Concept of Golang HTML render engine with frontend components and dynamic behavior

SSC Engine An HTML render engine concept that brings frontend-like components experience to the server side with native html/template on steroids. Sup

Yurii Zinets 503 Jun 13, 2022
solution lock for golang, locallock and remote lock base on redis.

solution lock for golang, locallock and remote lock base on redis.

Te Nguyen 5 Feb 11, 2022
GoDynamic can load and run Golang dynamic library compiled by -buildmode=shared -linkshared

GoDynamic can load and run Golang dynamic library compiled by -buildmode=shared -linkshared How does it work? GoDynamic works like a dynamic

pkujhd 7 Apr 13, 2022
encLib is a simple golang package for quickly encoding and decoding string data in hex

encLib is a simple golang package for quickly encoding and decoding string data in hex

null 0 Nov 1, 2021
A golang library to validate and format swiss social security numbers

s3n is a golang library to validate and format swiss social security numbers (aka. AVS in french and AHV in german).

Julien M'Poy 0 Nov 15, 2021