Optimized bit-level Reader and Writer for Go.

Overview

bitio

Build Status GoDoc Go Report Card codecov

Package bitio provides an optimized bit-level Reader and Writer for Go.

You can use Reader.ReadBits() to read arbitrary number of bits from an io.Reader and return it as an uint64, and Writer.WriteBits() to write arbitrary number of bits of an uint64 value to an io.Writer.

Both Reader and Writer also provide optimized methods for reading / writing 1 bit of information in the form of a bool value: Reader.ReadBool() and Writer.WriteBool(). These make this package ideal for compression algorithms that use Huffman coding for example, where decision whether to step left or right in the Huffman tree is the most frequent operation.

Reader and Writer give a bit-level view of the underlying io.Reader and io.Writer, but they also provide a byte-level view (io.Reader and io.Writer) at the same time. This means you can also use the Reader.Read() and Writer.Write() methods to read and write slices of bytes. These will give you best performance if the underlying io.Reader and io.Writer are aligned to a byte boundary (else all the individual bytes are assembled from / spread to multiple bytes). You can ensure byte boundary alignment by calling the Align() method of Reader and Writer. As an extra, io.ByteReader and io.ByteWriter are also implemented.

Bit order

The more general highest-bits-first order is used. So for example if the input provides the bytes 0x8f and 0x55:

HEXA    8    f     5    5
BINARY  1100 1111  0101 0101
        aaaa bbbc  ccdd dddd

Then ReadBits will return the following values:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a, err := r.ReadBits(4) //   1100 = 0x08
b, err := r.ReadBits(3) //    111 = 0x07
c, err := r.ReadBits(3) //    101 = 0x05
d, err := r.ReadBits(6) // 010101 = 0x15

Writing the above values would result in the same sequence of bytes:

b := &bytes.Buffer{}
w := NewWriter(b)
err := w.WriteBits(0x08, 4)
err = w.WriteBits(0x07, 3)
err = w.WriteBits(0x05, 3)
err = w.WriteBits(0x15, 6)
err = w.Close()
// b will hold the bytes: 0x8f and 0x55

Error handling

All ReadXXX() and WriteXXX() methods return an error which you are expected to handle. For convenience, there are also matching TryReadXXX() and TryWriteXXX() methods which do not return an error. Instead they store the (first) error in the Reader.TryError / Writer.TryError field which you can inspect later. These TryXXX() methods are a no-op if a TryError has been encountered before, so it's safe to call multiple TryXXX() methods and defer the error checking.

For example:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a := r.TryReadBits(4) //   1100 = 0x08
b := r.TryReadBits(3) //    111 = 0x07
c := r.TryReadBits(3) //    101 = 0x05
d := r.TryReadBits(6) // 010101 = 0x15
if r.TryError != nil {
    // Handle error
}

This allows you to easily convert the result of individual ReadBits(), like this:

r := NewReader(bytes.NewBuffer([]byte{0x8f, 0x55}))
a := byte(r.TryReadBits(4))   //   1100 = 0x08
b := int32(r.TryReadBits(3))  //    111 = 0x07
c := int64(r.TryReadBits(3))  //    101 = 0x05
d := uint16(r.TryReadBits(6)) // 010101 = 0x15
if r.TryError != nil {
    // Handle error
}

And similarly:

b := &bytes.Buffer{}
w := NewWriter(b)
w.TryWriteBits(0x08, 4)
w.TryWriteBits(0x07, 3)
w.TryWriteBits(0x05, 3)
w.TryWriteBits(0x15, 6)
if w.TryError != nil {
    // Handle error
}
err = w.Close()
// b will hold the bytes: 0x8f and 0x55
Issues
  • New Reader convenience methods for reading into {int, uint8, uint16, uint32} without caller type conversion.

    New Reader convenience methods for reading into {int, uint8, uint16, uint32} without caller type conversion.

    These methods significantly simplified my code, when parsing a bitstream into a struct.

    It extends the API, and does not modify existing method signatures.

    opened by jpap 9
  • Flush to output to file

    Flush to output to file

    How can I call Flush so that my file gets written?

    func WriteToFile() {
    	// open output file
    	fo, err := os.Create("output.txt")
    	if err != nil {
    		panic(err)
    	}
    	// close fo on exit and check for its returned error
    	defer func() {
    		if err := fo.Close(); err != nil {
    			panic(err)
    		}
    	}()
    	// make a write buffer
    	w := bitio.NewWriter(fo)
    	err = w.WriteBits(0x08, 8)
    	if err != nil {
    		panic(err)
    	}
    //????
    	//w.Flush()
    	err = w.Close()
    	if err != nil {
    		panic(err)
    	}
    }
    
    opened by telemmaite 7
  • Support querying total number of bits read/written

    Support querying total number of bits read/written

    This will enhance the bitio reader/writer by a function to query the total number of bits read/written. This is useful is users want to put/parse data in fixed offsets from each other.

    opened by Danaozhong 6
  • Use uint8 rather than byte

    Use uint8 rather than byte

    I was confused for a second with n being of type byte. Using uint8 might help to distinguish between unsigned integer vs. a byte value.

    func (r *Reader) ReadBits(n byte) (u uint64, err error) would become func (r *Reader) ReadBits(n uint8) (u uint64, err error)

    opened by saracen 3
Owner
András Belicza
Earn $100 and help me by using my DigitalOcean ref link: https://m.do.co/c/7bdc66c1fe73
András Belicza
A minimalist Go PDF writer in 1982 lines. Draws text, images and shapes. Helps understand the PDF format. Used in production for reports.

one-file-pdf - A minimalist PDF generator in <2K lines and 1 file The main idea behind this project was: "How small can I make a PDF generator for it

Ali Bala 444 Jul 15, 2022
A modification (and a bit of simplification) of the tracerr package.

Decrr A modification (and a bit of simplification) of the tracerr package. This essentially does pretty much the same, but instead of returning anothe

Reinaldy Rafli 5 Nov 24, 2021
Print debugging, but a little bit nicer

testlog Print debugging, but a little bit nicer. The use case this is primarily designed for is effectively debugging problematic, flaky tests.

Roman Volosatovs 5 Oct 11, 2021
other glyph sets for high-dpi 1-bit monochrome

hd1b_other other glyph sets for high-dpi 1-bit monochrome Currently included glyph sets: Hangul (Korean): U+1100..U+11FF, U+3131..U+318E, U+AC00..U+D7

Sam Blenny 0 Jan 13, 2022
Eightbit - A converter to create shitty 8-bit like images

eightbit A converter to create shitty 8-bit like images. Usage To install: go in

Jeff Palm 0 Jan 8, 2022
Minimalist and opinionated feed reader

Miniflux 2 Miniflux is a minimalist and opinionated feed reader: Written in Go (Golang) Works only with Postgresql Doesn't use any ORM Doesn't use any

Miniflux 3.8k Aug 2, 2022
Load environment variables from `.env` or `io.Reader` in Go.

gotenv Load environment variables from .env or io.Reader in Go. Usage Put the gotenv package on your import statement: import "github.com/subosito/got

Alif Rachmawadi 225 Jul 14, 2022
wkhtmltopdf Go bindings and high level interface for HTML to PDF conversion

wkhtmltopdf Go bindings and high level interface for HTML to PDF conversion. Implements wkhtmltopdf Go bindings. It can be used to convert HTML docume

Adrian-George Bostan 119 Jul 24, 2022
User level X Keyboard Grabber

xkg - X Keyboard Grabber Installation go get gopkg.in/xkg.v0 Usage example: package main import ( "fmt" "gopkg.in/xkg.v0" ) func main() { var ke

null 53 Mar 5, 2022
Assembly syntax that makes you feel like you're writing code in a high-level language.

shasm Assembly syntax that makes you feel like you're writing code in a high-level language. Shasm is not an Assembler. Shasm simply compiles Shasm sy

Shoyaaa 14 Jun 5, 2021
[TOOL, CLI] - Filter and examine Go type structures, interfaces and their transitive dependencies and relationships. Export structural types as TypeScript value object or bare type representations.

typex Examine Go types and their transitive dependencies. Export results as TypeScript value objects (or types) declaration. Installation go get -u gi

Daniel T. Gorski 151 Jul 5, 2022
:chart_with_upwards_trend: Monitors Go MemStats + System stats such as Memory, Swap and CPU and sends via UDP anywhere you want for logging etc...

Package stats Package stats allows for gathering of statistics regarding your Go application and system it is running on and sent them via UDP to a se

Go Playgound 162 Jul 17, 2022
James is your butler and helps you to create, build, debug, test and run your Go projects

go-james James is your butler and helps you to create, build, debug, test and run your Go projects. When you often create new apps using Go, it quickl

Pieter Claerhout 53 Aug 1, 2022
GoThanks automatically stars Go's official repository and your go.mod github dependencies, providing a simple way to say thanks to the maintainers of the modules you use and the contributors of Go itself.

Give thanks (in the form of a GitHub ★) to your fellow Go modules maintainers. About GoThanks performs the following operations Sends a star to Go's r

psampaz 112 Jun 12, 2022
A simple Cron library for go that can execute closures or functions at varying intervals, from once a second to once a year on a specific date and time. Primarily for web applications and long running daemons.

Cron.go This is a simple library to handle scheduled tasks. Tasks can be run in a minimum delay of once a second--for which Cron isn't actually design

Robert K 210 Jul 27, 2022
Library to work with MimeHeaders and another mime types. Library support wildcards and parameters.

Mime header Motivation This library created to help people to parse media type data, like headers, and store and match it. The main features of the li

Anton Ohorodnyk 28 Jul 25, 2022
The new home of the CUE language! Validate and define text-based and dynamic configuration

The CUE Data Constraint Language Configure, Unify, Execute CUE is an open source data constraint language which aims to simplify tasks involving defin

null 2.8k Aug 1, 2022
Hack this repo and add your name to the list above. Creativity and style encouraged in both endeavors.

Hack this repo and add your name to the list above. Creativity and style encouraged in both endeavors.

Danger 2 Oct 1, 2021
A comprehensive training, nutrition, and social platform for martial artists and combat sport athletes

COMHRAC Comhrac (Gaelic for "Combat") is a comprehensive training, nutrition, and social platform for martial artists and combat sport athletes. Devel

Jack Hegarty 0 Oct 17, 2021