A binary stream packer and unpacker

Overview

binpacker Build Status GoDoc Go Report Card

A binary packer and unpacker.

Install

go get github.com/zhuangsirui/binpacker

Examples

Packer

buffer := new(bytes.Buffer)
packer := binpacker.NewPacker(binary.BigEndian, buffer)
packer.PushByte(0x01)
packer.PushBytes([]byte{0x02, 0x03})
packer.PushUint16(math.MaxUint16)
// You can push data like this
buffer := new(bytes.Buffer)
packer := binpacker.NewPacker(binary.BigEndian, buffer)
packer.PushByte(0x01).PushBytes([]byte{0x02, 0x03}).PushUint16(math.MaxUint16)
packer.Error() // Make sure error is nil

Unpacker

Example data

buffer := new(bytes.Buffer)
packer := binpacker.NewPacker(binary.BigEndian, buffer)
unpacker := binpacker.NewUnpacker(binary.BigEndian, buffer)
packer.PushByte(0x01)
packer.PushUint16(math.MaxUint16)
var val1 byte
var val2 uint16
var err error
val1, err = unpacker.ShiftByte()
val2, err = unpacker.ShiftUint16()
var val1 byte
var val2 uint16
var err error
unpacker.FetchByte(&val1).FetchUint16(&val2)
unpacker.Error() // Make sure error is nil
Comments
  • Explicitly set byte order in new

    Explicitly set byte order in new

    I spent all day to find this problem :disappointed:. It was absolutely not obvious. https://github.com/zhuangsirui/binpacker/commit/f1b53c477754cc3d873952fd6bc1b759aa01a464#commitcomment-24315680

    Byte order is only one important option when you need work with binary. There are sense to add it directly to signature.

    I propose explicitly add byte order to new functions. It's go way. It should be obvious.

    func NewPacker(writer io.Writer, byteOrder binary.ByteOrder) *Packer{}
    
    func NewUnpacker(reader io.Reader, byteOrder binary.ByteOrder) *Unpacker{}
    

    I thought about NewPackerWithOrder but it's not good idea. binpacker single purpose package and it should be concise. SetByteOrder would be unused.

    :warning:Comparability is broken. :warning: At least people will see immediately compilation error.

    opened by sturfee-petrl 5
  • Typos in StringWithUint16Perfix, StringWithUint32Perfix etc.

    Typos in StringWithUint16Perfix, StringWithUint32Perfix etc.

    Hello,

    The following functions have typos in them. They should end in Prefix, not Perfix

    • StringWithUint16Perfix
    • StringWithUint32Perfix
    • StringWithUint64Perfix
    • BytesWithUint16Perfix
    • BytesWithUint32Perfix
    • BytesWithUint64Perfix
    opened by jazz-sh 3
  • Semanting versioning

    Semanting versioning

    Hi! It was as 5 months when you proposed to use the semantic versioning. https://github.com/zhuangsirui/binpacker/issues/8#issuecomment-331052246

    We Sturfeeinc use your binpacker in critical part of our system. I had big problems at previous time when I didn't use dependency management tools (#8). Now we have to be more responsible because we launch API soon.

    Could you please to add version tags?

    Recently I added versions tags to my library UTM.

    Instruction

    I propose 08a1b297435a414bec3ccf4215ff546dba418159 as v1.0.0

    git checkout 08a1b297435a414bec3ccf4215ff546dba418159 .

    Please change a message if you don't like it.

    git tag -a v1.0.0 -m "Library was used in production for long time. We found one serious error. It considered stable now."

    git push origin refs/tags/v1.0.0

    opened by im7mortal 2
  • Fix issue when io.Reader returns a partial read

    Fix issue when io.Reader returns a partial read

    First of all, thanks for great open source project. We use binpacker to unpack length prefixed data sent over network (using BytesWithUint32Prefix).

    We found our data corrupted when we would be unpacking longer messages. It turned out to be an issue when io.Reader would return a partial read. The issue was caused here: buffer.Write(buf). If a read was smaller than allocated buf, data would be padded with a bunch of 0s at the end and all of it written to the final output buffer. The issue can be solved with a change to buffer.Write(buf[:rn]) but after I examined the method I thought it was better to leverage golang standard library instead. The bytes.Buffer usage was unnecessary since we know the final size in advance and we read it all anyway.

    Let me know if you see any issues and I'll be happy to tweak this PR.

    opened by pranas 1
  • a typo in util.go

    a typo in util.go

    I believe there is a typo in util.go, I assume you've meant "prefixes" instead of "perfix"?

    P.S. Thanks, this is a useful library, helps to avoid writing boilerplate.

    opened by agubarev 1
  • support for int64

    support for int64

    I see there is support for packing uint64. It will be really great if you could support int64.

    May be there is a way to do it already and it is not documented? I see that you have worked on erlang :) How about implementing something like iolist_to_binary

    opened by evnix 1
  • gofmt

    gofmt

    Just ran

    gofmt -w .
    

    on the project root. That's all.

    https://blog.golang.org/go-fmt-your-code


    I made this PR with a project going on over at https://github.com/rotblauer/gofmt-att, and it's definitely a work in progress. So if I got something wrong, or this is annoying at all, please file an issue over there and we'll sort it out.

    opened by whilei 1
Releases(v1.2.0-beta.1)
Owner
Zhuang Sirui
Ruby, Rails, Golang
Zhuang Sirui
A Go implementation of a radix tree, that uses binary searches to speed up insert, retrieve and delete operations on dense trees

radixs A Go implementation of a radix tree, that uses binary searches to speed up insert, retrieve and delete operations on dense trees. This implemen

Bruno Moura 0 Feb 14, 2022
A Left-Leaning Red-Black (LLRB) implementation of balanced binary search trees for Google Go

GoLLRB GoLLRB is a Left-Leaning Red-Black (LLRB) implementation of 2-3 balanced binary search trees in Go Language. Overview As of this writing and to

Petar Maymounkov 742 Nov 21, 2022
Simple dense bitmap index in Go with binary operators

This package contains a bitmap index which is backed by uint64 slice, easily encodable to/from a []byte without copying memory around so it can be present in both disk and memory. As opposed to something as roaring bitmaps, this is a simple implementation designed to be used for small to medium dense collections.

Roman Atachiants 184 Nov 20, 2022
Golang channel example with equivalent binary tree

golang_channel_example_with_equivalent_binary_tree Exercise: Equivalent Binary Trees There can be many different binary trees with the same sequence o

web_developer 1 Oct 9, 2021
A project that deals with implementations of a binary tree

Binary Search Tree This is a project that deals with implementations of a binary tree and the following functions. Print Prints the entire tree. Argum

Victor Zeddys 1 Nov 1, 2021
A slice backed binary heap with support for generic type parameters.

go-binaryheap A slice backed binary heap where the order can be customized by a comparison function. The main branch now requires go 1.18 because the

null 3 Jun 13, 2022
estruct traverses javascript projects and maps all the dependencies and relationships to a JSON. the output can be used to build network visualizations of the project and document the architecture.

EStruct traverses javascript projects and maps all the dependencies and relationships to a JSON. The output can be used to build network visualizations of the project and document the architecture.

Ray Luxembourg 11 Jan 27, 2022
Golang string comparison and edit distance algorithms library, featuring : Levenshtein, LCS, Hamming, Damerau levenshtein (OSA and Adjacent transpositions algorithms), Jaro-Winkler, Cosine, etc...

Go-edlib : Edit distance and string comparison library Golang string comparison and edit distance algorithms library featuring : Levenshtein, LCS, Ham

Hugo Bollon 365 Nov 20, 2022
Go package for mapping values to and from space-filling curves, such as Hilbert and Peano curves.

Hilbert Go package for mapping values to and from space-filling curves, such as Hilbert and Peano curves. Documentation available here This is not an

Google 255 Sep 26, 2022
Levenshtein distance and similarity metrics with customizable edit costs and Winkler-like bonus for common prefix.

A Go package for calculating the Levenshtein distance between two strings This package implements distance and similarity metrics for strings, based o

AGExt 73 Nov 20, 2022
Decode / encode XML to/from map[string]interface{} (or JSON); extract values with dot-notation paths and wildcards. Replaces x2j and j2x packages.

mxj - to/from maps, XML and JSON Decode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-

Charles Banning 533 Nov 22, 2022
Dasel - Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool.

Select, put and delete data from JSON, TOML, YAML, XML and CSV files with a single tool. Supports conversion between formats and can be used as a Go package.

Tom Wright 3.8k Nov 18, 2022
💯 Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving

Package validator implements value validations for structs and individual fields based on tags.

Flamego 10 Nov 9, 2022
Go translations of the algorithms and clients in the textbook Algorithms, 4th Edition by Robert Sedgewick and Kevin Wayne.

Overview Go translations of the Java source code for the algorithms and clients in the textbook Algorithms, 4th Edition by Robert Sedgewick and Kevin

youngzy 172 Nov 24, 2022
Package iter provides generic, lazy iterators, functions for producing them from primitive types, as well as functions and methods for transforming and consuming them.

iter Package iter provides generic, lazy iterators, functions for producing them from primitive types, as well as functions and methods for transformi

Matthew Toohey 26 Sep 29, 2022
Snackbox - Snackbox can make it easier for customers to order snacks and rice boxes and do tracking

Catering Ecommerce Platform API Docs · Wireflow · Use Case Diagram · Entity Rela

Muhammad Arif Furqon 1 Oct 18, 2022
An app with Trie tree and Breve search Implementation CLI and HTTP both 🥳

Introduction LifeLongLearner project consists of two different parts. My English Vocabulary My Technical Book Notes All of them provided by me within

A.Samet İleri 15 Jul 1, 2022
A collection of useful, performant, and threadsafe Go datastructures.

go-datastructures Go-datastructures is a collection of useful, performant, and threadsafe Go datastructures. NOTE: only tested with Go 1.3+. Augmented

Workiva 6.7k Nov 23, 2022
Go native library for fast point tracking and K-Nearest queries

Geo Index Geo Index library Overview Splits the earth surface in a grid. At each cell we can store data, such as list of points, count of points, etc.

Hailo Network IP Ltd 342 Nov 19, 2022