LevelDB key/value database in Go.

Overview

This is an implementation of the LevelDB key/value database in the Go programming language.

Build Status

Installation

go get github.com/syndtr/goleveldb/leveldb

Requirements

  • Need at least go1.5 or newer.

Usage

Create or open a database:

// The returned DB instance is safe for concurrent use. Which mean that all
// DB's methods may be called concurrently from multiple goroutine.
db, err := leveldb.OpenFile("path/to/db", nil)
...
defer db.Close()
...

Read or modify the database content:

// Remember that the contents of the returned slice should not be modified.
data, err := db.Get([]byte("key"), nil)
...
err = db.Put([]byte("key"), []byte("value"), nil)
...
err = db.Delete([]byte("key"), nil)
...

Iterate over database content:

iter := db.NewIterator(nil, nil)
for iter.Next() {
	// Remember that the contents of the returned slice should not be modified, and
	// only valid until the next call to Next.
	key := iter.Key()
	value := iter.Value()
	...
}
iter.Release()
err = iter.Error()
...

Seek-then-Iterate:

iter := db.NewIterator(nil, nil)
for ok := iter.Seek(key); ok; ok = iter.Next() {
	// Use key/value.
	...
}
iter.Release()
err = iter.Error()
...

Iterate over subset of database content:

iter := db.NewIterator(&util.Range{Start: []byte("foo"), Limit: []byte("xoo")}, nil)
for iter.Next() {
	// Use key/value.
	...
}
iter.Release()
err = iter.Error()
...

Iterate over subset of database content with a particular prefix:

iter := db.NewIterator(util.BytesPrefix([]byte("foo-")), nil)
for iter.Next() {
	// Use key/value.
	...
}
iter.Release()
err = iter.Error()
...

Batch writes:

batch := new(leveldb.Batch)
batch.Put([]byte("foo"), []byte("value"))
batch.Put([]byte("bar"), []byte("another value"))
batch.Delete([]byte("baz"))
err = db.Write(batch, nil)
...

Use bloom filter:

o := &opt.Options{
	Filter: filter.NewBloomFilter(10),
}
db, err := leveldb.OpenFile("path/to/db", o)
...
defer db.Close()
...

Documentation

You can read package documentation here.

Issues
  • Require explicit general database closing.

    Require explicit general database closing.

    To be idiomatic and play nicely as a library, control of the database should exist solely in the hands of the library user and not in that of the memory manager.

    Major Caveat: A number of internal types around iterators and versions retain their finalizers for now. I need to take some time to diagram their lifecycle to propose a solution for this. This domain is where, I believe, the nondeterminism I found was, but I want to start fixing the easy part first.

    @syndtr, if you have some recommendations for how to approach this, I would love to talk with you. If you look at other LevelDB implementations, they explicitly require the user to call Iterator.Close to free associated resources, including the an official but incomplete implementation from Google in Go at https://code.google.com/p/leveldb-go/source/browse/leveldb/db/db.go#72.

    opened by matttproud 23
  • Slice bounds out of range error during compaction

    Slice bounds out of range error during compaction

    Here's the stackdump:

    panic: runtime error: slice bounds out of range [recovered] panic: runtime error: slice bounds out of range

    goroutine 50 [running]: github.com/syndtr/goleveldb/leveldb.func·015() /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:392 +0x172 github.com/syndtr/goleveldb/leveldb.iKey.ukey(0xc2006180df, 0x0, 0xa7, 0x0, 0xa7, ...) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/key.go:73 +0x75 github.com/syndtr/goleveldb/leveldb.(_iComparer).Compare(0xc20016a660, 0xc2006180df, 0x0, 0xa7, 0xc201207720, ...) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/comparer.go:21 +0x85 github.com/syndtr/goleveldb/leveldb/iterator.(_MergedIterator).smallest(0xc200afca50) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/iterator/merged_iter.go:178 +0x158 github.com/syndtr/goleveldb/leveldb/iterator.(_MergedIterator).Next(0xc200afca50, 0xc20061802d) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/iterator/merged_iter.go:113 +0x185 github.com/syndtr/goleveldb/leveldb.func·013(0x0, 0x0) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:247 +0x1cc github.com/syndtr/goleveldb/leveldb.(_DB).transact(0xc20012d180, 0x6c7a50) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:136 +0xaa github.com/syndtr/goleveldb/leveldb.(_DB).doCompaction(0xc20012d180, 0xc200172540, 0x6c7f00) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:365 +0x7e0 github.com/syndtr/goleveldb/leveldb.(_DB).compaction(0xc20012d180) /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:463 +0x1cb created by github.com/syndtr/goleveldb/leveldb.open /Users/sascha/Development/Go/projects/src/github.com/syndtr/goleveldb/leveldb/db.go:70 +0x1ec

    bug 
    opened by x6j8x 21
  • Possible memory leak

    Possible memory leak

    I'm using goleveldb to keep an index from certain kinds metadata to entries in my logfiles. I have four leveldbs for each day which are rotated at midnight. After the process is running several days the memory consumption keeps growing and growing. I'm using a write buffer of 16 Mbytes.

    Here's the output from lsof:

    process 30008 user 68uW REG 253,2 0 15310869 ./20140527-indexdb.ldb/servicenames/LOCK process 30008 user 70w REG 253,2 165 15310870 ./20140527-indexdb.ldb/servicenames/LOG process 30008 user 74w REG 253,2 54 15310871 ./20140527-indexdb.ldb/servicenames/MANIFEST-000000 process 30008 user 82w REG 253,2 8740127 15310873 ./20140527-indexdb.ldb/servicenames/000001.log process 30008 user 129uW REG 253,2 0 15310875 ./20140527-indexdb.ldb/servicemetadata/LOCK process 30008 user 130w REG 253,2 165 15310876 ./20140527-indexdb.ldb/servicemetadata/LOG process 30008 user 131w REG 253,2 54 15310877 ./20140527-indexdb.ldb/servicemetadata/MANIFEST-000000 process 30008 user 134w REG 253,2 361647 15310879 ./20140527-indexdb.ldb/servicemetadata/000001.log process 30008 user 146uW REG 253,2 0 15310851 ./20140527-indexdb.ldb/base/LOCK process 30008 user 153w REG 253,2 24820721 15310912 ./20140527-indexdb.ldb/transactions/000002.log process 30008 user 155w REG 253,2 165 15310852 ./20140527-indexdb.ldb/base/LOG process 30008 user 159w REG 253,2 54 15310853 ./20140527-indexdb.ldb/base/MANIFEST-000000 process 30008 user 161w REG 253,2 59070 15310855 ./20140527-indexdb.ldb/base/000001.log process 30008 user 164uW REG 253,2 0 15310857 ./20140527-indexdb.ldb/transactions/LOCK process 30008 user 171w REG 253,2 365 15310858 ./20140527-indexdb.ldb/transactions/LOG process 30008 user 175w REG 253,2 128 15310859 ./20140527-indexdb.ldb/transactions/MANIFEST-000000 process 30008 user 192uW REG 253,2 0 15310863 ./20140527-indexdb.ldb/message/LOCK process 30008 user 193w REG 253,2 166 15310864 ./20140527-indexdb.ldb/message/LOG process 30008 user 194w REG 253,2 54 15310865 ./20140527-indexdb.ldb/message/MANIFEST-000000 process 30008 user 196w REG 253,2 26185266 15310867 ./20140527-indexdb.ldb/message/000001.log

    Here's the output from pprof:

    screenshot-1

    bug 
    opened by x6j8x 20
  • Effort to reduce heap allocations

    Effort to reduce heap allocations

    Hi!

    I use goleveldb in syncthing (https://github.com/syncthing/syncthing) and have been doing a fair bit of profiling and hacking lately to reduce syncthing's memory footprint. I'm now at the point where goleveldb represents ~70-80 MB of a 100 MB footprint in my benchmarks. To a certain extent this is fair, with caches etc. But a large part of it is temporary heap allocations, in fact pretty much all of them made by snappy.Decode() from readRawBlock(). I made a naive attempt to reuse buffers with a sync.Pool in https://github.com/calmh/goleveldb/tree/reduce-heap, and it makes a dent but it's not huge. Probably the reason it's not very effective is that the pool isn't filled very often, and I'm not sure about the code paths that result in the data buffers becoming unused to hook into.

    This isn't a bug report as such, more a humble request to give it some work at some point, or pass me some hints and I'm happy to root around in it. :)

    enhancement 
    opened by calmh 18
  • Compaction time can be quite big

    Compaction time can be quite big

    Hey! I am investigation the performance decline in our software https://github.com/tendermint/tendermint/issues/1835. We're using goleveldb as a primary DB for storing blocks plus some other data.

    Looks like saving a batch of data (~1MB) can result in ~ 15 sec. due to compaction process or something:

    https://github.com/tendermint/tendermint/issues/1835#issuecomment-402054099 (the time seems to be increasing with every spike)

    The write speed is approximately 1MB per second.

    Is this expected and we need to fine tune compaction options or this looks like a bug to you?

    Thank you!

    opened by melekes 15
  • Possible corruption on the block data

    Possible corruption on the block data

    This could be caused by bug in the table writer or block buffer.

    Corruption on the block buffer could be caused by same buffer acquired more than once from buffer pool by two different block instance (or probably added to buffer pool more than once?).

    https://gist.github.com/syndtr/a97bef48b12df8181d6b

    Recovering the DB doesn't indicate any corruption but the corrupted table may already been deleted.

    Could be related to syncthing/syncthing#987.

    hardwareissue 
    opened by syndtr 13
  • db corruption on 'out of disk space'

    db corruption on 'out of disk space'

    Using btcd, which currently uses goleveldb on commit 92de35cffa094e330ee854d44ccbcb954e344d08, if disk space runs out (write /tmp/mainnet/blocks_leveldb/000054.log: no space left on device), btcd can be restarted without problem.

    After updating goleveldb to commit 62cc4d4e8b1ba35c2d1597e54892c022c376995f, goleveldb corrupts the database on the same error. When btcd starts back up, it says leveldb: bad record in batch

    bug 
    opened by dajohi 13
  • Significant memory usage when trying to insert data

    Significant memory usage when trying to insert data

    I am trying to profile the memory usage of my go program, and found that the majority of memory seems to be consumed within leveldb functions (github.com/syndtr/goleveldb/leveldb/memdb.(*DB).Put).

    Below is the top profile output:

    (pprof) top
    6815712.39kB of 6881450.49kB total (99.04%)
    Dropped 372 nodes (cum <= 34407.25kB)
    Showing top 10 nodes out of 28 (cum >= 242528.01kB)
          flat  flat%   sum%        cum   cum%
     6471904kB 94.05% 94.05%  6471904kB 94.05%  github.com/syndtr/goleveldb/leveldb/memdb.(*DB).Put
    242528.01kB  3.52% 97.57% 242528.01kB  3.52%  bytes.makeSlice
       55040kB   0.8% 98.37%    55040kB   0.8%  lib.FP.GenerateHashMap
    46004.76kB  0.67% 99.04% 54040.69kB  0.79%  main.main
      149.16kB 0.0022% 99.04% 6483852.34kB 94.22%  github.com/syndtr/goleveldb/leveldb.(*DB).Write
       68.67kB 0.001% 99.04% 319400.85kB  4.64%  main.Worker
       13.77kB 0.0002% 99.04% 6507694.47kB 94.57%  main.Worker1
        4.03kB 5.9e-05% 99.04% 6495736.37kB 94.39%  github.com/syndtr/goleveldb/leveldb.(*DB).Put
             0     0% 99.04%   196616kB  2.86%  bytes.(*Buffer).WriteString
             0     0% 99.04% 242528.01kB  3.52%  bytes.(*Buffer).grow
    
    bug 
    opened by xia0pin9 12
  • disable auto compaction

    disable auto compaction

    Is there a way to disable the auto db compaction goleveldb does? My db keeps being corrupted, and if it does panic, it is always in db_compaction.go

    I want to test filling my DB without goleveldb running compaction at all to see if it still corrupts.

    opened by dajohi 12
  • Compression change behind the scenes

    Compression change behind the scenes

    btcd (github.com/conformal/btcd) uses goleveldb to store data.

    The database is opened with:

            myCache := cache.NewEmptyCache()
            opts := &opt.Options{Flag: flag,
                    BlockCache:      myCache,
                    MaxOpenFiles:    256,
                    CompressionType: opt.NoCompression,
            }
    

    After a while, the database throws an error about a missing data.

    I added two fmt.Printfs to reader.go in readRawBlock()'s select case to print the type of Compression being used. I also added spew.Dump(bh) We have a tool to dump data.

    (table.blockHandle) {
     offset: (uint64) 1937402,
     length: (uint64) 4827
    }
    (uint8) 0
    No Compression
    

    On a different block:

    (table.blockHandle) {
     offset: (uint64) 0,
     length: (uint64) 22302
    }
    (uint8) 1
    Snappy Compression
    1381438195854721313 [Warn] unable to locate block sha 00000000000033640c6b34c40b8b95ba4fae759ec7970e546660a3abd9f9314c err snappy: unsupported COPY_4 tag
    1381438195854866024 [Info] Invalid block 00000000000033640c6b34c40b8b95ba4fae759ec7970e546660a3abd9f9314c
    

    And another:
    (table.blockHandle) {
     offset: (uint64) 1237318,
     length: (uint64) 3572
    }
    (uint8) 1
    Snappy Compression
    

    Yet another:

    (table.blockHandle) {
     offset: (uint64) 1119730,
     length: (uint64) 4331
    }
    (uint8) 0
    No Compression
    

    Any idea what is causing the compression to be changed? Multiple people are seeing this on different machines. I am the only one getting the COPY_4 tag issue.

    opened by dajohi 11
  • go test fail on Windows/386

    go test fail on Windows/386

    Hello,

    I saw you fixed #14 so I wanted to test your package and here what I got:

    go version
    go version devel +d29da2ced72b Mon May 13 10:28:12 2013 +0400 windows/386
    

    Windows 7 x32

    Error:

    D:\Golibs\os\src\github.com\syndtr\goleveldb\leveldb>go test
    fatal error: runtime: cannot map pages in arena address space
    
    goroutine 328 [running]:
    [fp=0x16c787c] runtime.throw(0x665341)
            c:/Go/src/pkg/runtime/panic.c:473 +0x65
    [fp=0x16c789c] runtime.SysMap(0x3ffa0000, 0x100000)
            c:/Go/src/pkg/runtime/mem_windows.c:70 +0x78
    [fp=0x16c78bc] runtime.MHeap_SysAlloc(0x1730000, 0x100000)
            c:/Go/src/pkg/runtime/zmalloc_windows_386.c:360 +0x103
    [fp=0x16c78e4] MHeap_Grow(0x1730000, 0x10)
            c:/Go/src/pkg/runtime/mheap.c:224 +0x4f
    [fp=0x16c7904] MHeap_AllocLocked(0x1730000, 0x4, 0x2b)
            c:/Go/src/pkg/runtime/mheap.c:108 +0x286
    [fp=0x16c7918] runtime.MHeap_Alloc(0x1730000, 0x4, 0x2b, 0x0, 0x1, ...)
            c:/Go/src/pkg/runtime/mheap.c:77 +0x5c
    [fp=0x16c7950] MCentral_Grow(0x1b35d4c)
            c:/Go/src/pkg/runtime/mcentral.c:203 +0x81
    [fp=0x16c7974] runtime.MCentral_AllocList(0x1b35d4c, 0xc, 0x16c7988)
            c:/Go/src/pkg/runtime/mcentral.c:47 +0x4c
    [fp=0x16c7994] runtime.MCache_Alloc(0x15d0000, 0x2b, 0x1400, 0x1)
            c:/Go/src/pkg/runtime/mcache.c:24 +0x72
    [fp=0x16c79c4] runtime.mallocgc(0x1400, 0x1, 0x1, 0x1)
            c:/Go/src/pkg/runtime/zmalloc_windows_386.c:47 +0xdf
    [fp=0x16c79e4] makeslice1(0x5021e0, 0x13e0, 0x13e0, 0x16c7a18)
            c:/Go/src/pkg/runtime/slice.c:61 +0x7f
    [fp=0x16c7a04] runtime.makeslice(0x5021e0, 0x13e0, 0x0, 0x13e0, 0x0, ...)
            c:/Go/src/pkg/runtime/slice.c:34 +0x9c
    [fp=0x16c7a58] github.com/syndtr/goleveldb/leveldb/table.(*bInfo).readAll(0x3f9feb80, 0x201818c0, 0x3f9b4a98, 0x20181800, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/table/block.go:66 +0x74
    [fp=0x16c7a90] github.com/syndtr/goleveldb/leveldb/table.(*Reader).getBlock(0x3f9bd0f0, 0x3f9feb80, 0x20181440, 0x3f9b4988, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:171 +0x8e
    [fp=0x16c7ac8] github.com/syndtr/goleveldb/leveldb/table.(*Reader).getDataIter(0x3f9bd0f0, 0x3f9feb80, 0x20181440, 0x3f9b4988, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:202 +0x174
    [fp=0x16c7afc] github.com/syndtr/goleveldb/leveldb/table.(*indexIter).Get(0x3f9b19c0, 0x0, 0x0, 0x0, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:234 +0xd0
    [fp=0x16c7b14] github.com/syndtr/goleveldb/leveldb/iterator.(*IndexedIterator).setData(0x3f9b19e0, 0x1)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/iterator/indexed_iter.go:138
    +0x2d
    [fp=0x16c7b20] github.com/syndtr/goleveldb/leveldb/iterator.(*IndexedIterator).Next(0x3f9b19e0, 0x3ff9d800)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/iterator/indexed_iter.go:85 +0x7f
    [fp=0x16c7b2c] github.com/syndtr/goleveldb/leveldb/iterator.(*IndexedIterator).Next(0x3f9b17e0, 0x4ba7c0)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/iterator/indexed_iter.go:84 +0x45
    [fp=0x16c7b70] github.com/syndtr/goleveldb/leveldb/iterator.(*MergedIterator).Next(0x3ea83ea0, 0x3fdb03a0)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/iterator/merged_iter.go:112 +0x124
    [fp=0x16c7c1c] github.com/syndtr/goleveldb/leveldb.func┬Ě015(0x0, 0x0)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:247 +0x172
    [fp=0x16c7c6c] github.com/syndtr/goleveldb/leveldb.(*DB).transact(0x200bd500, 0x16c7cac)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:136 +0x74
    [fp=0x16c7f2c] github.com/syndtr/goleveldb/leveldb.(*DB).doCompaction(0x200bd500, 0x3e997480, 0x16c7f00)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:365 +0x75d
    [fp=0x16c7fd4] github.com/syndtr/goleveldb/leveldb.(*DB).compaction(0x200bd500)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:463 +0x17b
    [fp=0x16c7fd8] runtime.goexit()
            c:/Go/src/pkg/runtime/proc.c:1223
    created by github.com/syndtr/goleveldb/leveldb.openDB
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db.go:72 +0x1e6
    
    goroutine 1 [chan receive]:
    testing.RunTests(0x58aa7c, 0x666dc0, 0x3c, 0x3c, 0x1, ...)
            c:/Go/src/pkg/testing/testing.go:434 +0x69e
    testing.Main(0x58aa7c, 0x666dc0, 0x3c, 0x3c, 0x665a60, ...)
            c:/Go/src/pkg/testing/testing.go:365 +0x68
    main.main()
            C:/Users/Dobek/AppData/Local/Temp/go-build124087311/github.com/syndtr/goleveldb/leveldb/_test/_testmain.go:199 +0x80
    
    goroutine 327 [chan send]:
    github.com/syndtr/goleveldb/leveldb.(*DB).flush(0x200bd500, 0x0, 0x0, 0x0)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_write.go:62 +0x176
    github.com/syndtr/goleveldb/leveldb.(*DB).Write(0x200bd500, 0x3ea85f20, 0x204cd010, 0x0, 0x0, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_write.go:109 +0x181
    github.com/syndtr/goleveldb/leveldb.(*DB).Put(0x200bd500, 0x3ea6ecb0, 0xb, 0xb, 0x3eaa6400, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_write.go:162 +0x84
    github.com/syndtr/goleveldb/leveldb.(*dbHarness).put(0x204f5480, 0x3ea6eca0, 0xb, 0x21ba8800, 0x3e8, ...)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_test.go:122 +0xb5
    github.com/syndtr/goleveldb/leveldb.TestDb_SparseMerge(0x2012b540)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_test.go:928 +0x165
    testing.tRunner(0x2012b540, 0x666f4c)
            c:/Go/src/pkg/testing/testing.go:353 +0x86
    created by testing.RunTests
            c:/Go/src/pkg/testing/testing.go:433 +0x683
    
    goroutine 329 [chan receive]:
    github.com/syndtr/goleveldb/leveldb.(*DB).writeJournal(0x200bd500)
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db_write.go:25 +0x3b
    created by github.com/syndtr/goleveldb/leveldb.openDB
            D:/Golibs/os/src/github.com/syndtr/goleveldb/leveldb/db.go:73 +0x1fb
    
    goroutine 7 [runnable]:
    exit status 2
    FAIL    github.com/syndtr/goleveldb/leveldb     15.997s
    

    Best regards, Dobrosław Żybort

    opened by matrixik 11
  • leveldb: fix table file leaks when manifest is rotated

    leveldb: fix table file leaks when manifest is rotated

    When switched to the master branch, the size of my project's result db is much larger than before. The size is shrunk when restarted and many logs [email protected] removing table-xxx printed. I found that it's caused by the recently merge PR #380.

    opened by qianbin 0
  • util: improve BufferPool, eliminate unexpected allocations

    util: improve BufferPool, eliminate unexpected allocations

    Using sync.Pool to manage bare slices is not the proper way, since it'll bring unexpected allocations. This PR solves this problem with small tricks and makes BufferPool a bit faster.

    benchmarks:

    $ benchstat before.txt after.txt 
    name          old time/op    new time/op    delta
    BufferPool-8    55.9ns ± 0%    34.8ns ± 0%   -37.71%  (p=0.008 n=5+5)
    
    name          old alloc/op   new alloc/op   delta
    BufferPool-8     24.0B ± 0%      0.0B       -100.00%  (p=0.008 n=5+5)
    
    name          old allocs/op  new allocs/op  delta
    BufferPool-8      1.00 ± 0%      0.00       -100.00%  (p=0.008 n=5+5)
    
    opened by qianbin 0
  • Browser indexeddb in golang / wasm

    Browser indexeddb in golang / wasm

    There is also an indexeddb implementation in golang here. It compiles to wasm and uses the browsers built in indexeddb.

    https://github.com/hack-pad/go-indexeddb

    it would be cool to setup tests to run against both implementations.

    opened by gedw99 0
  • memdb: merge nodeData and kvData to speedup access

    memdb: merge nodeData and kvData to speedup access

    This PR makes memdb save skiplist nodes and kvs together in one byte-slice. The performance of memdb is significantly improved with this change(it's believed that the CPU cache hit-rate goes up).

    Benchmarks:

    $ benchstat before.txt after.txt 
    name         old time/op  new time/op  delta
    Put-8        1.08µs ± 1%  1.00µs ± 1%   -7.98%  (p=0.008 n=5+5)
    PutRandom-8  1.85µs ± 1%  1.51µs ± 0%  -18.42%  (p=0.008 n=5+5)
    Get-8        1.24µs ± 1%  1.13µs ± 1%   -9.54%  (p=0.008 n=5+5)
    GetRandom-8  2.09µs ± 1%  1.69µs ± 1%  -19.17%  (p=0.008 n=5+5)
    
    opened by qianbin 1
Releases(v1.0.0)
Owner
Suryandaru Triandana
Suryandaru Triandana
Nipo is a powerful, fast, multi-thread, clustered and in-memory key-value database, with ability to configure token and acl on commands and key-regexes written by GO

Welcome to NIPO Nipo is a powerful, fast, multi-thread, clustered and in-memory key-value database, with ability to configure token and acl on command

Morteza Bashsiz 16 Jun 13, 2022
pure golang key database support key have value. 非常高效实用的键值数据库。

orderfile32 pure golang key database support key have value The orderfile32 is standard alone fast key value database. It have two version. one is thi

null 3 Apr 30, 2022
Simple key value database that use json files to store the database

KValDB Simple key value database that use json files to store the database, the key and the respective value. This simple database have two gRPC metho

Francisco Santos 0 Nov 13, 2021
levigo is a Go wrapper for LevelDB

levigo levigo is a Go wrapper for LevelDB. The API has been godoc'ed and is available on the web. Questions answered at [email protected]

Jeff Hodges 404 May 5, 2022
Persistent stacks and queues for Go backed by LevelDB

Goque Goque provides embedded, disk-based implementations of stack and queue data structures. Motivation for creating this project was the need for a

null 821 Jun 10, 2022
An embedded key/value database for Go.

bbolt bbolt is a fork of Ben Johnson's Bolt key/value store. The purpose of this fork is to provide the Go community with an active maintenance and de

etcd-io 5.7k Jun 30, 2022
BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support

BuntDB is a low-level, in-memory, key/value store in pure Go. It persists to disk, is ACID compliant, and uses locking for multiple readers and a sing

Josh Baker 3.8k Jun 24, 2022
ACID key-value database.

Coffer Simply ACID* key-value database. At the medium or even low latency it tries to provide greater throughput without losing the ACID properties of

Eduard 30 Jul 3, 2022
Graviton Database: ZFS for key-value stores.

Graviton Database: ZFS for key-value stores. Graviton Database is simple, fast, versioned, authenticated, embeddable key-value store database in pure

null 406 Jul 1, 2022
Membin is an in-memory database that can be stored on disk. Data model smiliar to key-value but values store as JSON byte array.

Membin Docs | Contributing | License What is Membin? The Membin database system is in-memory database smiliar to key-value databases, target to effici

Membin 3 Jun 3, 2021
An embedded, hardened key/value database for Go.

Bolt Bolt is a pure Go key/value store inspired by Howard Chu's LMDB project. The goal of the project is to provide a simple, fast, and reliable datab

null 0 Nov 4, 2021
BadgerDB is an embeddable, persistent and fast key-value (KV) database written in pure Go

BadgerDB BadgerDB is an embeddable, persistent and fast key-value (KV) database written in pure Go. It is the underlying database for Dgraph, a fast,

Blizard 1 Dec 10, 2021
GalaxyDB is a hobbyist key-value database written in Go.

GalaxyDB GalaxyDB is a hobbyist key-value database written in Go Author: Andrew N ([email protected]) Features Data is stored via keys Operations Grafana

nijmeh.xyz 3 Mar 30, 2022
Fast key-value DB in Go.

BadgerDB BadgerDB is an embeddable, persistent and fast key-value (KV) database written in pure Go. It is the underlying database for Dgraph, a fast,

Dgraph 11k Jun 26, 2022
🔑A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout (LSM+WAL) similar to Riak.

bitcask A high performance Key/Value store written in Go with a predictable read/write performance and high throughput. Uses a Bitcask on-disk layout

James Mills 7 Apr 15, 2022
A disk-backed key-value store.

What is diskv? Diskv (disk-vee) is a simple, persistent key-value store written in the Go language. It starts with an incredibly simple API for storin

Peter Bourgon 1.2k Jul 1, 2022
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

go-cache go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major

Patrick Mylund Nielsen 6.3k Jun 29, 2022
A simple, fast, embeddable, persistent key/value store written in pure Go. It supports fully serializable transactions and many data structures such as list, set, sorted set.

NutsDB English | 简体中文 NutsDB is a simple, fast, embeddable and persistent key/value store written in pure Go. It supports fully serializable transacti

徐佳军 2.3k Jun 26, 2022
Embedded key-value store for read-heavy workloads written in Go

Pogreb Pogreb is an embedded key-value store for read-heavy workloads written in Go. Key characteristics 100% Go. Optimized for fast random lookups an

Artem Krylysov 899 Jun 24, 2022