grocksdb, RocksDB wrapper for Go

Overview

grocksdb, RocksDB wrapper for Go

Go Report Card Coverage Status godoc

This is a Fork from tecbot/gorocksdb. I respect the author work and community contribution. The LICENSE still remains as upstream.

Why I made a patched clone instead of PR:

  • Supports almost C API (unlike upstream). Catching up with latest version of Rocksdb as promise.
  • This fork contains no defer in codebase (my side project requires as less overhead as possible). This introduces loose convention of how/when to free c-mem, thus break the rule of tecbot/gorocksdb.

Install

Prerequisite

  • librocksdb
  • libsnappy
  • libz
  • liblz4
  • libzstd

Please follow this guide: https://github.com/facebook/rocksdb/blob/master/INSTALL.md to build above libs.

Build

After that, you can install and build grocksdb using the following commands:

go get -u github.com/linxGnu/grocksdb

CGO_CFLAGS="-I/path/to/rocksdb/include" \
CGO_LDFLAGS="-L/path/to/rocksdb -lrocksdb -lstdc++ -lm -lz -lsnappy -llz4 -lzstd" \
  go build

Or just:

go build // if prerequisites are in linker paths

Usage

See also: doc

Builtin Static

grocksdb bundles static version of RocksDB, build with env:

  • centos 7 x86_64
  • gcc 4.8

You could give it a try:

go get -u github.com/linxGnu/grocksdb

go build -tags builtin_static

API Support

Almost C API, excepts:

  • putv/mergev/deletev/delete_rangev
  • compaction_filter/compaction_filter_factory/compaction_filter_context
Issues
  • 1.6.34 build scripts.

    1.6.34 build scripts.

    Build is failing and package ignores -tags builtin_shared and defaults to only ever using the prebuilt libs in ./dist

    Why are binaries included in the SOURCE package?

    The libs included are compiled for a system that has AVX2 support, meaning that when used on any system that doesn't have AVX2, we get "Illegal Instruction" errors.

    opened by xrpdevs 19
  • Compiler error on arm64 / rpi

    Compiler error on arm64 / rpi

    I´m trying to compile the project mentioned in #28 on a raspberry Pi4 running ubuntu 20.10 arm64.

    The compiler interrupts with

    /root/go/pkg/mod/github.com/linx!gnu/[email protected]/backup.go:44:13: could not determine kind of name for C.rocksdb_backup_engine_open_opts /root/go/pkg/mod/github.com/linx!gnu/[email protected]/backup.go:136:2: could not determine kind of name for C.rocksdb_backup_engine_restore_db_from_backup

    Do you have any idea how to get around this?

    opened by svenger87 12
  • Iterator works weird with SetIterateUpperBound and 10.000 keys

    Iterator works weird with SetIterateUpperBound and 10.000 keys

    I can't test iterator with 10.000 keys. I try to run iteration several times on them, using SetIterateUpperBound in ReadOptions.

    Expected behavior

    Successful logs

    /go/src/github.com/fubss/grocksdb-test-2$ GOFLAGS="-count=1" go test -timeout 30s -run ^TestDrop$ github.com/fubss/grocksdb-test-2 -v === RUN TestDrop -------->NewProvider intialization... -------->Opening DB and checking format... -------->RocksDB constructing... -------->Opening DB... -------->ParanoidChecks is: true -------->DB was successfully opened in path: [ /tmp/tests/grocksdbtest2 ] -------->Checkin if DB is empty... -------->Checking for emptiness has finished -------->rocks db IsEmpty()=true -------->Getting key [_f] from RocksDB... -------->got data [] -------->Checking for db format at path [/tmp/tests/grocksdbtest2] format is latest, nothing to do provider_test.go:69: expextedSetup_0 endKey is nil -------->endKey is nil: eKey=[db1([]byte{0x64, 0x62, 0x31, 0x1})] -------->Constructing iterator with sKey=[db1([]byte{0x64, 0x62, 0x31, 0x0})] and eKey=[db1([]byte{0x64, 0x62, 0x31, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db1 ([100 98 49 0]: [db1 ([100 98 49 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:69: expextedSetup_1 endKey is nil -------->endKey is nil: eKey=[db2([]byte{0x64, 0x62, 0x32, 0x1})] -------->Constructing iterator with sKey=[db2([]byte{0x64, 0x62, 0x32, 0x0})] and eKey=[db2([]byte{0x64, 0x62, 0x32, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db2 ([100 98 50 0]: [db2 ([100 98 50 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:69: expextedSetup_2 endKey is nil -------->endKey is nil: eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Constructing iterator with sKey=[db3([]byte{0x64, 0x62, 0x33, 0x0})] and eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db3 ([100 98 51 0]: [db3 ([100 98 51 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:69: expextedSetup_3 endKey is nil -------->endKey is nil: eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Constructing iterator with sKey=[db3([]byte{0x64, 0x62, 0x33, 0x0})] and eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db3 ([100 98 51 0]: [db3 ([100 98 51 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:69: expextedSetup_4 endKey is nil -------->endKey is nil: eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Constructing iterator with sKey=[db3([]byte{0x64, 0x62, 0x33, 0x0})] and eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db3 ([100 98 51 0]: [db3 ([100 98 51 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore endKey is nil -------->endKey is nil: eKey=[db1([]byte{0x64, 0x62, 0x31, 0x1})] -------->Constructing iterator with sKey=[db1([]byte{0x64, 0x62, 0x31, 0x0})] and eKey=[db1([]byte{0x64, 0x62, 0x31, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db1 ([100 98 49 0]: [db1 ([100 98 49 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid -------->WritingBatch.Count()=[20] endKey is nil -------->endKey is nil: eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Constructing iterator with sKey=[db3([]byte{0x64, 0x62, 0x33, 0x0})] and eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db3 ([100 98 51 0]: [db3 ([100 98 51 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid -------->WritingBatch.Count()=[8404] -------->Have removed 8404 entries for channel db3 in rocksdb /tmp/tests/grocksdbtest2 -------->WritingBatch.Count()=[1596] provider_test.go:106: expextedResults_0 endKey is nil -------->endKey is nil: eKey=[db1([]byte{0x64, 0x62, 0x31, 0x1})] -------->Constructing iterator with sKey=[db1([]byte{0x64, 0x62, 0x31, 0x0})] and eKey=[db1([]byte{0x64, 0x62, 0x31, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db1 ([100 98 49 0]: [db1 ([100 98 49 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is not Valid -------->iterator is not valid anymore provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:106: expextedResults_1 endKey is nil -------->endKey is nil: eKey=[db2([]byte{0x64, 0x62, 0x32, 0x1})] -------->Constructing iterator with sKey=[db2([]byte{0x64, 0x62, 0x32, 0x0})] and eKey=[db2([]byte{0x64, 0x62, 0x32, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db2 ([100 98 50 0]: [db2 ([100 98 50 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:106: expextedResults_2 endKey is nil -------->endKey is nil: eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Constructing iterator with sKey=[db3([]byte{0x64, 0x62, 0x33, 0x0})] and eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db3 ([100 98 51 0]: [db3 ([100 98 51 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is not Valid -------->iterator is not valid anymore provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore -------->Closing db... endKey is nil -------->endKey is nil: eKey=[db2([]byte{0x64, 0x62, 0x32, 0x1})] -------->Constructing iterator with sKey=[db2([]byte{0x64, 0x62, 0x32, 0x0})] and eKey=[db2([]byte{0x64, 0x62, 0x32, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db2 ([100 98 50 0]: [db2 ([100 98 50 1])] -------->itr.Err()=[error while obtaining db iterator: rocksdb: closed github.com/fubss/grocksdb-test-2.(*DB).GetIterator /home/ivanl/go/src/github.com/fubss/grocksdb-test-2/helper.go:151 github.com/fubss/grocksdb-test-2.(*DBHandle).GetIterator /home/ivanl/go/src/github.com/fubss/grocksdb-test-2/provider.go:263 github.com/fubss/grocksdb-test-2.(*DBHandle).deleteAll /home/ivanl/go/src/github.com/fubss/grocksdb-test-2/provider.go:168 github.com/fubss/grocksdb-test-2.TestDrop /home/ivanl/go/src/github.com/fubss/grocksdb-test-2/provider_test.go:115 testing.tRunner /usr/local/go/src/testing/testing.go:1259 runtime.goexit /usr/local/go/src/runtime/asm_amd64.s:1581]. Impossible to create an iterator -------->Error! Closing iterator... --- PASS: TestDrop (0.39s) PASS ok github.com/fubss/grocksdb-test-2 0.406s

    Actual behavior

    Sometimes test passes, sometimes not. When it doesn't, iterator becomes invalid in unexpected (every time new) place during iteration within 10.000 keys. itr.Err() returns nil in this case.

    Example logs of wrong behavior

    /go/src/github.com/fubss/grocksdb-test-2$ GOFLAGS="-count=1" go test -timeout 30s -run ^TestDrop$ github.com/fubss/grocksdb-test-2 -v === RUN TestDrop -------->NewProvider intialization... -------->Opening DB and checking format... -------->RocksDB constructing... -------->Opening DB... -------->ParanoidChecks is: true -------->DB was successfully opened in path: [ /tmp/tests/grocksdbtest2 ] -------->Checkin if DB is empty... -------->Checking for emptiness has finished -------->rocks db IsEmpty()=true -------->Getting key [_f] from RocksDB... -------->got data [] -------->Checking for db format at path [/tmp/tests/grocksdbtest2] format is latest, nothing to do provider_test.go:69: expextedSetup_0 endKey is nil -------->endKey is nil: eKey=[db1([]byte{0x64, 0x62, 0x31, 0x1})] -------->Constructing iterator with sKey=[db1([]byte{0x64, 0x62, 0x31, 0x0})] and eKey=[db1([]byte{0x64, 0x62, 0x31, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db1 ([100 98 49 0]: [db1 ([100 98 49 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:69: expextedSetup_1 endKey is nil -------->endKey is nil: eKey=[db2([]byte{0x64, 0x62, 0x32, 0x1})] -------->Constructing iterator with sKey=[db2([]byte{0x64, 0x62, 0x32, 0x0})] and eKey=[db2([]byte{0x64, 0x62, 0x32, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db2 ([100 98 50 0]: [db2 ([100 98 50 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:69: expextedSetup_2 endKey is nil -------->endKey is nil: eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Constructing iterator with sKey=[db3([]byte{0x64, 0x62, 0x33, 0x0})] and eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db3 ([100 98 51 0]: [db3 ([100 98 51 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:69: expextedSetup_3 endKey is nil -------->endKey is nil: eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Constructing iterator with sKey=[db3([]byte{0x64, 0x62, 0x33, 0x0})] and eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db3 ([100 98 51 0]: [db3 ([100 98 51 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] -------->iterator is not valid anymore provider_test.go:69: expextedSetup_4 endKey is nil -------->endKey is nil: eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Constructing iterator with sKey=[db3([]byte{0x64, 0x62, 0x33, 0x0})] and eKey=[db3([]byte{0x64, 0x62, 0x33, 0x1})] -------->Getting new RocksDB Iterator... for start key: [db3 ([100 98 51 0]: [db3 ([100 98 51 1])] -------->if-case: endKey!=nil, UpperBound set -------->PurgeOnIterCleanup = true -------->ni is not Valid, err=[] -------->itr is Valid provider_test.go:125: Iterator error is: [%!s()] provider_test.go:129: Error Trace: provider_test.go:129 provider_test.go:72 Error: Not equal: expected: 9999 actual : 6356 Test: TestDrop --- FAIL: TestDrop (0.27s) FAIL FAIL github.com/fubss/grocksdb-test-2 0.280s FAIL /go/src/github.com/fubss/grocksdb-test-2$

    Steps to reproduce the behavior

    1. go get my test git repo
    2. run GOFLAGS="-count=1" go test -timeout 30s -run ^TestDrop$ github.com/fubss/grocksdb-test-2 -v

    Additional information

    I'm using Ubuntu 20 LTS.

    go env

    GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/home/ivanl/.cache/go-build" GOENV="/home/ivanl/.config/go/env" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/home/ivanl/go/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/home/ivanl/go" GOPRIVATE="" GOPROXY="https://proxy.golang.org,direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64" GOVCS="" GOVERSION="go1.17.3" GCCGO="gccgo" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="/home/ivanl/go/src/github.com/fubss/grocksdb-test-2/go.mod" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2854272429=/tmp/go-build -gno-record-gcc-switches"

    my RocksDB LOG file

    2021/12/15-16:23:43.954005 7f39f45b2ac0 [/db_impl/db_impl_open.cc:184] file size check will be skipped during open. 2021/12/15-16:23:43.954290 7f39f45b2ac0 RocksDB version: 6.22.1 2021/12/15-16:23:43.954304 7f39f45b2ac0 Git sha 51b540921dd7495c9cf2265eb58942dad1f2ef72 2021/12/15-16:23:43.954307 7f39f45b2ac0 Compile date 2021-12-04 14:52:07 2021/12/15-16:23:43.954316 7f39f45b2ac0 DB SUMMARY 2021/12/15-16:23:43.954317 7f39f45b2ac0 DB Session ID: 6449I48JEUTH264QRR4L 2021/12/15-16:23:43.954347 7f39f45b2ac0 SST files in /tmp/tests/grocksdbtest2 dir, Total Num: 0, files: 2021/12/15-16:23:43.954351 7f39f45b2ac0 Write Ahead Log file in /tmp/tests/grocksdbtest2: 2021/12/15-16:23:43.954353 7f39f45b2ac0 Options.error_if_exists: 0 2021/12/15-16:23:43.954355 7f39f45b2ac0 Options.create_if_missing: 1 2021/12/15-16:23:43.954357 7f39f45b2ac0 Options.paranoid_checks: 0 2021/12/15-16:23:43.954359 7f39f45b2ac0 Options.flush_verify_memtable_count: 1 2021/12/15-16:23:43.954360 7f39f45b2ac0 Options.track_and_verify_wals_in_manifest: 0 2021/12/15-16:23:43.954362 7f39f45b2ac0 Options.env: 0x7f39f55571c0 2021/12/15-16:23:43.954364 7f39f45b2ac0 Options.fs: Posix File System 2021/12/15-16:23:43.954366 7f39f45b2ac0 Options.info_log: 0x18aa860 2021/12/15-16:23:43.954368 7f39f45b2ac0 Options.max_file_opening_threads: 16 2021/12/15-16:23:43.954369 7f39f45b2ac0 Options.statistics: (nil) 2021/12/15-16:23:43.954371 7f39f45b2ac0 Options.use_fsync: 0 2021/12/15-16:23:43.954373 7f39f45b2ac0 Options.max_log_file_size: 0 2021/12/15-16:23:43.954374 7f39f45b2ac0 Options.max_manifest_file_size: 1073741824 2021/12/15-16:23:43.954376 7f39f45b2ac0 Options.log_file_time_to_roll: 0 2021/12/15-16:23:43.954378 7f39f45b2ac0 Options.keep_log_file_num: 1000 2021/12/15-16:23:43.954379 7f39f45b2ac0 Options.recycle_log_file_num: 0 2021/12/15-16:23:43.954381 7f39f45b2ac0 Options.allow_fallocate: 1 2021/12/15-16:23:43.954382 7f39f45b2ac0 Options.allow_mmap_reads: 0 2021/12/15-16:23:43.954384 7f39f45b2ac0 Options.allow_mmap_writes: 0 2021/12/15-16:23:43.954385 7f39f45b2ac0 Options.use_direct_reads: 0 2021/12/15-16:23:43.954387 7f39f45b2ac0 Options.use_direct_io_for_flush_and_compaction: 0 2021/12/15-16:23:43.954388 7f39f45b2ac0 Options.create_missing_column_families: 0 2021/12/15-16:23:43.954390 7f39f45b2ac0 Options.db_log_dir: 2021/12/15-16:23:43.954391 7f39f45b2ac0 Options.wal_dir: /tmp/tests/grocksdbtest2 2021/12/15-16:23:43.954393 7f39f45b2ac0 Options.table_cache_numshardbits: 6 2021/12/15-16:23:43.954394 7f39f45b2ac0 Options.WAL_ttl_seconds: 0 2021/12/15-16:23:43.954396 7f39f45b2ac0 Options.WAL_size_limit_MB: 0 2021/12/15-16:23:43.954397 7f39f45b2ac0 Options.max_write_batch_group_size_bytes: 1048576 2021/12/15-16:23:43.954399 7f39f45b2ac0 Options.manifest_preallocation_size: 4194304 2021/12/15-16:23:43.954401 7f39f45b2ac0 Options.is_fd_close_on_exec: 1 2021/12/15-16:23:43.954402 7f39f45b2ac0 Options.advise_random_on_open: 1 2021/12/15-16:23:43.954404 7f39f45b2ac0 Options.db_write_buffer_size: 0 2021/12/15-16:23:43.954406 7f39f45b2ac0 Options.write_buffer_manager: 0x1898340 2021/12/15-16:23:43.954407 7f39f45b2ac0 Options.access_hint_on_compaction_start: 1 2021/12/15-16:23:43.954408 7f39f45b2ac0 Options.new_table_reader_for_compaction_inputs: 0 2021/12/15-16:23:43.954410 7f39f45b2ac0 Options.random_access_max_buffer_size: 1048576 2021/12/15-16:23:43.954411 7f39f45b2ac0 Options.use_adaptive_mutex: 0 2021/12/15-16:23:43.954413 7f39f45b2ac0 Options.rate_limiter: (nil) 2021/12/15-16:23:43.954433 7f39f45b2ac0 Options.sst_file_manager.rate_bytes_per_sec: 0 2021/12/15-16:23:43.954435 7f39f45b2ac0 Options.wal_recovery_mode: 2 2021/12/15-16:23:43.954436 7f39f45b2ac0 Options.enable_thread_tracking: 0 2021/12/15-16:23:43.954438 7f39f45b2ac0 Options.enable_pipelined_write: 0 2021/12/15-16:23:43.954439 7f39f45b2ac0 Options.unordered_write: 0 2021/12/15-16:23:43.954441 7f39f45b2ac0 Options.allow_concurrent_memtable_write: 1 2021/12/15-16:23:43.954442 7f39f45b2ac0 Options.enable_write_thread_adaptive_yield: 1 2021/12/15-16:23:43.954444 7f39f45b2ac0 Options.write_thread_max_yield_usec: 100 2021/12/15-16:23:43.954446 7f39f45b2ac0 Options.write_thread_slow_yield_usec: 3 2021/12/15-16:23:43.954448 7f39f45b2ac0 Options.row_cache: None 2021/12/15-16:23:43.954449 7f39f45b2ac0 Options.wal_filter: None 2021/12/15-16:23:43.954451 7f39f45b2ac0 Options.avoid_flush_during_recovery: 0 2021/12/15-16:23:43.954452 7f39f45b2ac0 Options.allow_ingest_behind: 0 2021/12/15-16:23:43.954454 7f39f45b2ac0 Options.preserve_deletes: 0 2021/12/15-16:23:43.954456 7f39f45b2ac0 Options.two_write_queues: 0 2021/12/15-16:23:43.954457 7f39f45b2ac0 Options.manual_wal_flush: 0 2021/12/15-16:23:43.954459 7f39f45b2ac0 Options.atomic_flush: 0 2021/12/15-16:23:43.954460 7f39f45b2ac0 Options.avoid_unnecessary_blocking_io: 0 2021/12/15-16:23:43.954462 7f39f45b2ac0 Options.persist_stats_to_disk: 0 2021/12/15-16:23:43.954463 7f39f45b2ac0 Options.write_dbid_to_manifest: 0 2021/12/15-16:23:43.954465 7f39f45b2ac0 Options.log_readahead_size: 0 2021/12/15-16:23:43.954466 7f39f45b2ac0 Options.file_checksum_gen_factory: Unknown 2021/12/15-16:23:43.954468 7f39f45b2ac0 Options.best_efforts_recovery: 0 2021/12/15-16:23:43.954469 7f39f45b2ac0 Options.max_bgerror_resume_count: 2147483647 2021/12/15-16:23:43.954471 7f39f45b2ac0 Options.bgerror_resume_retry_interval: 1000000 2021/12/15-16:23:43.954473 7f39f45b2ac0 Options.allow_data_in_errors: 0 2021/12/15-16:23:43.954474 7f39f45b2ac0 Options.db_host_id: hostname 2021/12/15-16:23:43.954476 7f39f45b2ac0 Options.max_background_jobs: 2 2021/12/15-16:23:43.954478 7f39f45b2ac0 Options.max_background_compactions: -1 2021/12/15-16:23:43.954480 7f39f45b2ac0 Options.max_subcompactions: 1 2021/12/15-16:23:43.954481 7f39f45b2ac0 Options.avoid_flush_during_shutdown: 0 2021/12/15-16:23:43.954483 7f39f45b2ac0 Options.writable_file_max_buffer_size: 1048576 2021/12/15-16:23:43.954484 7f39f45b2ac0 Options.delayed_write_rate : 16777216 2021/12/15-16:23:43.954486 7f39f45b2ac0 Options.max_total_wal_size: 0 2021/12/15-16:23:43.954488 7f39f45b2ac0 Options.delete_obsolete_files_period_micros: 21600000000 2021/12/15-16:23:43.954489 7f39f45b2ac0 Options.stats_dump_period_sec: 600 2021/12/15-16:23:43.954491 7f39f45b2ac0 Options.stats_persist_period_sec: 600 2021/12/15-16:23:43.954492 7f39f45b2ac0 Options.stats_history_buffer_size: 1048576 2021/12/15-16:23:43.954493 7f39f45b2ac0 Options.max_open_files: -1 2021/12/15-16:23:43.954495 7f39f45b2ac0 Options.bytes_per_sync: 0 2021/12/15-16:23:43.954496 7f39f45b2ac0 Options.wal_bytes_per_sync: 0 2021/12/15-16:23:43.954498 7f39f45b2ac0 Options.strict_bytes_per_sync: 0 2021/12/15-16:23:43.954499 7f39f45b2ac0 Options.compaction_readahead_size: 0 2021/12/15-16:23:43.954501 7f39f45b2ac0 Options.max_background_flushes: -1 2021/12/15-16:23:43.954502 7f39f45b2ac0 Compression algorithms supported: 2021/12/15-16:23:43.954514 7f39f45b2ac0 kZSTDNotFinalCompression supported: 1 2021/12/15-16:23:43.954518 7f39f45b2ac0 kZSTD supported: 1 2021/12/15-16:23:43.954520 7f39f45b2ac0 kXpressCompression supported: 0 2021/12/15-16:23:43.954521 7f39f45b2ac0 kLZ4HCCompression supported: 1 2021/12/15-16:23:43.954523 7f39f45b2ac0 kLZ4Compression supported: 1 2021/12/15-16:23:43.954525 7f39f45b2ac0 kBZip2Compression supported: 1 2021/12/15-16:23:43.954526 7f39f45b2ac0 kZlibCompression supported: 1 2021/12/15-16:23:43.954528 7f39f45b2ac0 kSnappyCompression supported: 1 2021/12/15-16:23:43.954543 7f39f45b2ac0 Fast CRC32 supported: Supported on x86 2021/12/15-16:23:43.955343 7f39f45b2ac0 [/db_impl/db_impl_open.cc:285] Creating manifest 1 2021/12/15-16:23:43.957575 7f39f45b2ac0 [/version_set.cc:4577] Recovering from manifest file: /tmp/tests/grocksdbtest2/MANIFEST-000001 2021/12/15-16:23:43.957766 7f39f45b2ac0 [/column_family.cc:600] --------------- Options for column family [default]: 2021/12/15-16:23:43.957770 7f39f45b2ac0 Options.comparator: leveldb.BytewiseComparator 2021/12/15-16:23:43.957771 7f39f45b2ac0 Options.merge_operator: None 2021/12/15-16:23:43.957772 7f39f45b2ac0 Options.compaction_filter: None 2021/12/15-16:23:43.957772 7f39f45b2ac0 Options.compaction_filter_factory: None 2021/12/15-16:23:43.957773 7f39f45b2ac0 Options.sst_partitioner_factory: None 2021/12/15-16:23:43.957774 7f39f45b2ac0 Options.memtable_factory: SkipListFactory 2021/12/15-16:23:43.957775 7f39f45b2ac0 Options.table_factory: BlockBasedTable 2021/12/15-16:23:43.957802 7f39f45b2ac0 table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x18a6af0) cache_index_and_filter_blocks: 0 cache_index_and_filter_blocks_with_high_priority: 1 pin_l0_filter_and_index_blocks_in_cache: 0 pin_top_level_index_and_filter: 1 index_type: 0 data_block_index_type: 0 index_shortening: 1 data_block_hash_table_util_ratio: 0.750000 hash_index_allow_collision: 1 checksum: 1 no_block_cache: 0 block_cache: 0x18a6b40 block_cache_name: LRUCache block_cache_options: capacity : 8388608 num_shard_bits : 4 strict_capacity_limit : 0 memory_allocator : None high_pri_pool_ratio: 0.000 block_cache_compressed: (nil) persistent_cache: (nil) block_size: 4096 block_size_deviation: 10 block_restart_interval: 16 index_block_restart_interval: 1 metadata_block_size: 4096 partition_filters: 0 use_delta_encoding: 1 filter_policy: nullptr whole_key_filtering: 1 verify_compression: 0 read_amp_bytes_per_bit: 0 format_version: 5 enable_index_compression: 1 block_align: 0 max_auto_readahead_size: 262144 prepopulate_block_cache: 0 2021/12/15-16:23:43.957804 7f39f45b2ac0 Options.write_buffer_size: 67108864 2021/12/15-16:23:43.957804 7f39f45b2ac0 Options.max_write_buffer_number: 2 2021/12/15-16:23:43.957807 7f39f45b2ac0 Options.compression: Snappy 2021/12/15-16:23:43.957808 7f39f45b2ac0 Options.bottommost_compression: Disabled 2021/12/15-16:23:43.957809 7f39f45b2ac0 Options.prefix_extractor: nullptr 2021/12/15-16:23:43.957809 7f39f45b2ac0 Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021/12/15-16:23:43.957810 7f39f45b2ac0 Options.num_levels: 7 2021/12/15-16:23:43.957811 7f39f45b2ac0 Options.min_write_buffer_number_to_merge: 1 2021/12/15-16:23:43.957812 7f39f45b2ac0 Options.max_write_buffer_number_to_maintain: 0 2021/12/15-16:23:43.957812 7f39f45b2ac0 Options.max_write_buffer_size_to_maintain: 0 2021/12/15-16:23:43.957813 7f39f45b2ac0 Options.bottommost_compression_opts.window_bits: -14 2021/12/15-16:23:43.957814 7f39f45b2ac0 Options.bottommost_compression_opts.level: 32767 2021/12/15-16:23:43.957815 7f39f45b2ac0 Options.bottommost_compression_opts.strategy: 0 2021/12/15-16:23:43.957816 7f39f45b2ac0 Options.bottommost_compression_opts.max_dict_bytes: 0 2021/12/15-16:23:43.957816 7f39f45b2ac0 Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021/12/15-16:23:43.957831 7f39f45b2ac0 Options.bottommost_compression_opts.parallel_threads: 1 2021/12/15-16:23:43.957832 7f39f45b2ac0 Options.bottommost_compression_opts.enabled: false 2021/12/15-16:23:43.957833 7f39f45b2ac0 Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2021/12/15-16:23:43.957834 7f39f45b2ac0 Options.compression_opts.window_bits: -14 2021/12/15-16:23:43.957835 7f39f45b2ac0 Options.compression_opts.level: 32767 2021/12/15-16:23:43.957835 7f39f45b2ac0 Options.compression_opts.strategy: 0 2021/12/15-16:23:43.957836 7f39f45b2ac0 Options.compression_opts.max_dict_bytes: 0 2021/12/15-16:23:43.957837 7f39f45b2ac0 Options.compression_opts.zstd_max_train_bytes: 0 2021/12/15-16:23:43.957838 7f39f45b2ac0 Options.compression_opts.parallel_threads: 1 2021/12/15-16:23:43.957838 7f39f45b2ac0 Options.compression_opts.enabled: false 2021/12/15-16:23:43.957839 7f39f45b2ac0 Options.compression_opts.max_dict_buffer_bytes: 0 2021/12/15-16:23:43.957840 7f39f45b2ac0 Options.level0_file_num_compaction_trigger: 4 2021/12/15-16:23:43.957840 7f39f45b2ac0 Options.level0_slowdown_writes_trigger: 20 2021/12/15-16:23:43.957841 7f39f45b2ac0 Options.level0_stop_writes_trigger: 36 2021/12/15-16:23:43.957842 7f39f45b2ac0 Options.target_file_size_base: 67108864 2021/12/15-16:23:43.957843 7f39f45b2ac0 Options.target_file_size_multiplier: 1 2021/12/15-16:23:43.957843 7f39f45b2ac0 Options.max_bytes_for_level_base: 268435456 2021/12/15-16:23:43.957844 7f39f45b2ac0 Options.level_compaction_dynamic_level_bytes: 0 2021/12/15-16:23:43.957845 7f39f45b2ac0 Options.max_bytes_for_level_multiplier: 10.000000 2021/12/15-16:23:43.957847 7f39f45b2ac0 Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021/12/15-16:23:43.957848 7f39f45b2ac0 Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021/12/15-16:23:43.957848 7f39f45b2ac0 Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021/12/15-16:23:43.957849 7f39f45b2ac0 Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021/12/15-16:23:43.957850 7f39f45b2ac0 Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021/12/15-16:23:43.957850 7f39f45b2ac0 Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021/12/15-16:23:43.957851 7f39f45b2ac0 Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021/12/15-16:23:43.957852 7f39f45b2ac0 Options.max_sequential_skip_in_iterations: 8 2021/12/15-16:23:43.957852 7f39f45b2ac0 Options.max_compaction_bytes: 1677721600 2021/12/15-16:23:43.957853 7f39f45b2ac0 Options.arena_block_size: 1048576 2021/12/15-16:23:43.957854 7f39f45b2ac0 Options.soft_pending_compaction_bytes_limit: 68719476736 2021/12/15-16:23:43.957855 7f39f45b2ac0 Options.hard_pending_compaction_bytes_limit: 274877906944 2021/12/15-16:23:43.957855 7f39f45b2ac0 Options.rate_limit_delay_max_milliseconds: 100 2021/12/15-16:23:43.957856 7f39f45b2ac0 Options.disable_auto_compactions: 0 2021/12/15-16:23:43.957857 7f39f45b2ac0 Options.compaction_style: kCompactionStyleLevel 2021/12/15-16:23:43.957858 7f39f45b2ac0 Options.compaction_pri: kMinOverlappingRatio 2021/12/15-16:23:43.957859 7f39f45b2ac0 Options.compaction_options_universal.size_ratio: 1 2021/12/15-16:23:43.957860 7f39f45b2ac0 Options.compaction_options_universal.min_merge_width: 2 2021/12/15-16:23:43.957861 7f39f45b2ac0 Options.compaction_options_universal.max_merge_width: 4294967295 2021/12/15-16:23:43.957861 7f39f45b2ac0 Options.compaction_options_universal.max_size_amplification_percent: 200 2021/12/15-16:23:43.957862 7f39f45b2ac0 Options.compaction_options_universal.compression_size_percent: -1 2021/12/15-16:23:43.957863 7f39f45b2ac0 Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021/12/15-16:23:43.957863 7f39f45b2ac0 Options.compaction_options_fifo.max_table_files_size: 1073741824 2021/12/15-16:23:43.957872 7f39f45b2ac0 Options.compaction_options_fifo.allow_compaction: 0 2021/12/15-16:23:43.957875 7f39f45b2ac0 Options.table_properties_collectors: 2021/12/15-16:23:43.957876 7f39f45b2ac0 Options.inplace_update_support: 0 2021/12/15-16:23:43.957877 7f39f45b2ac0 Options.inplace_update_num_locks: 10000 2021/12/15-16:23:43.957877 7f39f45b2ac0 Options.memtable_prefix_bloom_size_ratio: 0.000000 2021/12/15-16:23:43.957879 7f39f45b2ac0 Options.memtable_whole_key_filtering: 0 2021/12/15-16:23:43.957879 7f39f45b2ac0 Options.memtable_huge_page_size: 0 2021/12/15-16:23:43.957880 7f39f45b2ac0 Options.bloom_locality: 0 2021/12/15-16:23:43.957881 7f39f45b2ac0 Options.max_successive_merges: 0 2021/12/15-16:23:43.957881 7f39f45b2ac0 Options.optimize_filters_for_hits: 0 2021/12/15-16:23:43.957882 7f39f45b2ac0 Options.paranoid_file_checks: 0 2021/12/15-16:23:43.957883 7f39f45b2ac0 Options.force_consistency_checks: 1 2021/12/15-16:23:43.957883 7f39f45b2ac0 Options.report_bg_io_stats: 0 2021/12/15-16:23:43.957884 7f39f45b2ac0 Options.ttl: 2592000 2021/12/15-16:23:43.957885 7f39f45b2ac0 Options.periodic_compaction_seconds: 0 2021/12/15-16:23:43.957886 7f39f45b2ac0 Options.enable_blob_files: false 2021/12/15-16:23:43.957886 7f39f45b2ac0 Options.min_blob_size: 0 2021/12/15-16:23:43.957887 7f39f45b2ac0 Options.blob_file_size: 268435456 2021/12/15-16:23:43.957888 7f39f45b2ac0 Options.blob_compression_type: NoCompression 2021/12/15-16:23:43.957889 7f39f45b2ac0 Options.enable_blob_garbage_collection: false 2021/12/15-16:23:43.957889 7f39f45b2ac0 Options.blob_garbage_collection_age_cutoff: 0.250000 2021/12/15-16:23:43.960088 7f39f45b2ac0 [/version_set.cc:4617] Recovered from manifest file:/tmp/tests/grocksdbtest2/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021/12/15-16:23:43.960094 7f39f45b2ac0 [/version_set.cc:4632] Column family [default] (ID 0), log number is 0 2021/12/15-16:23:43.960191 7f39f45b2ac0 [/version_set.cc:4123] Creating manifest 4 2021/12/15-16:23:43.962209 7f39f45b2ac0 [DEBUG] [/db_impl/db_impl_files.cc:334] [JOB 1] Delete /tmp/tests/grocksdbtest2/MANIFEST-000001 type=3 #1 -- OK 2021/12/15-16:23:43.964586 7f39f45b2ac0 [/db_impl/db_impl_open.cc:1759] SstFileManager instance 0x18a9140

    • I managed to find out that the problem in the lines with SetIterateUpperBound. When I comment them, Iterator works stable. But I want to use all iterator features with upper bound setting.
    • When I set exactly upper bound equal the last key name, iterator seems also working fine.

    I'm new in DBs, golang, github, sorry if anything wrong.

    opened by fubss 8
  • Custom Comparator in Go

    Custom Comparator in Go

    Comparator interface needs Native method

    Native() *C.rocksdb_comparator_t

    Cgo translates C types into equivalent unexported Go types. Because the translations are unexported, a Go package should not expose C types in its exported API: a C type used in one Go package is different from the same C type used in another.

    Because of this, it is not possible to implement this interface.

    opened by muthukrishnan24 7
  • Support for Eventlistener?

    Support for Eventlistener?

    Hello!

    First I really like to thank you for your awesome work. This is outstanding!

    We will switch our open source project after months of trying badger, boltdb, pebbledb to your lib I guess. For our usecase rocksdb it much faster with less CPU and Memory usage.

    The only thing I can't find is a way to register the EventListener: https://github.com/facebook/rocksdb/blob/29e8f6a698acf0d00d88f21a2692fcdd974a74e5/include/rocksdb/listener.h#L358

    Do I miss something, or is there no way yet?

    Especially the compaction events would be nice for our usecase, to know when compaction is running in the background:

    https://github.com/facebook/rocksdb/blob/29e8f6a698acf0d00d88f21a2692fcdd974a74e5/include/rocksdb/listener.h#L400 https://github.com/facebook/rocksdb/blob/29e8f6a698acf0d00d88f21a2692fcdd974a74e5/include/rocksdb/listener.h#L415

    Can you give me a hint how to implement this if it is missing? Thank you for your help!

    opened by muXxer 6
  • go test failed

    go test failed

    I met this bug when I start the go test to run the inner unit test I got the error told me that I have no linked with Lz4 or other Lib, Why cause this error ?

    opened by g302ge 4
  • how do u do v:=value.Data() , value.Free() without losing

    how do u do v:=value.Data() , value.Free() without losing "v" value?

    how do u do v:=value.Data() value.Free()

    fmt.Printf("v = %v\n",v) // <- from here onwards, "v" is lost. how to prevent v from being lost? this shld be mentioned as part of the caveats of this module coz it took me some time to realise this was the case and it was not obvious

            value, _ := db.Get(ro, k)
            //v := value.Data()
            b := make([]byte,value.Size())
            copy(b,value.Data())
            //defer value.Free()
            //fmt.Printf("gv = %v = %v = %v\n",v,value.Data(),b)
            value.Free()
            //fmt.Printf("gv1 = %v = %v = %v\n",v,value.Data(),b)
            return b
    
    opened by hiqsociety 3
  • Can't call the method

    Can't call the method "CreateColumnFamilyWithTTL", build error _Ctype_int errorr

    in project,call the method CreateColumnFamilyWithTTL, build error _Ctype_int

    image

    image

    i change the package code param type (C.int to int), and in the project call the method by int param, to fixed the error image

    image

    @linxGnu

    opened by binyoyo 2
  • TransactionDB is not safe-concurrent on writes!

    TransactionDB is not safe-concurrent on writes!

    Hi @linxGnu,

    When I write into the same rocksdb file with 2 gorutines, i have this unexpected panic.

    // configure
    func (db *DB) init() error {
    	cache := grocksdb.NewLRUCache(8 * common.MiB)
    
    	blockBaseTableOptions := grocksdb.NewDefaultBlockBasedTableOptions()
    	blockBaseTableOptions.SetBlockCache(cache)
    
    	opts := grocksdb.NewDefaultOptions()
    	opts.SetCreateIfMissing(true)
    	opts.SetBlockBasedTableFactory(blockBaseTableOptions)
    	opts.SetWriteBufferSize(4 * common.MiB)
    
    	txDBOpts := grocksdb.NewDefaultTransactionDBOptions()
    
    	rdb, err := grocksdb.OpenTransactionDb(opts, txDBOpts, db.path)
    	if err != nil {
    		return errors.Wrap(err, "failed to open db")
    	}
    
    	db.db = rdb
    	db.ro = grocksdb.NewDefaultReadOptions()
    	db.wo = grocksdb.NewDefaultWriteOptions()
    	db.txOpts = grocksdb.NewDefaultTransactionOptions()
    
    	db.ro.SetFillCache(true)
    	db.wo.SetSync(db.fsync)
    
    	return nil
    }
    
    // test writes
    func run(database *rocksdb.DB, chRenewTx chan struct{}) {
    	for i := 0; i < expectedTotalMessages; i++ {
    		for j := 0; j < 5; j++ {
    			key := []byte("key-" + strconv.Itoa(i) + "-" + strconv.Itoa(j))
    			value := []byte("value-" + strconv.Itoa(i) + "-" + strconv.Itoa(j))
    
    			if _, err := database.Get(key); err != nil {
    				panic(err)
    			}
    
    			if err := database.Set(key, value); err != nil {
    				panic(err)
    			}
    
    			if i > 0 && (i+j)%messagesToCommit == 0 {
    				chRenewTx <- struct{}{}
    			}
    		}
    	}
    }
    
    func main() {
    	database, err := rocksdbwrapper.New("./.db_data", false)
    	if err != nil {
    		panic(err)
    	}
    
    	if err := database.Begin(); err != nil {
    		panic(err)
    	}
    
    	chRenewTx := make(chan struct{}, 1000)
    
    	go func() {
    		for range chRenewTx {
    			// fmt.Println("Renewing tx...")
    			if err := database.RenewTransaction(); err != nil {
    				panic(err)
    			}
    		}
    	}()
    
    	now := time.Now()
    
    	go run(database, chRenewTx) // If i disable this gorutine, does not panic!
    	run(database, chRenewTx)
    
    	since := time.Since(now)
    
    	fmt.Println("Time:", since)
    	fmt.Println("Rate:", float64(expectedTotalMessages)/since.Seconds())
    }
    
    main(11175,0x700008e78000) malloc: *** error for object 0x5028c00: pointer being freed was not allocated
    main(11175,0x700008e78000) malloc: *** set a breakpoint in malloc_error_break to debug
    SIGABRT: abort
    PC=0x7fff2061092e m=3 sigcode=0
    
    goroutine 0 [idle]:
    runtime: unknown pc 0x7fff2061092e
    stack: frame={sp:0x700008e77ad8, fp:0x0} stack=[0x700008df8350,0x700008e77f50)
    0000700008e779d8:  00007fff2048dba3  0000000000000000 
    0000700008e779e8:  0000700008e77a38  0000700008e77a30 
    0000700008e779f8:  0000000004ff8000  0000700008e78000 
    0000700008e77a08:  00007ffeefbff465  00007fff2048db2e 
    0000700008e77a18:  0000000000000050  0000700008e77b00 
    0000700008e77a28:  00007fff20683627  0000003000000018 
    0000700008e77a38:  0000700008e77b10  0000700008e77a50 
    0000700008e77a48:  00007fff2068376d  0000000000000000 
    0000700008e77a58:  0000000000000000  00007fff2048db2e 
    0000700008e77a68:  0000000004ff804b  0000000000000000 
    0000700008e77a78:  0000000000000000  0000700008e77c68 
    0000700008e77a88:  0000700008e77af8  0000000004ff7028 
    0000700008e77a98:  0000000000000050  00007fff809382a0 
    0000700008e77aa8:  00007fff8096bae8  00007fff2054d896 
    0000700008e77ab8:  0000000000000000  0000700008e77b00 
    0000700008e77ac8:  00007fff205502c6  0000000000000000 
    0000700008e77ad8: <00007fff2063f5bd  0000000000000003 
    0000700008e77ae8:  0000000004ff7028  0000700008e77b20 
    0000700008e77af8:  0000000000000000  0000700008e77b40 
    0000700008e77b08:  00007fff20594411  0000000004ff7028 
    0000700008e77b18:  00007fff2060b829  00007ffffffff9df 
    0000700008e77b28:  ffffffff04ff7000  0000000004ff7000 
    0000700008e77b38:  0000000000000000  0000700008e77ba0 
    0000700008e77b48:  00007fff204741f5  000000002d004b10 
    0000700008e77b58:  0000700008e78000  0000700008e77c60 
    0000700008e77b68:  0000000000000000  00007fff2048b93d 
    0000700008e77b78:  00000000000017fd  00000000000017fd 
    0000700008e77b88:  0000000000003000  000000002c830040 
    0000700008e77b98:  000000002e00c200  0000700008e77c80 
    0000700008e77ba8:  00007fff2047734a  0000000400000000 
    0000700008e77bb8:  0000000004dc7380  0000000005028c00 
    0000700008e77bc8:  0000000000001400  0000000005000000 
    runtime: unknown pc 0x7fff2061092e
    stack: frame={sp:0x700008e77ad8, fp:0x0} stack=[0x700008df8350,0x700008e77f50)
    0000700008e779d8:  00007fff2048dba3  0000000000000000 
    0000700008e779e8:  0000700008e77a38  0000700008e77a30 
    0000700008e779f8:  0000000004ff8000  0000700008e78000 
    0000700008e77a08:  00007ffeefbff465  00007fff2048db2e 
    0000700008e77a18:  0000000000000050  0000700008e77b00 
    0000700008e77a28:  00007fff20683627  0000003000000018 
    0000700008e77a38:  0000700008e77b10  0000700008e77a50 
    0000700008e77a48:  00007fff2068376d  0000000000000000 
    0000700008e77a58:  0000000000000000  00007fff2048db2e 
    0000700008e77a68:  0000000004ff804b  0000000000000000 
    0000700008e77a78:  0000000000000000  0000700008e77c68 
    0000700008e77a88:  0000700008e77af8  0000000004ff7028 
    0000700008e77a98:  0000000000000050  00007fff809382a0 
    0000700008e77aa8:  00007fff8096bae8  00007fff2054d896 
    0000700008e77ab8:  0000000000000000  0000700008e77b00 
    0000700008e77ac8:  00007fff205502c6  0000000000000000 
    0000700008e77ad8: <00007fff2063f5bd  0000000000000003 
    0000700008e77ae8:  0000000004ff7028  0000700008e77b20 
    0000700008e77af8:  0000000000000000  0000700008e77b40 
    0000700008e77b08:  00007fff20594411  0000000004ff7028 
    0000700008e77b18:  00007fff2060b829  00007ffffffff9df 
    0000700008e77b28:  ffffffff04ff7000  0000000004ff7000 
    0000700008e77b38:  0000000000000000  0000700008e77ba0 
    0000700008e77b48:  00007fff204741f5  000000002d004b10 
    0000700008e77b58:  0000700008e78000  0000700008e77c60 
    0000700008e77b68:  0000000000000000  00007fff2048b93d 
    0000700008e77b78:  00000000000017fd  00000000000017fd 
    0000700008e77b88:  0000000000003000  000000002c830040 
    0000700008e77b98:  000000002e00c200  0000700008e77c80 
    0000700008e77ba8:  00007fff2047734a  0000000400000000 
    0000700008e77bb8:  0000000004dc7380  0000000005028c00 
    0000700008e77bc8:  0000000000001400  0000000005000000 
    
    goroutine 7 [syscall]:
    runtime.cgocall(0x4161b70, 0xc00005ed80, 0xc0000109d8)
            /usr/local/go/src/runtime/cgocall.go:154 +0x5b fp=0xc00005ed50 sp=0xc00005ed18 pc=0x400445b
    github.com/linxGnu/grocksdb._Cfunc_rocksdb_transaction_put(0xc50a970, 0xc000173528, 0x8, 0xc000173530, 0xa, 0xc0000109d8)
            _cgo_gotypes.go:8175 +0x45 fp=0xc00005ed80 sp=0xc00005ed50 pc=0x4153625
    github.com/linxGnu/grocksdb.(*Transaction).Put.func2(0xc0001820a0, 0xc000173528, 0xc000173528, 0x8, 0x8, 0xc000173530, 0xc000173530, 0xa, 0x10, 0xc0000109d8)
            /usr/local/godeps/pkg/mod/github.com/linx!gnu/[email protected]/transaction.go:128 +0xcb fp=0xc00005edc8 sp=0xc00005ed80 pc=0x415674b
    github.com/linxGnu/grocksdb.(*Transaction).Put(0xc0001820a0, 0xc000173528, 0x8, 0x8, 0xc000173530, 0xa, 0x10, 0x0, 0x0)
            /usr/local/godeps/pkg/mod/github.com/linx!gnu/[email protected]/transaction.go:128 +0x105 fp=0xc00005ee40 sp=0xc00005edc8 pc=0x4155865
    gitlab.com/etoshi/backend/normalizer-r2d2/pkg/db/providers/rocksdb.(*DB).Set(0xc00002c540, 0xc000173528, 0x8, 0x8, 0xc000173530, 0xa, 0x10, 0x0, 0x0)
            /Users/sergio/Code/backend/normalizer-r2d2/pkg/db/providers/rocksdb/rocksdb.go:160 +0x1dd fp=0xc00005eec8 sp=0xc00005ee40 pc=0x4159d7d
    main.run(0xc00002c540, 0xc00007c240)
            /Users/sergio/Code/backend/normalizer-r2d2/_examples/foo/main.go:28 +0x2d5 fp=0xc00005efd0 sp=0xc00005eec8 pc=0x415a6b5
    runtime.goexit()
            /usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00005efd8 sp=0xc00005efd0 pc=0x406a401
    created by main.main
            /Users/sergio/Code/backend/normalizer-r2d2/_examples/foo/main.go:62 +0x133
    
    goroutine 1 [runnable]:
    github.com/linxGnu/grocksdb._Cfunc_rocksdb_free(0xc510d30)
            _cgo_gotypes.go:2661 +0x3c
    github.com/linxGnu/grocksdb.(*Slice).Free.func1(0xc000600000)
            /usr/local/godeps/pkg/mod/github.com/linx!gnu/[email protected]/slice.go:53 +0x55
    github.com/linxGnu/grocksdb.(*Slice).Free(0xc000600000)
            /usr/local/godeps/pkg/mod/github.com/linx!gnu/[email protected]/slice.go:53 +0x3b
    gitlab.com/etoshi/backend/normalizer-r2d2/pkg/db/providers/rocksdb.(*DB).get(0xc00002c540, 0xc000580000, 0x8, 0x8, 0x4051925, 0xc000580010, 0xc00016fe58, 0xa, 0x10)
            /Users/sergio/Code/backend/normalizer-r2d2/pkg/db/providers/rocksdb/rocksdb.go:205 +0x165
    gitlab.com/etoshi/backend/normalizer-r2d2/pkg/db/providers/rocksdb.(*DB).Get(0xc00002c540, 0xc000580000, 0x8, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0)
            /Users/sergio/Code/backend/normalizer-r2d2/pkg/db/providers/rocksdb/rocksdb.go:214 +0xaf
    main.run(0xc00002c540, 0xc00007c240)
            /Users/sergio/Code/backend/normalizer-r2d2/_examples/foo/main.go:24 +0x269
    main.main()
            /Users/sergio/Code/backend/normalizer-r2d2/_examples/foo/main.go:63 +0x14b
    
    goroutine 6 [chan receive]:
    main.main.func1(0xc00007c240, 0xc00002c540)
            /Users/sergio/Code/backend/normalizer-r2d2/_examples/foo/main.go:52 +0x35
    created by main.main
            /Users/sergio/Code/backend/normalizer-r2d2/_examples/foo/main.go:51 +0xe5
    
    rax    0x0
    rbx    0x700008e78000
    rcx    0x700008e77ad8
    rdx    0x0
    rdi    0x903
    rsi    0x6
    rbp    0x700008e77b00
    rsp    0x700008e77ad8
    r8     0x0
    r9     0x0
    r10    0x700008e78000
    r11    0x246
    r12    0x903
    r13    0x50
    r14    0x6
    r15    0x16
    rip    0x7fff2061092e
    rflags 0x246
    cs     0x7
    fs     0x0
    gs     0x0
    exit status 2
    
    opened by savsgio 2
  • opening more than 11 column families results in error

    opening more than 11 column families results in error

                        cfNames = []string{"default", "m", "v", "c", "e", "x", "k", "l", "s", "h", "z", "bxb"}
                        //cfNames = []string{"default", "m", "v", "c", "e", "x", "k", "l", "s", "h", "z", "b", "p"}
                        dbr[i], cfh, err = gorocksdb.OpenDbColumnFamilies(opts[i], defaultDBDir+"/d/"+ks, cfNames, []*gorocksdb.Options{opts[i], opts[i], opts[i], opts[i], opts[i], opts[i], opts[i], opts[i], opts[i], opts[i], opts[i], opts[i]})
    
    
    fatal error: unexpected signal during runtime execution
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x478 pc=0xc41992]
    
    runtime stack:
    runtime.throw(0x123210e, 0x2a)
            /usr/local/go/src/runtime/panic.go:1117 +0x72
    runtime.sigpanic()
            /usr/local/go/src/runtime/signal_unix.go:718 +0x2e5
    
    goroutine 1 [syscall, locked to thread]:
    runtime.cgocall(0x9062fc, 0xc000207478, 0xc000010588)
            /usr/local/go/src/runtime/cgocall.go:154 +0x5b fp=0xc000207448 sp=0xc000207410 pc=0x79123b
    git.a.com/a/grocksdb._Cfunc_rocksdb_open_column_families(0x7f4bb919fa00, 0x7f4bb903f260, 0xc, 0xc00005a480, 0xc00001e120, 0xc00001e180, 0xc000010588, 0x0)
            _cgo_gotypes.go:2705 +0x49 fp=0xc000207478 sp=0xc000207448 pc=0x874c29
    git.a.com/a/grocksdb.OpenDbColumnFamilies.func1(0xc000059900, 0x7f4bb903f260, 0xc, 0xc0002075d0, 0xc0002075b8, 0xc0002075e8, 0xc000010588, 0x8)
            /root/go/pkg/mod/git.a.com/a/[email protected]/db.go:158 +0x229 fp=0xc0002074e8 sp=0xc000207478 pc=0x878b49
    git.a.com/a/grocksdb.OpenDbColumnFamilies(0xc000059900, 0xc000016c80, 0x8, 0xc000168540, 0xc, 0xc, 0xc000207878, 0xc, 0xc, 0x0, ...)
            /root/go/pkg/mod/git.a.com/a/[email protected]/db.go:158 +0x285 fp=0xc000207610 sp=0xc0002074e8 pc=0x875f85
    
    opened by hiqsociety 2
  • Shared lib work in ubuntu but not in docker based alpine

    Shared lib work in ubuntu but not in docker based alpine

    Same code and dockerfile, github.com/tecbot/gorocksdb works, grocksdb not. Does it support alpine?

    Development environment works:

    Ubuntu 20.04.1 Linux m-pc 5.4.0-58-generic #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
    RocksDB 6.14.6 build from source use PORTABLE=1 make shared_lib
    github.com/linxGnu/grocksdb v1.6.28(with mod)
    

    Dockerfile not:

    FROM registry.cn-shanghai.aliyuncs.com/xm69/alpine-rocksdb-apk AS app
    WORKDIR /home
    COPY code .
    ENV GOPROXY=https://goproxy.cn
    RUN set -eux && \
      #设置源
      echo "http://mirrors.ustc.edu.cn/alpine/edge/main/" > /etc/apk/repositories && \
      echo "http://mirrors.ustc.edu.cn/alpine/edge/community/" >> /etc/apk/repositories && \
      echo "http://mirrors.ustc.edu.cn/alpine/edge/testing/" >> /etc/apk/repositories && \
      apk update && \
      #安装RocksDB
      apk add --allow-untrusted /rocksdb.apk /rocksdb-dev.apk && \
      #打印编译环境
      echo "编译环境:$(uname -a)" && \
      #安装Golang环境
      apk add build-base go && \
      #下载并校验Golang模块
      go mod download && go mod verify && \
      #编译Golang应用
      go build -o /home/main
    

    registry.cn-shanghai.aliyuncs.com/xm69/alpine-rocksdb-apk

    If use Static lib, build failure. Without -tags build ok, but can not run.

    Error pthread set mutex attr: Invalid argument SIGABRT: abort PC=0x7f4cf3f92a71 m=0 sigcode=18446744073709551610

    goroutine 0 [idle]: runtime: unknown pc 0x7f4cf3f92a71 stack: frame={sp:0x7ffe49c331b8, fp:0x0} stack=[0x7ffe49c15480,0x7ffe49c344c0) 00007ffe49c330b8: 000000000000000a 0000000000000000 00007ffe49c330c8: 00007ffe49c33100 0000000000002710 00007ffe49c330d8: 0000000000000000 0000000000000000 00007ffe49c330e8: 00000000009a8000 0000000000000000 00007ffe49c330f8: 0000000000000000 0000000000000000 00007ffe49c33108: 00007f4cf3f70eac 0000000000800000 <crypto/tls.(*Config).Clone+32> 00007ffe49c33118: 0000000000000380 000000000000001b 00007ffe49c33128: 000000000000001b 0000001400000007 00007ffe49c33138: 00000000022f2c90 0000000000000004 00007ffe49c33148: 00007f4cf3f70eac 00000000f3fe0d88 00007ffe49c33158: 0000000000000020 0000000000000000 00007ffe49c33168: 00007f4c00000000 0000000000000026 00007ffe49c33178: 00000000022f46b0 0000000000000001 00007ffe49c33188: 000000000101dbcb 00007ffe49c332b0 00007ffe49c33198: 00000000022f4328 00000000013d59f0 00007ffe49c331a8: 00000000013d7c40 00000000022f4328 00007ffe49c331b8: <00007f4cf3f92bcc 0000000000000000 00007ffe49c331c8: 0000003000000010 00007ffe49c332a0 00007ffe49c331d8: 00007ffe49c331e0 00000000f3fe37c4 00007ffe49c331e8: 000000000000001f 000000000101dbcb 00007ffe49c331f8: 00007f4cf3fb1834 0000000000000016 00007ffe49c33208: 00000000022f4320 0000000000000001 00007ffe49c33218: 0000000000000040 0000000000000001 00007ffe49c33228: 0000000000000001 0000000000000040 00007ffe49c33238: 00000000022f4310 0000000000000008 00007ffe49c33248: 000000000101dbcb 00007ffe49c332b0 00007ffe49c33258: 00007f4cf3f6a713 00007f4cf3fe0a80 00007ffe49c33268: 00007f4cf3f71539 00007ffe49c33370 00007ffe49c33278: 0000000000b11713 00007ffe49c33310 00007ffe49c33288: 0000000000000120 000000000101dbcb 00007ffe49c33298: 0000000000517a38 0000000000000016 00007ffe49c332a8: 00007ffe49c332c0 00007ffe49c332e0 runtime: unknown pc 0x7f4cf3f92a71 stack: frame={sp:0x7ffe49c331b8, fp:0x0} stack=[0x7ffe49c15480,0x7ffe49c344c0) 00007ffe49c330b8: 000000000000000a 0000000000000000 00007ffe49c330c8: 00007ffe49c33100 0000000000002710 00007ffe49c330d8: 0000000000000000 0000000000000000 00007ffe49c330e8: 00000000009a8000 0000000000000000 00007ffe49c330f8: 0000000000000000 0000000000000000 00007ffe49c33108: 00007f4cf3f70eac 0000000000800000 <crypto/tls.(*Config).Clone+32> 00007ffe49c33118: 0000000000000380 000000000000001b 00007ffe49c33128: 000000000000001b 0000001400000007 00007ffe49c33138: 00000000022f2c90 0000000000000004 00007ffe49c33148: 00007f4cf3f70eac 00000000f3fe0d88 00007ffe49c33158: 0000000000000020 0000000000000000 00007ffe49c33168: 00007f4c00000000 0000000000000026 00007ffe49c33178: 00000000022f46b0 0000000000000001 00007ffe49c33188: 000000000101dbcb 00007ffe49c332b0 00007ffe49c33198: 00000000022f4328 00000000013d59f0 00007ffe49c331a8: 00000000013d7c40 00000000022f4328 00007ffe49c331b8: <00007f4cf3f92bcc 0000000000000000 00007ffe49c331c8: 0000003000000010 00007ffe49c332a0 00007ffe49c331d8: 00007ffe49c331e0 00000000f3fe37c4 00007ffe49c331e8: 000000000000001f 000000000101dbcb 00007ffe49c331f8: 00007f4cf3fb1834 0000000000000016 00007ffe49c33208: 00000000022f4320 0000000000000001 00007ffe49c33218: 0000000000000040 0000000000000001 00007ffe49c33228: 0000000000000001 0000000000000040 00007ffe49c33238: 00000000022f4310 0000000000000008 00007ffe49c33248: 000000000101dbcb 00007ffe49c332b0 00007ffe49c33258: 00007f4cf3f6a713 00007f4cf3fe0a80 00007ffe49c33268: 00007f4cf3f71539 00007ffe49c33370 00007ffe49c33278: 0000000000b11713 00007ffe49c33310 00007ffe49c33288: 0000000000000120 000000000101dbcb 00007ffe49c33298: 0000000000517a38 0000000000000016 00007ffe49c332a8: 00007ffe49c332c0 00007ffe49c332e0

    goroutine 1 [syscall]: runtime.cgocall(0x93e930, 0xc0000cfd88, 0xc0000b6580) /usr/lib/go/src/runtime/cgocall.go:133 +0x5b fp=0xc0000cfd58 sp=0xc0000cfd20 pc=0x53d37b github.com/linxGnu/grocksdb._Cfunc_rocksdb_open(0x22d06a0, 0x22d2120, 0xc0000b6580, 0x0) _cgo_gotypes.go:3395 +0x4e fp=0xc0000cfd88 sp=0xc0000cfd58 pc=0x7528ae github.com/linxGnu/grocksdb.OpenDb.func1(0xc0000afa80, 0x22d2120, 0xc0000b6580, 0xc0000ec000) /root/go/pkg/mod/github.com/linx!gnu/[email protected]/db.go:37 +0xa7 fp=0xc0000cfdc0 sp=0xc0000cfd88 pc=0x775347 github.com/linxGnu/grocksdb.OpenDb(0xc0000afa80, 0xc0000b4d80, 0xc, 0xc0000b4d80, 0xc, 0x0) /root/go/pkg/mod/github.com/linx!gnu/[email protected]/db.go:37 +0x7a fp=0xc0000cfe20 sp=0xc0000cfdc0 pc=0x762e3a main.main() /home/main.go:673 +0x870 fp=0xc0000cff88 sp=0xc0000cfe20 pc=0x93bff0 runtime.main() /usr/lib/go/src/runtime/proc.go:204 +0x209 fp=0xc0000cffe0 sp=0xc0000cff88 pc=0x571a69 runtime.goexit() /usr/lib/go/src/runtime/asm_amd64.s:1374 +0x1 fp=0xc0000cffe8 sp=0xc0000cffe0 pc=0x5a3461

    rax 0x0 rbx 0x0 rcx 0x7f4cf3f92a71 rdx 0x0 rdi 0x2 rsi 0x7ffe49c331c0 rbp 0x7ffe49c331c0 rsp 0x7ffe49c331b8 r8 0x7f4cf3fe00c0 r9 0x7f4cf3fdd933 r10 0x8 r11 0x246 r12 0x22f4328 r13 0x13d59f0 r14 0x13d7c40 r15 0x22d26f8 rip 0x7f4cf3f92a71 rflags 0x246 cs 0x33 fs 0x0 gs 0x0

    opened by alx696 2
  • i did the dbr.CompactRange(grocksdb.Range{nil, nil}) to trigger manual compaction but hv error below

    i did the dbr.CompactRange(grocksdb.Range{nil, nil}) to trigger manual compaction but hv error below

    i did the dbr.CompactRange(grocksdb.Range{nil, nil}) to trigger manual compaction but hv error below

    dbr.CompactRange(grocksdb.Range{nil, nil}) (no value) used as value

    i saw the code in _test so just copied and paste but doesnt work.

    how to make it work? thx in advance

    opened by hiqsociety 0
  • Adjust block max size for Flash

    Adjust block max size for Flash

    @unrealhoang @Permagate

    As I research a lot on leveldb/rocksdb sst file, I recommend setting Block Max Size to at least 8MB and at most 32MB for Flash raft store.

    opened by linxGnu 2
Releases(v1.7.3)
Owner
Linh Tran Tuan
Go, Rust Addict
Linh Tran Tuan
A simple wrapper around badgerDB that can be used across multiple projects

mstore Mstore is a simple wrapper around badgerDB for platform applications that require a quick persistent cache close to the consumer. It's intended

null 0 Dec 14, 2021
RocksDB/LevelDB inspired key-value database in Go

Pebble Nightly benchmarks Pebble is a LevelDB/RocksDB inspired key-value store focused on performance and internal usage by CockroachDB. Pebble inheri

CockroachDB 2.9k Jun 30, 2022
Go-video-preview-ffmpeg-wrapper - A simple helper wrapper to generate small webm video previews using ffmpeg, useful for web previews.

Go-video-preview-ffmpeg-wrapper A simple helper wrapper to generate small webm video previews using ffmpeg, useful for web previews. Getting Started u

Robert van Alphen 0 Jan 5, 2022
Via Cep Wrapper is a api wrapper used to find address by zipcode (Brazil only)

Viacep Wrapper Viacep Wrapper is an API wrapper built with Golang used to find address by zipcode (Brazil only). This project was developed for study

PATRICK SEGANTINE 1 Jan 25, 2022
Golang telegram bot API wrapper, session-based router and middleware

go-tgbot Pure Golang telegram bot API wrapper generated from swagger definition, session-based routing and middlewares. Usage benefits No need to lear

Oleg Lebedev 112 Jun 13, 2022
JSON or YAML configuration wrapper with convenient access methods.

Config Package config provides convenient access methods to configuration stored as JSON or YAML. This is a fork of the original version. This version

Oleg Lebedev 250 May 5, 2022
Go wrapper for LZO compression library

This is a cgo wrapper around the LZO real-time compression library. LZO is available at http://www.oberhumer.com/opensource/lzo/ lzo.go is the go pack

Damian Gryski 13 Mar 4, 2022
An easy-to-use XChaCha20-encryption wrapper for io.ReadWriteCloser (even lossy UDP) using ECDH key exchange algorithm, ED25519 signatures and Blake3+Poly1305 checksums/message-authentication for Go (golang). Also a multiplexer.

Quick start Prepare keys (on both sides): [ -f ~/.ssh/id_ed25519 ] && [ -f ~/.ssh/id_ed25519.pub ] || ssh-keygen -t ed25519 scp ~/.ssh/id_ed25519.pub

null 25 May 27, 2022
A light libxml wrapper for Go

Gokogiri LibXML bindings for the Go programming language. By Zhigang Chen and Hampton Catlin This is a major rewrite from v0 in the following places:

Moovweb 590 Jun 9, 2022
goxml - A thin wrapper around libxml2

golibxml golibxml is a simple wrapper for libxml. The goal is to avoid any extra magic so that a more friendly library can be written to sit on top of

Joshua Bussdieker 11 Oct 25, 2020
levigo is a Go wrapper for LevelDB

levigo levigo is a Go wrapper for LevelDB. The API has been godoc'ed and is available on the web. Questions answered at [email protected]

Jeff Hodges 404 May 5, 2022
Database wrapper that manage read write connections

rwdb Database wrapper that manage read write connections Install go get github.com/andizzle/rwdb Create connections package main import "github.com/

Andy Zhang 14 Jan 23, 2022
A simple wrapper around sql.DB to help with structs. Not quite an ORM.

go-modeldb A simple wrapper around sql.DB to help with structs. Not quite an ORM. Philosophy: Don't make an ORM Example: // Setup require "modeldb" db

Jae Kwon 17 Nov 16, 2019
levigo is a Go wrapper for LevelDB

levigo levigo is a Go wrapper for LevelDB. The API has been godoc'ed and is available on the web. Questions answered at [email protected]

Jeff Hodges 404 May 5, 2022
Golang wrapper for Exiftool : extract as much metadata as possible (EXIF, ...) from files (pictures, pdf, office documents, ...)

go-exiftool go-exiftool is a golang library that wraps ExifTool. ExifTool's purpose is to extract as much metadata as possible (EXIF, IPTC, XMP, GPS,

null 114 Jun 22, 2022
Go (golang) wrapper for GDAL, the Geospatial Data Abstraction Library

------------- About ------------- The gdal.go package provides a go wrapper for GDAL, the Geospatial Data Abstraction Library. More information about

null 223 Jul 1, 2022
Go wrapper for taglib

go-taglib Go wrapper for taglib Dependencies You must have the static taglib libraries installed in order to compile go-taglib. OSX: brew install tagl

Trevor Olson 78 Feb 17, 2022
Go wrapper around the Iup GUI toolset

Iup Go Wrapper iup is a Go wrapper around the Iup GUI toolkit. The project was started on April 27, 2011. Fork https://github.com/grd/iup is a fork of

grd 24 Nov 28, 2020
Go Wrapper for the wxWidgets GUI

This is the source code for wxGo a Go wrapper of the wxWidgets library. The actuall wxWidgets source code is not included and will need to be downloa

Jeroen Dirks 55 Jun 9, 2022
A simple wrapper around libpcap for the Go programming language

PCAP This is a simple wrapper around libpcap for Go. Originally written by Andreas Krennmair [email protected] and only minorly touched up by Mark Smith

Andreas Krennmair 444 Jun 21, 2022
A protoc-gen-go wrapper including an RPC stub generator

// Copyright 2013 Google. All rights reserved. // // Use of this source code is governed by a BSD-style // license that can be found in the LICENSE fi

Kyle Lemons 36 Nov 5, 2021
A tiny wrapper around NSQ topic and channel :rocket:

Event Bus NSQ A tiny wrapper around go-nsq topic and channel. Protect nsq calls with gobreaker. Installation go get -u github.com/rafaeljesus/nsq-even

Rafael Jesus 73 May 24, 2022
A tiny wrapper over amqp exchanges and queues 🚌 ✨

Rabbus ?? ✨ A tiny wrapper over amqp exchanges and queues. In memory retries with exponential backoff for sending messages. Protect producer calls wit

Rafael Jesus 93 May 29, 2022
An easy-to-use XChaCha20-encryption wrapper for io.ReadWriteCloser (even lossy UDP) using ECDH key exchange algorithm, ED25519 signatures and Blake3+Poly1305 checksums/message-authentication for Go (golang). Also a multiplexer.

Quick start Prepare keys (on both sides): [ -f ~/.ssh/id_ed25519 ] && [ -f ~/.ssh/id_ed25519.pub ] || ssh-keygen -t ed25519 scp ~/.ssh/id_ed25519.pub

null 25 May 27, 2022
Wrapper package for Go's template/html to allow for easy file-based template inheritance.

Extemplate Extemplate is a small wrapper package around html/template to allow for easy file-based template inheritance. File: templates/parent.tmpl <

Danny van Kooten 49 Jun 14, 2022
Simple golang airtable API wrapper

Golang Airtable API A simple #golang package to access the Airtable API. Table of contents Golang Airtable API Table of contents Installation Basic us

mehanizm 40 Jun 21, 2022
This is a Golang wrapper for working with TMDb API. It aims to support version 3.

This is a Golang wrapper for working with TMDb API. It aims to support version 3. An API Key is required. To register for one, head over to themoviedb

Cyro Dubeux 53 Jun 25, 2022
⚡️ SharePoint authentication, HTTP client & fluent API wrapper for Go (Golang)

Gosip - SharePoint authentication, HTTP client & fluent API wrapper for Go (Golang) Main features Unattended authentication using different strategies

Andrew Koltyakov 79 Jun 18, 2022
Trello API wrapper for Go

Go Trello API A #golang package to access the Trello API. Nearly 100% of the read-only surface area of the API is covered, as is creation and modifica

Aaron Longwell 199 Jun 15, 2022