Pure Go Redis server for Go unittests

Overview

Miniredis

Pure Go Redis test server, used in Go unittests.

Sometimes you want to test code which uses Redis, without making it a full-blown integration test. Miniredis implements (parts of) the Redis server, to be used in unittests. It enables a simple, cheap, in-memory, Redis replacement, with a real TCP interface. Think of it as the Redis version of net/http/httptest.

It saves you from using mock code, and since the redis server lives in the test process you can query for values directly, without going through the server stack.

There are no dependencies on external binaries, so you can easily integrate it in automated build processes.

Be sure to import v2:

import "github.com/alicebob/miniredis/v2"

Commands

Implemented commands:

  • Connection (complete)
    • AUTH -- see RequireAuth()
    • ECHO
    • HELLO -- see RequireUserAuth()
    • PING
    • SELECT
    • SWAPDB
    • QUIT
  • Key
    • DEL
    • EXISTS
    • EXPIRE
    • EXPIREAT
    • KEYS
    • MOVE
    • PERSIST
    • PEXPIRE
    • PEXPIREAT
    • PTTL
    • RENAME
    • RENAMENX
    • RANDOMKEY -- see m.Seed(...)
    • SCAN
    • TOUCH
    • TTL
    • TYPE
    • UNLINK
  • Transactions (complete)
    • DISCARD
    • EXEC
    • MULTI
    • UNWATCH
    • WATCH
  • Server
    • DBSIZE
    • FLUSHALL
    • FLUSHDB
    • TIME -- returns time.Now() or value set by SetTime()
  • String keys (complete)
    • APPEND
    • BITCOUNT
    • BITOP
    • BITPOS
    • DECR
    • DECRBY
    • GET
    • GETBIT
    • GETRANGE
    • GETSET
    • INCR
    • INCRBY
    • INCRBYFLOAT
    • MGET
    • MSET
    • MSETNX
    • PSETEX
    • SET
    • SETBIT
    • SETEX
    • SETNX
    • SETRANGE
    • STRLEN
  • Hash keys (complete)
    • HDEL
    • HEXISTS
    • HGET
    • HGETALL
    • HINCRBY
    • HINCRBYFLOAT
    • HKEYS
    • HLEN
    • HMGET
    • HMSET
    • HSET
    • HSETNX
    • HSTRLEN
    • HVALS
    • HSCAN
  • List keys (complete)
    • BLPOP
    • BRPOP
    • BRPOPLPUSH
    • LINDEX
    • LINSERT
    • LLEN
    • LPOP
    • LPUSH
    • LPUSHX
    • LRANGE
    • LREM
    • LSET
    • LTRIM
    • RPOP
    • RPOPLPUSH
    • RPUSH
    • RPUSHX
  • Pub/Sub (complete)
    • PSUBSCRIBE
    • PUBLISH
    • PUBSUB
    • PUNSUBSCRIBE
    • SUBSCRIBE
    • UNSUBSCRIBE
  • Set keys (complete)
    • SADD
    • SCARD
    • SDIFF
    • SDIFFSTORE
    • SINTER
    • SINTERSTORE
    • SISMEMBER
    • SMEMBERS
    • SMOVE
    • SPOP -- see m.Seed(...)
    • SRANDMEMBER -- see m.Seed(...)
    • SREM
    • SUNION
    • SUNIONSTORE
    • SSCAN
  • Sorted Set keys (complete)
    • ZADD
    • ZCARD
    • ZCOUNT
    • ZINCRBY
    • ZINTERSTORE
    • ZLEXCOUNT
    • ZPOPMIN
    • ZPOPMAX
    • ZRANGE
    • ZRANGEBYLEX
    • ZRANGEBYSCORE
    • ZRANK
    • ZREM
    • ZREMRANGEBYLEX
    • ZREMRANGEBYRANK
    • ZREMRANGEBYSCORE
    • ZREVRANGE
    • ZREVRANGEBYLEX
    • ZREVRANGEBYSCORE
    • ZREVRANK
    • ZSCORE
    • ZUNIONSTORE
    • ZSCAN
  • Stream keys
    • XACK
    • XADD
    • XDEL
    • XGROUP CREATE
    • XINFO STREAM -- partly
    • XLEN
    • XRANGE
    • XREAD -- partly
    • XREADGROUP -- partly
    • XREVRANGE
  • Scripting
    • EVAL
    • EVALSHA
    • SCRIPT LOAD
    • SCRIPT EXISTS
    • SCRIPT FLUSH
  • GEO
    • GEOADD
    • GEODIST
    • GEOHASH
    • GEOPOS
    • GEORADIUS
    • GEORADIUS_RO
    • GEORADIUSBYMEMBER
    • GEORADIUSBYMEMBER_RO
  • Server
    • COMMAND -- partly
  • Cluster
    • CLUSTER SLOTS
    • CLUSTER KEYSLOT
    • CLUSTER NODES

TTLs, key expiration, and time

Since miniredis is intended to be used in unittests TTLs don't decrease automatically. You can use TTL() to get the TTL (as a time.Duration) of a key. It will return 0 when no TTL is set.

m.FastForward(d) can be used to decrement all TTLs. All TTLs which become <= 0 will be removed.

EXPIREAT and PEXPIREAT values will be converted to a duration. For that you can either set m.SetTime(t) to use that time as the base for the (P)EXPIREAT conversion, or don't call SetTime(), in which case time.Now() will be used.

SetTime() also sets the value returned by TIME, which defaults to time.Now(). It is not updated by FastForward, only by SetTime.

Randomness and Seed()

Miniredis will use math/rand's global RNG for randomness unless a seed is provided by calling m.Seed(...). If a seed is provided, then miniredis will use its own RNG based on that seed.

Commands which use randomness are: RANDOMKEY, SPOP, and SRANDMEMBER.

Example

import (
    ...
    "github.com/alicebob/miniredis/v2"
    ...
)

func TestSomething(t *testing.T) {
	s, err := miniredis.Run()
	if err != nil {
		panic(err)
	}
	defer s.Close()

	// Optionally set some keys your code expects:
	s.Set("foo", "bar")
	s.HSet("some", "other", "key")

	// Run your code and see if it behaves.
	// An example using the redigo library from "github.com/gomodule/redigo/redis":
	c, err := redis.Dial("tcp", s.Addr())
	_, err = c.Do("SET", "foo", "bar")

	// Optionally check values in redis...
	if got, err := s.Get("foo"); err != nil || got != "bar" {
		t.Error("'foo' has the wrong value")
	}
	// ... or use a helper for that:
	s.CheckGet(t, "foo", "bar")

	// TTL and expiration:
	s.Set("foo", "bar")
	s.SetTTL("foo", 10*time.Second)
	s.FastForward(11 * time.Second)
	if s.Exists("foo") {
		t.Fatal("'foo' should not have existed anymore")
	}
}

Not supported

Commands which will probably not be implemented:

  • CLUSTER (all)
    • CLUSTER *
    • READONLY
    • READWRITE
  • HyperLogLog (all) -- unless someone needs these
    • PFADD
    • PFCOUNT
    • PFMERGE
  • Key
    • DUMP
    • MIGRATE
    • OBJECT
    • RESTORE
    • WAIT
  • Scripting
    • SCRIPT DEBUG
    • SCRIPT KILL
  • Server
    • BGSAVE
    • BGWRITEAOF
    • CLIENT *
    • CONFIG *
    • DEBUG *
    • INFO
    • LASTSAVE
    • MONITOR
    • ROLE
    • SAVE
    • SHUTDOWN
    • SLAVEOF
    • SLOWLOG
    • SYNC

&c.

Integration tests are run against Redis 6.0.10. The ./integration subdir compares miniredis against a real redis instance.

The Redis 6 RESP3 protocol is supported. If there are problems, please open an issue.

If you want to test Redis Sentinel have a look at minisentinel.

A changelog is kept at CHANGELOG.md.

Build Status Go Reference

Issues
  • Will timeouts be considered to adding to the repo?

    Will timeouts be considered to adding to the repo?

    I found that timeout are not implemented, is there any plan to implement it? And is there any technical challenge in the implementation? Do you accept the contribution from outside? Thank you!

    opened by chowchow316 14
  • v2.5.0 tag is gone

    v2.5.0 tag is gone

    Hi @alicebob -- I have encountered an error in a go library which requires v2.5.0 of this package. I am guessing that the tag was removed recently. Can we verify and/or re-add this tag?

    opened by sureifyshain 12
  • Simulate server unavailable?

    Simulate server unavailable?

    I have a health check that pings redis to make sure it's alive. How do I configure or mock miniredis to pretend that is unavailable/unresponsive? Also specifying a timeout on the request?

    opened by elliotchance 11
  • Support for new cluster and streams commands.

    Support for new cluster and streams commands.

    • COMMAND (partly, only full command list);
    • CLUSTER SLOTS. For go-redis/redis cluster connection;
    • XGROUP CREATE;
    • XINFO STREAM (partly, only stream length);
    • XREADGROUP (partly, BLOCK and NOACK arguments not processed);
    • XACK;
    • XDEL.
    opened by kak-tus 9
  • ZRANGEBYLEX: Fix min resulting in empty set returning full set instead

    ZRANGEBYLEX: Fix min resulting in empty set returning full set instead

    If ZRANGEBYLEN was used with a Min that would result in an empty set, the break condition would never trigger so the full set would get returned instead.

    Fixes #99

    opened by shazow 9
  • Add PFADD, PFCOUNT, PFMERGE commands

    Add PFADD, PFCOUNT, PFMERGE commands

    Hi there! We're using your awesome miniredis library for redis-related unit tests in the company I'm working at. Once we started using HLL-related commands of redis, we also definitely wanted to cover the corresponding logic with unit tests but unfortunately found out that HLL-related commands are not supported in miniredis.

    I'm adding HLL-related commands to miniredis with this PR using one quite popular implementation of HLL in golang https://github.com/axiomhq/hyperloglog but not reimplementing the actual C++ implementation of HLL https://download.redis.io/redis-stable/src/hyperloglog.c. Why I did this choice - in fact the miniredis is mainly used for testing purposes and hence it's not necessary to have the 100% compatibility of all the internal details, it's enough to have the compatibility of the protocol and the functionality. Basically the implementation of HLL used in this PR behaves almost the same as redis' variant of HLL. One potential difference I could think of is that the set cardinality estimation accuracy could slightly differ on big sets (containing over 1M of elements), but anyway I'm not sure that somebody plans to have sets of over 1M cardinality in unit tests. Please let me know what you think about it.

    Also one arguable question - not sure if it's OK for you that I'm adding a new dependency. I could have added the implementation of HLL explicitly as it's been done for geohash library.

    Thanks, please let's discuss all the things. If you have any concern about what might be improved, please let me know.

    opened by ilbaktin 8
  • Allow custom command implementations

    Allow custom command implementations

    My specific scenario was the need to implement INFO and it seemed like this would be relatively easy to do if the underlying Server.Register was accessible.

    In my private fork, I ended up exposing the Server like here. If exposing the full Server object is too much, we could just proxy the Register() method.

    Thoughts?

    opened by vk-outreach 8
  • Can't store values with '\n' character

    Can't store values with '\n' character

    There is an issue where you can't store any values that contain a \n character. This is causing some problems for me as I'm trying to marshal proto into redis (which separates values with a \n char). It works on a live redis server but not in miniredis. Here's a simple test I just wrote up:

    func TestNewLine(t *testing.T) {
    	redisServer := setup()
    	defer teardown(redisServer)
    
    	// Query to check that it exists in DB
    	redisClient := redis.NewClient(&redis.Options{
    		Addr:     redisServer.Addr(),
    		Password: "", // no password set
    		DB:       0,  // use default DB
    	})
    
    	err := redisClient.ZAdd("effects", redis.Z{
    		Score:  float64(0),
    		Member: "value",
    	}).Err()
    	if err != nil {
    		t.Errorf("got unexpected error: %v", err)
    	}
    
    	err = redisClient.ZAdd("effects", redis.Z{
    		Score:  float64(0),
    		Member: "value2\n",
    	}).Err()
    	if err != nil {
    		t.Errorf("got unexpected error: %v", err)
    	}
    
    	keys, _ := redisClient.ZScan("effects", 0, "*", 0).Val()
    	fmt.Println(keys)
    	if len(keys) != 4 {
    		t.Errorf("got %v, want %v", len(keys), 4)
    	}
    }
    

    Output:

    [value 0]
    --- FAIL: TestNewLine (0.00s)
        /Users/zachary/Documents/code/github.com/terrariumai/simulation/pkg/datacom/redis_test.go:776: got 2, want 4
    FAIL
    FAIL	github.com/terrariumai/simulation/pkg/datacom	0.033s
    Error: Tests failed.
    

    Looking at the print out, only the first ZAdd is working. Whats more interesting is that the second doesn't error out. The value is just not getting added.

    opened by diericx 8
  • Add GEOADD and GEORADIUS commands

    Add GEOADD and GEORADIUS commands

    This PR adds basic support for redis Geo commands GEOADD and GEORADIUS. I've also added a very simple test.

    Credits: Thank you @alicebob for your help :-)

    opened by sklinkert 8
  • ERR unknown command 'brpop'

    ERR unknown command 'brpop'

    This issue is rather a question. I see commands like brpop are not implemented. Can we have a discussion about this? Why are such commands not implemented? One of my pet projects makes use of brpop. Now I added miniredis to test my storage implementations. It would be supreme to have support for these commands. Cheers.

    opened by xh3b4sd 8
  • Fix race condition in blocking()

    Fix race condition in blocking()

    Currently there is a race condition in blocking().

    The code is waiting for the m.signal condition to fire in a goroutine, and that routine is supposed to be stopped in case of timeouts or context termination with an extra Broadcast() call.

    When the global server context is already done upon entering that function however, the broadcast will race against the scheduling of the goroutine, and in case it misses its target, the waitgroup operation will never finish and the process deadlocks.

    To fix this, turn the logic on its head and wait for the condition synchronously, and send a signal from a goroutine in case we hit a timeout or the context is terminated.

    This isn't an artificial case. The bug can be reproduced with go-redis/v8 and the following snippet:

    package main
    
    import (
    	"context"
    	"fmt"
    	"sync"
    
    	"github.com/alicebob/miniredis/v2"
    	"github.com/go-redis/redis/v8"
    )
    
    func main() {
    	mini, err := miniredis.Run()
    	if err != nil {
    		panic(err)
    	}
    
    	redis := redis.NewUniversalClient(&redis.UniversalOptions{
    		Addrs: []string{mini.Addr()},
    	})
    
    	ctx, cancel := context.WithCancel(redis.Context())
    
    	redis.LPush(ctx, "x", "a")
    
    	var wg sync.WaitGroup
    	wg.Add(1)
    
    	go func() {
    		for {
    			_, err := redis.BRPop(ctx, 0, "x").Result()
    			if err == nil {
    				wg.Done()
    			}
    		}
    	}()
    
    	fmt.Printf("Waiting ...\n")
    	wg.Wait()
    
    	fmt.Printf("Cancelling ...\n")
    	cancel()
    	redis.Close()
    	mini.Close()
    
    	fmt.Printf("Done.\n")
    }
    
    
    opened by zonque 7
  • Can miniredis be used in production server?

    Can miniredis be used in production server?

    First,thanks for this useful library.

    I will integrate unit test in jenkins.Both in test and production server,I want to run unit test first.If pass,then build the project.Otherwise,build fails. If miniredis can't be used in production server, why?Cost,security,or others? I just run unit test in production server in the process of jenkins build,not use miniredis really in production environment .

    opened by abc123xxxyh 1
  • Lua call redis.call('SET', ...) returns string type in miniredis but table type in real Redis

    Lua call redis.call('SET', ...) returns string type in miniredis but table type in real Redis

    The response from certain Redis calls in my Lua script does not conform to those from real Redis instance. This means I cannot test my code that uses Lua scripts correctly when the same Lua script is used in unit tests (miniredis) and integration tests (real Redis).

    For example, this code behaves differently in my unit tests (using miniredis) and my integration test (using a docker instance of redis:6.2.5-buster):

    # The issue is here where miniredis returns a string but real redis return a table
    local result = redis.call('SET', lockKey, lockId, 'NX')
    
    # Real Redis returns the result as a table. If the outcome is successful, then the table
    # has one entry with key='ok' and value=' OK' (result['ok'] = 'OK'). If the  outcome is 
    # an error, then the table has one entry with key='err' and value='the error msg'.
    # NOTE: From what I have read, I am fairly sure the value for the 'err' is the error message
    # but have not proven that, I have proven the 'ok' part which was enough to identify this
    # as an issue.
    
    # So, this works in real Redis but fails in miniredis because result['ok'] on the 
    # string-based miniredis response evaluates to a nil.
    
    if result['ok] == 'OK' then
       do other stuff
    else
      return result
    end
    
    # For it to work in miniredis, you have to do this, expecting a string instead of a table:
    
    if result == 'OK' then
       do other stuff
    else
      return result
    end
    

    I suppose I could work around this by testing the type so that one common Lua script can be used that works with both miniredis and real Redis but it would be nice if miniredis mirrored the real Redis in this case.

    Nevertheless, miniredis is a great tool and thank you for creating it for the community!

    opened by mpetronic 2
  • Support command XAUTOCLAIM

    Support command XAUTOCLAIM

    We leverage streams and the auto claim functionality, it would be great if miniredis supported this. It would allow us to write unit tests instead of integration tests around this functionality.

    opened by vrecan 2
  • LPOS is not supported command for LISTs

    LPOS is not supported command for LISTs

    LPOS is the command to return the index of matching elements inside a Redis list. It doesn't seem to supported at the moment. https://redis.io/commands/lpos

    Wondering whether there are plans to add it.

    opened by bpatel85 2
Releases(v2.22.0)
Owner
Harmen
Harmen
A tool that integrates SQL, HTTP,interface,Redis mock

Mockit 目标:将mock变得简单,让代码维护变得容易 分支介绍 main 主分支,覆盖了单元测试 light 轻分支,去除了单元测试,简化了依赖项,方便其他团队使用 常见Mock难点 不同中间件,mock库设计模式不一致,学习代价高,差异化明显 mock方案强依赖服务端,无法灵活解耦 单元测试

SHIHUO 14 Apr 19, 2022
A go server which will respond with data to mock a server!

Mocker A go program which will respond with data to mock a server; mainly useful while developing a frontend application, whose backend is not running

BE 0 Jan 29, 2022
Fortio load testing library, command line tool, advanced echo server and web UI in go (golang). Allows to specify a set query-per-second load and record latency histograms and other useful stats.

Fortio Fortio (Φορτίο) started as, and is, Istio's load testing tool and now graduated to be its own project. Fortio is also used by, among others, Me

Fortio (Φορτίο) 2.6k Jun 24, 2022
Quick and Easy server testing/validation

Goss - Quick and Easy server validation Goss in 45 seconds Note: For an even faster way of doing this, see: autoadd Note: For testing docker container

Simon Bäumer 25 Dec 29, 2021
Create your own mock server with just a JSON file!

Gmocker Run a blazing fast mock server in just seconds! ?? All you need is to make a json file that contains path and response mapping. See an example

Ananto 49 Apr 13, 2022
Create your own blazing fast mock server with just a JSON file!

Gmocker Run a blazing fast mock server in just seconds! ?? All you need is to make a json file that contains path and response mapping. See an example

Ananto 49 Apr 13, 2022
Vitaly Berg 7 Aug 10, 2021
A simple and expressive HTTP server mocking library for end-to-end tests in Go.

mockhttp A simple and expressive HTTP server mocking library for end-to-end tests in Go. Installation go get -d github.com/americanas-go/mockhttp Exa

Americanas Go 6 Dec 19, 2021
Http test server written in Golang.

Dummy Server Http test server written in Golang. Can get any request method and log headers, body, path and remote address. Useful if you need to know

Yury Krylov 0 Oct 31, 2021
Just Dance Unlimited mock-up server written on Golang and uses a popular Gin framework for Go.

BDCS Just Dance Unlimited mock-up server written on Golang and uses a popular Gin framework for Go. Features Security Authorization works using UbiSer

Mikhail 0 Nov 10, 2021
mock server to aid testing the jaguar-java client API

stripe-mock stripe-mock is a mock HTTP server that responds like the real Stripe API. It can be used instead of Stripe's test mode to make test suites

knr.data 1 Dec 24, 2021
A simple mock server configurable via JSON, built using GoLang.

GoMock Server A simple mock server configurable via JSON, built using GoLang. How To A file name endpoint.json must be placed in the context root, wit

Miguel Alexandre 0 Jan 9, 2022
Golang Server Helpers

go-server-helpers Package Changes NOTE: This package is a work in progress and subject to change. Before v1.0.0 will use minor version tags to mark if

Rienzi Gokea 1 Mar 8, 2022
Client tool for testing HTTP server timeouts

HTTP timeout test client While testing Go HTTP server timeouts I wrote this little tool to help me test. It allows for slowing down header write and b

Adam Pritchard 7 May 11, 2022
Mockserver - Super slim & blazing fast mock server to replace the Java/NPM counterpart mockserver

Gmocker Run a blazing fast mock server in just seconds! ?? All you need is to ma

null 0 Jan 30, 2022
Grpcmock - Mock grpc server with golang

grpcmock Mock gRPC server. Inspired by Prism. Add example responses to your prot

Eric Butler 3 May 8, 2022
Redis-shake is a tool for synchronizing data between two redis databases. Redis-shake是一个用于在两个redis之间同步数据的工具,满足用户非常灵活的同步、迁移需求。

RedisShake is mainly used to synchronize data from one redis to another. Thanks to the Douyu's WSD team for the support. 中文文档 English tutorial 中文使用文档

Alibaba 2.4k Jun 24, 2022
:tophat: Small self-contained pure-Go web server with Lua, Markdown, HTTP/2, QUIC, Redis and PostgreSQL support

Web server with built-in support for QUIC, HTTP/2, Lua, Markdown, Pongo2, HyperApp, Amber, Sass(SCSS), GCSS, JSX, BoltDB (built-in, stores the databas

Alexander F. Rødseth 2k Jun 25, 2022
GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right away.

GoBigdis GoBigdis is a persistent database that implements the Redis server protocol. Any Redis client can interface with it and start to use it right

Riccardo 5 Apr 27, 2022
Golang-redis-webserver - Web server using redis

Web Server using Redis Api REST Database SQLITE3 Cache Redis # Creating record s

Paulo Lopes Estevão 1 Jun 19, 2022
Golang client for redislabs' ReJSON module with support for multilple redis clients (redigo, go-redis)

Go-ReJSON - a golang client for ReJSON (a JSON data type for Redis) Go-ReJSON is a Go client for ReJSON Redis Module. ReJSON is a Redis module that im

Nitish Malhotra 268 Jun 5, 2022
Redis client Mock Provide mock test for redis query

Redis client Mock Provide mock test for redis query, Compatible with github.com/go-redis/redis/v8 Install Confirm that you are using redis.Client the

null 120 Jun 22, 2022
LFU Redis implements LFU Cache algorithm using Redis as data storage

LFU Redis cache library for Golang LFU Redis implements LFU Cache algorithm using Redis as data storage LFU Redis Package gives you control over Cache

Mohamed Shapan 6 Apr 23, 2021
Redis inventory is a tool to analyse Redis memory usage by key patterns and displaying it hierarchically

Redis inventory is a tool to analyse Redis memory usage by key patterns and displaying it hierarchically. The name is inspired by "Disk Inventory X" tool doing similar analysis for disk usage.

Aleks Obukhov 176 May 30, 2022
The source code for workshop Scalable architecture using Redis as backend database using Golang + Redis

The source code for workshop Scalable architecture using Redis as backend database using Golang + Redis

3DS INTERACTIVE 5 Dec 7, 2021
redis-util business-friendly encapsulation of redis operations, such as the common cache set get operation

redis-util 方便业务使用的redis操作封装,比如常见的缓存set get操作, 一行代码搞定,不像开源库需要写好多行 使用方法

connor 1 Oct 22, 2021
skiptable is a jump table that mimics redis' zset using go, and implements most of the features of redis zset

skiptable is a jump table that mimics redis' zset using go, and implements most of the features of redis zset

Qx 1 Oct 25, 2021
Toy-redis-mock - Experimentation with parsing the redis wire protocol from scratch

Overview Simple app for practicing implementing server-side Redis wire protocol

Jeannette Pepin 0 Jan 9, 2022
Bxd redis benchmark - Redis benchmark tool for golang

使用 redis benchmark 工具, 测试 10 20 50 100 200 1k 5k 字节 value 大小,redis get set 性能。 r

bingxindan 0 Jan 22, 2022