groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.

Related tags

Caching groupcache
Overview

groupcache

Summary

groupcache is a distributed caching and cache-filling library, intended as a replacement for a pool of memcached nodes in many cases.

For API docs and examples, see http://godoc.org/github.com/golang/groupcache

Comparison to memcached

Like memcached, groupcache:

  • shards by key to select which peer is responsible for that key

Unlike memcached, groupcache:

  • does not require running a separate set of servers, thus massively reducing deployment/configuration pain. groupcache is a client library as well as a server. It connects to its own peers, forming a distributed cache.

  • comes with a cache filling mechanism. Whereas memcached just says "Sorry, cache miss", often resulting in a thundering herd of database (or whatever) loads from an unbounded number of clients (which has resulted in several fun outages), groupcache coordinates cache fills such that only one load in one process of an entire replicated set of processes populates the cache, then multiplexes the loaded value to all callers.

  • does not support versioned values. If key "foo" is value "bar", key "foo" must always be "bar". There are neither cache expiration times, nor explicit cache evictions. Thus there is also no CAS, nor Increment/Decrement. This also means that groupcache....

  • ... supports automatic mirroring of super-hot items to multiple processes. This prevents memcached hot spotting where a machine's CPU and/or NIC are overloaded by very popular keys/values.

  • is currently only available for Go. It's very unlikely that I ([email protected]) will port the code to any other language.

Loading process

In a nutshell, a groupcache lookup of Get("foo") looks like:

(On machine #5 of a set of N machines running the same code)

  1. Is the value of "foo" in local memory because it's super hot? If so, use it.

  2. Is the value of "foo" in local memory because peer #5 (the current peer) is the owner of it? If so, use it.

  3. Amongst all the peers in my set of N, am I the owner of the key "foo"? (e.g. does it consistent hash to 5?) If so, load it. If other callers come in, via the same process or via RPC requests from peers, they block waiting for the load to finish and get the same answer. If not, RPC to the peer that's the owner and get the answer. If the RPC fails, just load it locally (still with local dup suppression).

Users

groupcache is in production use by dl.google.com (its original user), parts of Blogger, parts of Google Code, parts of Google Fiber, parts of Google production monitoring systems, etc.

Presentations

See http://talks.golang.org/2013/oscon-dl.slide

Help

Use the golang-nuts mailing list for any discussion or questions.

Issues
  • defaultReplicas = 3 causes badly distributed hash rings

    defaultReplicas = 3 causes badly distributed hash rings

    Currently the hash ring has 3 replicas per node, that can cause statistical imbalances of key distribution, especially with low number of machines.

    The number of replicas should be configurable. For example, a similar Python hash ring library has 40 replicas per node. From tests I made, this is indeed the vicinity of the replicas number where the distribution stabilizes, regardless of the number of nodes.

    I made a few benchmarks directly on the consistent hash to illustrate this (I can post the code to generate this if you want, it uses random "ip addrsses" and keys). The bars are the number of random keys out of 10k keys that mapped to each random node. all tests on the same number of nodes were made with the same node "ips".

    2 Nodes X  3 Replicas:
        36.75% | ####################################
        63.25% | ###############################################################
    
    2 Nodes X  33 Replicas:
        49.77% | #################################################
        50.23% | ##################################################
    
    3 Nodes X  3 Replicas:
        23.05% | #######################
        33.45% | #################################
        43.50% | ###########################################
    
    3 Nodes X  33 Replicas:
        31.50% | ###############################
        31.14% | ###############################
        37.36% | #####################################
    
    7 Nodes X  3 Replicas:
        30.08% | ##############################
        11.24% | ###########
        6.44% | ######
        7.29% | #######
        27.42% | ###########################
        9.26% | #########
        8.27% | ########
    
    7 Nodes X  43 Replicas:
        13.53% | #############
        16.00% | ################
        16.92% | ################
        10.31% | ##########
        14.78% | ##############
        12.74% | ############
        15.72% | ###############
    
    opened by dvirsky 21
  • A bug in getting from peers

    A bug in getting from peers

    Reproduce steps

    pool := groupcache.NewHTTPPool("http://127.0.0.1:"+os.Getenv("PORT"))
    cache := groupcache.NewGroup("cacher", 64<<20, someGetterFunc)
    pool.Set("http://127.0.0.1:50000", "http://127.0.0.1:50001")
    

    and run it in two port :50000 and :50001 Now, if use /xxx as a key, then the groupcacher will act like there is no any peer. i.e. It will only get data from locally, instead of getting from peers.

    The problem is the slash in the head of the key (i.e. if the key is xxx then everything goes right), and I still can't figure out why.

    opened by lazywei 15
  • Added prefix matching for consistenthash lookups

    Added prefix matching for consistenthash lookups

    Changes consistenthash's get() function from O(log n) to O(1), with a memory overhead of 6*n (where n is the number of virtual nodes).

    On an AWS t3.2xlarge dedicated instance, this reduces the consistenthash benchmark times by 56% (8 nodes) to 72% (512 nodes). Data available at https://docs.google.com/spreadsheets/d/1K_kmk0_Lqk6iaSDUytjkT8RNGPTptBwBEWO8q4uAn3w/edit?usp=sharing

    cla: no 
    opened by bobhansen 14
  • allow all options of HTTPPool to be specified by users

    allow all options of HTTPPool to be specified by users

    The default constructor NewHTTPPool is making too many assumptions about how people would use the library. For example, for security purpose I'd want groupcache to listen on a different port by calling Handle on a different ServeMux other than the http.DefaultServeMux. Others have also requested to be able to specify the basePath perhaps to shorter paths for performance reasons https://github.com/golang/groupcache/issues/22 ?

    opened by fumin 14
  • fix HTTPPool can't find peers bug

    fix HTTPPool can't find peers bug

    Description

    The groupcache node canot download content from peer when peer has the right content When use groupcache.HTTPPool

    Details

    URL like http://10.246.14.51:5100/_groupcache/thumbnail/%2Fapi%2Fapks will return http status(301)

    client has to request to http://10.246.14.51:5100/_groupcache/thumbnail/api/apks again.

    But tr.RoundTrip cannot follow http redirect. So use http.Client will fix this problem.

    opened by codeskyblue 13
  • Best practice for updating a cache entry frequently

    Best practice for updating a cache entry frequently

    My question is a bit similar to issue https://github.com/golang/groupcache/issues/3.

    I have a map that is currently managed in the RAM of the go application on a single instance. I want to share this map between multiple instances for scaling. I am already using consul for discovery of peer instances and I am currently solving this with redis, however I am not happy with the fact that I am not leveraging each machine's RAM (so in that sense I feel that redis is more a DB than a cache). This is one reason why I love groupcache.

    I have a constraint though: my map changes all the time (I'm getting requests to update it via http). So for a key K1 in the map, it is likely that m[K1] will be updated very frequently (possibly every one second or less).

    So my questions are:

    1. Am I choosing the wrong architecture? Should I use something like Redis or memecached instead?
    2. If groupcache is a good solution for my use case, do I have to constantly remove and add (say in an LRU cache) or is there a smarter way?

    Thanks!

    opened by orcaman 9
  • Currently found in the use of HttpPool groupcache, key to

    Currently found in the use of HttpPool groupcache, key to "/" at the beginning of the peer can not exchange data between bug hope to solve the next?

    Currently found in the use of HttpPool groupcache, key to "/" at the beginning of the peer can not exchange data between bug hope to solve the next? thinks

    opened by ghost 7
  • consistenthash: replace linear search with binary search

    consistenthash: replace linear search with binary search

    The binary search quickly out-paces the linear search, even for a small number of shards and replicas.

    benchmark old ns/op new ns/op delta BenchmarkGet8 122 122 +0.00% BenchmarkGet32 471 137 -70.91% BenchmarkGet128 5619 254 -95.48% BenchmarkGet512 90302 406 -99.55%

    opened by dgryski 7
  • question - anyone tried this on app engine ?

    question - anyone tried this on app engine ?

    would be very handy. based on the docs, it looks like it could run in the same application on app engine too.

    Anyway, has anyone gotten this far with this ?

    opened by ghost 6
  • implement peerPicker with grpc

    implement peerPicker with grpc

    this PR implements GRPCPool which follows PeerPicker interface but the implementation is based on google's grpc. there are some issues which haven't been addressed in this PR, we could have them in a separated PR

    ==1. interface: grpc uses protobuf version 3, the generated stub is different with the one generated before. Like v2:

    type GetRequest struct {
            Group            *string `protobuf:"bytes,1,req,name=group" json:"group,omitempty"`
            Key              *string `protobuf:"bytes,2,req,name=key" json:"key,omitempty"`
            XXX_unrecognized []byte  `json:"-"`
    }
    

    v3:

    type GetRequest struct {
           Group string `protobuf:"bytes,1,opt,name=group" json:"group,omitempty"`
           Key   string `protobuf:"bytes,2,opt,name=key" json:"key,omitempty"`
    }
    

    If we switched to protoc 3 directly, we may break something. so we just workaround this. The interface is untouched.

    ==2. Context: right now the context used in groupcache is self-defined Context

    // ProtoGetter is the interface that must be implemented by a peer.
    type ProtoGetter interface {
            Get(context Context, in *pb.GetRequest, out *pb.GetResponse) error
    }
    

    and grpc use "golang.org/x/net/context" as default context. so we can pick one.

    ==3. performance comparison: we will do that after the implementation sounds good.

    cla: yes 
    opened by hsinhoyeh 6
  • Spread peers updates

    Spread peers updates

    Hello! First of all, thanks a lot for this project.

    My issue is: Is it possible to spread updates of the peers list initiated in one node to others automatically? You have this implementation

    func (p *HTTPPool) Set(peers ...string) {
        p.mu.Lock()
        defer p.mu.Unlock()
        p.peers = consistenthash.New(defaultReplicas, nil)
        p.peers.Add(peers...)
        p.httpGetters = make(map[string]*httpGetter, len(peers))
        for _, peer := range peers {
            p.httpGetters[peer] = &httpGetter{transport: p.Transport, baseURL: peer + p.basePath}
        }
    }
    

    I don't see anything to make it happens in code above. Or is it inconsistent with goals of the groupcache project? And if so why? Thanks!

    opened by olegdunkan 6
  • may you add default replicas ?

    may you add default replicas ?

    1.add default replicas groupcache/consistenthash.go: line 34. replicas must has a default value. if replicas <=0, then it will do nothing when call func (m *Map) Add(keys ...string)

    and i suggest the default replicas value is 23

    2.add sync.Mutex. when call Add and Get,we can lock the kes and hashMap

    opened by ZBIGBEAR 0
  • fix function TestConsistency

    fix function TestConsistency

    The Add function does not add the hash value of the string key itself to the list keys, so what is the meaning of 'Direct match' in the sentence "Direct matches should always return the same entry", I think this is a bug in the test code, Does 'Direct match' mean that the queried string directly corresponds to a replicated node? So I made some changes to the code.

    opened by 018429 0
  • Added Support for Linux on Power

    Added Support for Linux on Power

    Hi, I had added ppc64le(Linux on Power) support on travis-ci in the branch and looks like its been successfully added. I believe it is ready for the final review and merge. The travis ci build logs can be verified from the link below. https://travis-ci.com/github/ujjwalsh/groupcache/builds/194925870 Please have a look.

    Regards, ujjwal

    cla: yes 
    opened by ujjwalsh 0
Owner
Go
The Go Programming Language
Go
An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.

go-cache go-cache is an in-memory key:value store/cache similar to memcached that is suitable for applications running on a single machine. Its major

Patrick Mylund Nielsen 6.3k Jun 26, 2022
POC de caching en Go en utilisant go-redis/cache

Test-web POC de caching en Go en utilisant go-redis/cache, cette lib permet d'avoir un cache local et un cache redis (appel cache local puis cache red

HUANG Manutea 0 Nov 19, 2021
API Cache is a simple caching server, using grpc to accept messages.

API Cache is a simple caching server, using grpc to accept messages. It allows to store key-value pairs, where key is string and value is []byte.

null 1 Nov 16, 2021
Go Memcached client library #golang

About This is a memcache client library for the Go programming language (http://golang.org/). Installing Using go get $ go get github.com/bradfitz/gom

Brad Fitzpatrick 1.5k Jun 26, 2022
A memcached proxy that manages data chunking and L1 / L2 caches

Rend: Memcached-Compatible Server and Proxy Rend is a proxy whose primary use case is to sit on the same server as both a memcached process and an SSD

Netflix, Inc. 1.1k Jun 24, 2022
Cache library for golang. It supports expirable Cache, LFU, LRU and ARC.

GCache Cache library for golang. It supports expirable Cache, LFU, LRU and ARC. Features Supports expirable Cache, LFU, LRU and ARC. Goroutine safe. S

Jun Kimura 2k Jun 23, 2022
A memcached binary protocol toolkit for go.

gomemcached This is a memcached binary protocol toolkit in go. It provides client and server functionality as well as a little sample server showing h

Dustin Sallings 318 Jun 14, 2022
memcached operator

memcached-operator Operator SDK 中的 Go 编程语言支持可以利用 Operator SDK 中的 Go 编程语言支持,为 Memcached 构 建基于 Go 的 Operator 示例、分布式键值存储并管理其生命周期。 前置条件 安装 Docker Desktop,

jxlwqq 3 Apr 6, 2022
cyhone 102 Jun 22, 2022
Package cache is a middleware that provides the cache management for Flamego.

cache Package cache is a middleware that provides the cache management for Flamego. Installation The minimum requirement of Go is 1.16. go get github.

Flamego 10 Jan 16, 2022
A mem cache base on other populator cache, add following feacture

memcache a mem cache base on other populator cache, add following feacture add lazy load(using expired data, and load it asynchronous) add singlefligh

zhq 1 Oct 28, 2021
Cache - A simple cache implementation

Cache A simple cache implementation LRU Cache An in memory cache implementation

Stanislav Petrashov 1 Jan 25, 2022
Gin-cache - Gin cache middleware with golang

Gin-cache - Gin cache middleware with golang

Anson 38 Mar 16, 2022
Concurrency-safe Go caching library with expiration capabilities and access counters

cache2go Concurrency-safe golang caching library with expiration capabilities. Installation Make sure you have a working Go environment (Go 1.2 or hig

Christian Muehlhaeuser 1.8k Jun 23, 2022
Design and Implement an in-memory caching library for general use

Cache Implementation in GoLang Problem Statement Design and Implement an in-memory caching library for general use. Must Have Support for multiple Sta

null 1 Dec 28, 2021
MySQL to Redis caching made easy

redisql MySQL to Redis caching made easy

Tarek Ali 60 May 24, 2022
Redis caching layer for Cloudflare KV in Golang

Redis caching layer for Cloudflare KV in Golang

Dinesh Sonachalam 30 May 21, 2022
Multi-level caching service in Go

IgoVIUM Multi-level caching service in Go. Specifically: Distributed in-memory cache (L1) DB-based cache (L2) Long term historization on persistent vo

Andrea 7 Dec 23, 2021
A simple generic in-memory caching layer

sc sc is a simple in-memory caching layer for golang. Usage Wrap your function with sc - it will automatically cache the values for specified amount o

motoki317 7 Jun 27, 2022