Native ZooKeeper client for Go. This project is no longer maintained. Please use https://github.com/go-zookeeper/zk instead.

Related tags

Network go-zookeeper
Overview

Native Go Zookeeper Client Library

GoDoc Build Status Coverage Status

License

3-clause BSD. See LICENSE file.

This Repository is No Longer Maintained

Please use https://github.com/go-zookeeper/zk for an actively maintained fork.

Comments
  • AddingPowerSupport_golang-github-samuel-go-zookeeper

    AddingPowerSupport_golang-github-samuel-go-zookeeper

    Adding power support.

    Adding power support ppc64le This is part of the Ubuntu distribution for ppc64le. This helps us simplify testing later when distributions are re-building and re-releasing. For more info tag @gerrith3.

    opened by santosh653 13
  • Hostprovider

    Hostprovider

    Add variadic options to zk.Connect, and a HostProvider interface based on the Java code. An initial DNSHostprovider is the default, matching Java's implementation.

    opened by zellyn 12
  • Add an option to supply a callback to be run for each Event.

    Add an option to supply a callback to be run for each Event.

    Background:

    The channel approach causes lost events during period of scheduling contention. Of particular harm is the loss of HasSession since it's usually the state that triggers actions.

    The callback approch puts the responsiblity on the caller of the library to decide on the tradeoff between losing events and blocking the main event loop.

    opened by nomis52 10
  • Race condition causing receive on a nil channel for

    Race condition causing receive on a nil channel for "watcher" methods

    I found a race condition which meant that you could call receive on a nil channel and thus cause a thread to wait indefinitely. Proposed fix is to pass an already made channel to addWatcher() rather than return one.

    opened by domwong 10
  • Re-submit authentication data on reconnect

    Re-submit authentication data on reconnect

    In case when reconnection occurs, it is necessary to re-authenticate with ZK because it "forgets" the authentication data. Otherwise, the connection is unusable with messages like below popping up in the logs:

    2016/08/29 12:08:36 [ERR] core: failed to write seal configuration: zk: not authenticated
    2016/08/29 12:08:46 [ERR] core: failed to write seal configuration: zk: not authenticated
    2016/08/29 12:08:56 [ERR] core: failed to write seal configuration: zk: not authenticated
    

    The problem can be reproduced with this simple program:

    package main
    
    import (
        "fmt"
        "time"
    
        "github.com/samuel/go-zookeeper/zk"
    )
    
    func main() {
        c, _, err := zk.Connect([]string{"127.0.0.1"}, time.Second) //*10)
        if err != nil {
            panic(err)
        }
        defer c.Close()
    
        for c.State() != zk.StateHasSession {
            fmt.Println("State: ", c.State())
            time.Sleep(time.Second)
        }
        fmt.Printf("Connected to %s\n", c.Server())
    
        c.AddAuth("digest", []byte("userfoo:passbar"))
    
        acl := zk.DigestACL(zk.PermAll, "userfoo", "passbar")
    
        _, err = c.Create("/dir", nil, 0, acl)
        if err != nil && err != zk.ErrNodeExists {
            panic(err)
        }
    
        for {
            res_path, err := c.Create("/dir/dupa", []byte("maryna"), zk.FlagEphemeral|zk.FlagSequence, acl)
            if err != nil {
                fmt.Printf("Error creating ephemeral node: %s\n", err)
            } else {
                fmt.Printf("Created node %s\n", res_path)
            }
            time.Sleep(time.Second)
        }
    }
    
    

    One only needs to block tcp port 2181 traffic or pause/unpause ZK container/process.

    I have not found official ZK protocol specification but instead based my work on what ZK Python client Kazoo does: https://github.com/python-zk/kazoo/blob/master/kazoo/client.py#L722 The change basically stores all submitted auth credentials in connection structure, and on reconnect the sendLoop is stalled until all authentication data has been resubmitted. The re-submission happens synchronously, without involving sendLoop. After all data has been submitted, send loop starts and submits all queued requests. The recieveLoop is unchanged due to the fact that outstanding responses are purged on reconnect.

    This patch may probably be still improved (not re-submitting the same auth data, de-duplicating auth data for digest schema, etc...) and I am not sure if this is the right approach, but I think this is a step into right direction :)

    I would be grateful for feedback.

    Thanks! pr

    opened by vespian 9
  • validate paths before performing operations

    validate paths before performing operations

    Java and other clients will validate on the client side before sending it on. Although this should be handled server side, there are apparently some implemenations that don't do this server side, so this follows suit with the other client libraries.

    Fixes #63

    opened by nemith 8
  • Multi

    Multi

    I have just started using Multi call and have run into two issues.

    Issues:

    1. When a Multi call is preformed with zookeeper, it seems I do not get a response. -No sure what is causing this I will dig a little deeper and see what comes up.
    2. Because I never get a response, I never return from the call. -Their should be a time out on the receive channel, do you want me to add it?
    opened by openhi 7
  • Adding ZK 3.5 reconfig APIs

    Adding ZK 3.5 reconfig APIs

    Supersedes #205

    • changes travis tests to handle all versions of the ZK binary distribution.
    • Adding new APIs IncrementalReconfig and Reconfig to support 3.5 reconfig options.

    tested on ZK 3.5.4-beta 5 node zk clusters with simple add and remove actions beyond the integration tests included in this PR.

    Fixes #204

    opened by jeffbean 6
  • break up set-watches on reconnect into multiple packets

    break up set-watches on reconnect into multiple packets

    This fixes an issue wherein a client that has a very large number of watches (and/or very long path names that are being watched) could cause a disconnect to be effectively permanent: the client gets stuck in a loop of trying to reconnect but being continually disconnected by the server.

    This happens because the client may try to construct a request that is too large for the server to process. A ZK server, by default, limits incoming requests to 1mb. The hard-coded request buffer size in this client is 1.5mb.

    It also fixes a related possible bug where the connection is successfully re-established, but watches are silently lost (because the set-watches request is so big it results in ErrShortBuffer when trying to encode the set-watches-request packet).

    opened by jhump 6
  • max buffer size

    max buffer size

    Introduces a connection option that allows limiting the size of response packets that are buffered in the client. This matches the implementation of Jute in Java, which is the IDL and serialization format ZK uses.

    opened by jhump 6
  • Fix continuous loop of connect attempts

    Fix continuous loop of connect attempts

    Sometimes zookeeper just drops connection on invalid session data, we prefer to drop session and start from scratch when that event occurs instead of dropping into loop of connect/disconnect attempts.

    This can be emulated by stopping zookeeper, cleaning it's database and starting it again. But we experience same bug in production without cleaning zookeeper, but in case of network partition (not every partition event).

    opened by tailhook 6
  • data race in conn.go

    data race in conn.go

    Reproduce this case when test my app with go race. The log:

    2021/06/21 15:21:58 recv loop terminated: err=failed to read from connection: read tcp 192.168.1.101:56952->192.168.1.101:2181: i/o timeout
    2021/06/21 15:21:58 recv loop terminated: err=failed to read from connection: read tcp 192.168.1.101:56976->192.168.1.101:2181: i/o timeout
    ==================
    WARNING: DATA RACE
    Write at 0x00c0001ee000 by goroutine 77:
      github.com/samuel/go-zookeeper/zk.(*Conn).recvLoop()
          /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:910 +0xc4c
      github.com/samuel/go-zookeeper/zk.(*Conn).loop.func2()
          /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:521 +0x2f1
    
    Previous read at 0x00c0001ee000 by goroutine 8:
      github.com/samuel/go-zookeeper/zk.(*Conn).sendSetWatches()
          /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:637 +0x472
      github.com/samuel/go-zookeeper/zk.(*Conn).loop()
          /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:535 +0x5a4
      github.com/samuel/go-zookeeper/zk.Connect.func1()
          /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:223 +0x3c
    
    Goroutine 77 (running) created at:
      github.com/samuel/go-zookeeper/zk.(*Conn).loop()
          /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:516 +0x576
      github.com/samuel/go-zookeeper/zk.Connect.func1()
          /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:223 +0x3c
    
    Goroutine 8 (running) created at:
      github.com/samuel/go-zookeeper/zk.Connect()
          /gopath/src/github.com/samuel/go-zookeeper/zk/conn.go:222 +0x8c6
          
         // ... more application call stack
         // call like this
         // zkConn, chanConnect, err := zk.Connect([]string{zkAddr}, time.Second*30, zk.WithLogInfo(false))
    ==================
    

    centos 7 go 1.16.2 zookeeper 3.6.2

    Originally posted by @huaz836 in https://github.com/samuel/go-zookeeper/issues/178#issuecomment-864974329

    opened by huaz836 0
  • Why clear timeout when read a response from server?

    Why clear timeout when read a response from server?

    I found this line in conn.go

    https://github.com/samuel/go-zookeeper/blob/master/zk/conn.go#L717 c.conn.SetReadDeadline(time.Time{})

    In my view, if there occurs a problem in the network or server, clear the timeout deadline may cause this goroutine stuck for a long time even forever, then others stuck too because of the channel.

    I wonder why it was written like this, was there some reason I've missed it? If necessary, I'd like to post an mr.

    opened by opriz 1
  • why EventNodeDataChanged  trigger  children watch?

    why EventNodeDataChanged trigger children watch?

    as source code below, if data of path change , it will trigger three event. why not only trigger watchTypeData

    func (c *Conn) recvLoop(conn net.Conn) error { .... case EventNodeDeleted, EventNodeDataChanged: wTypes = append(wTypes, watchTypeExist, watchTypeData, watchTypeChild)

    .... }

    opened by darkterrorooo 0
  • If there is no parent node, the node is created level by level

    If there is no parent node, the node is created level by level

    The current method can only create nodes one level at a time, adding the method of creating nodes step by step. If there is no parent node, create the parent node first

    opened by aidelingyu 2
Owner
Samuel Stauffer
Samuel Stauffer
zk2etcd 是一款同步 zookeeper 数据到 etcd 的工具

zk2etcd zk2etcd 是一款同步 zookeeper 数据到 etcd 的工具 项目背景 在云原生大浪潮下,业务都逐渐上 k8s,许多业务以前使用 zookeeper 作为注册中心,现都逐渐倾向更加贴近云原生的 etcd。 在业务向云原生迁移改造的过程中,可能需要将 zookeeper 中

roc 23 Sep 1, 2022
Service registration and discovery, support etcd, zookeeper, consul, etc.

discox 支持类型 zookeeper etcd consul 示例 zookeeper server package main import ( "fmt" "github.com/goeasya/discox" "os" ) func main() { cfg := discox

goeasya 5 Aug 31, 2022
A fork of the simple WireGuard VPN server GUI community maintained

Subspace - A simple WireGuard VPN server GUI Subspace - A simple WireGuard VPN server GUI Slack Screenshots Features Contributing Setup 1. Get a serve

null 1.7k Dec 25, 2022
Use Consul to do service discovery, use gRPC +kafka to do message produce and consume. Use redis to store result.

目录 gRPC/consul/kafka简介 gRPC+kafka的Demo gRPC+kafka整体示意图 限流器 基于redis计数器生成唯一ID kafka生产消费 kafka生产消费示意图 本文kafka生产消费过程 基于pprof的性能分析Demo 使用pprof统计CPU/HEAP数据的

null 43 Jul 9, 2022
MOSN is a cloud native proxy for edge or service mesh. https://mosn.io

中文 MOSN is a network proxy written in Golang. It can be used as a cloud-native network data plane, providing services with the following proxy functio

MOSN 4k Dec 30, 2022
Openldap (LDAP) binding for Golang (go) ; no more support ; you may have a look at https://github.com/go-ldap/ldap

OpenLDAP this is Openldap binding in GO language. I don't work any more with golang, so, please fork this project. Installation : Installation is easy

Marc Quinton 45 Mar 4, 2021
An Etsy StatsD (https://github.com/etsy/statsd) implementation in Go

STATSD-GO Port of Etsy's statsd, written in Go. This was forked from https://github.com/amir/gographite to provide Ganglia submission support. USAGE U

Jeff Buchbinder 45 Mar 5, 2021
Super fault-tolerant gateway for HTTP clusters, written in Go. White paper for reference - https://github.com/gptankit/serviceq-paper

ServiceQ ServiceQ is a fault-tolerant gateway for HTTP clusters. It employs probabilistic routing to distribute load during partial cluster shutdown (

Ankit Gupta 66 Jul 16, 2022
Golang client for NATS, the cloud native messaging system.

NATS - Go Client A Go client for the NATS messaging system. Installation # Go client go get github.com/nats-io/nats.go/ # Server go get github.com/na

NATS - The Cloud Native Messaging System 4.3k Jan 4, 2023
🤘 The native golang ssh client to execute your commands over ssh connection. 🚀🚀

Golang SSH Client. Fast and easy golang ssh client module. Goph is a lightweight Go SSH client focusing on simplicity! Installation ❘ Features ❘ Usage

Mohamed El Bahja 1.2k Dec 24, 2022
null 43 Dec 13, 2022
gh is GitHub on the command line. It brings pull requests, issues, and other GitHub concepts to the terminal next to where you are already working with git and your code

gh is GitHub on the command line. It brings pull requests, issues, and other GitHub concepts to the terminal next to where you are already working with git and your code

frenchy77 0 Jan 24, 2022
High performance DNS over HTTPS client & server

DNS-over-HTTPS Client and server software to query DNS over HTTPS, using Google DNS-over-HTTPS protocol and IETF DNS-over-HTTPS (RFC 8484). Guides Tut

Star Brilliant 1.6k Jan 7, 2023
Tscert - Minimal package for just the HTTPS cert fetching part of the Tailscale client API

tscert This is a stripped down version of the tailscale.com/client/tailscale Go

Tailscale 15 Nov 27, 2022
The server-pubsub is the main backend of DATAVOC project that manages all the other web-server modules of the same project such as the processor

server-pubsub The server-pubsub is the main backend of DATAVOC project that manages all the other web-server modules of the same project such as the p

null 0 Dec 3, 2021
go-github client factory.

go-github-client go-github client factory.

Ken’ichiro Oyama 2 Nov 25, 2022
Simple chat client that uses github.com/tydar/stomper as a backing server.

STOMP Chat Chat client to operate through a STOMP pub/sub server. Designed to demonstrate my project stomper. Todo to finish: Allow runtime configurat

Tyler Darnell 0 Dec 23, 2021
A golang library about socks5, supports all socks5 commands. That Provides server and client and easy to use. Compatible with socks4 and socks4a.

socks5 This is a Golang implementation of the Socks5 protocol library. To see in this SOCKS Protocol Version 5. This library is also compatible with S

chenhao zhang 40 Nov 22, 2022
Self-hostable , easy-to-use , lightweight and feature-rich torrent client written in Go

Self-hostable , easy-to-use , lightweight and feature-rich torrent client written in Go . It comes with beautiful Web UI and Optional Multi-User Support . Run Locally or Host in Server . Open/Download/Stream Torrents in Browser Right Away!

Var Bhat 1.6k Jan 1, 2023