Golang driver for ClickHouse

Overview

ClickHouse Build Status Go Report Card codecov

Golang SQL database driver for Yandex ClickHouse

Key features

  • Uses native ClickHouse tcp client-server protocol
  • Compatibility with database/sql
  • Round Robin load-balancing
  • Bulk write support : begin->prepare->(in loop exec)->commit
  • LZ4 compression support (default to use pure go lz4, switch to use cgo lz4 by turn clz4 build tags on)
  • External Tables support

DSN

  • username/password - auth credentials
  • database - select the current default database
  • read_timeout/write_timeout - timeout in second
  • no_delay - disable/enable the Nagle Algorithm for tcp socket (default is 'true' - disable)
  • alt_hosts - comma separated list of single address host for load-balancing
  • connection_open_strategy - random/in_order (default random).
    • random - choose random server from set
    • in_order - first live server is choosen in specified order
    • time_random - choose random(based on current time) server from set. This option differs from random in that randomness is based on current time rather than on amount of previous connections.
  • block_size - maximum rows in block (default is 1000000). If the rows are larger then the data will be split into several blocks to send them to the server. If one block was sent to the server, the data will be persisted on the server disk, we can't rollback the transaction. So always keep in mind that the batch size no larger than the block_size if you want atomic batch insert.
  • pool_size - maximum amount of preallocated byte chunks used in queries (default is 100). Decrease this if you experience memory problems at the expense of more GC pressure and vice versa.
  • debug - enable debug output (boolean value)
  • compress - enable lz4 compression (integer value, default is '0')

SSL/TLS parameters:

  • secure - establish secure connection (default is false)
  • skip_verify - skip certificate verification (default is false)
  • tls_config - name of a TLS config with client certificates, registered using clickhouse.RegisterTLSConfig(); implies secure to be true, unless explicitly specified

example:

tcp://host1:9000?username=user&password=qwerty&database=clicks&read_timeout=10&write_timeout=20&alt_hosts=host2:9000,host3:9000

Supported data types

  • UInt8, UInt16, UInt32, UInt64, Int8, Int16, Int32, Int64
  • Float32, Float64
  • String
  • FixedString(N)
  • Date
  • DateTime
  • IPv4
  • IPv6
  • Enum
  • UUID
  • Nullable(T)
  • Array(T) (one-dimensional) godoc

TODO

  • Support other compression methods(zstd ...)

Install

go get -u github.com/ClickHouse/clickhouse-go

Example

package main

import (
	"database/sql"
	"fmt"
	"log"
	"time"

	"github.com/ClickHouse/clickhouse-go"
)

func main() {
	connect, err := sql.Open("clickhouse", "tcp://127.0.0.1:9000?debug=true")
	if err != nil {
		log.Fatal(err)
	}
	if err := connect.Ping(); err != nil {
		if exception, ok := err.(*clickhouse.Exception); ok {
			fmt.Printf("[%d] %s \n%s\n", exception.Code, exception.Message, exception.StackTrace)
		} else {
			fmt.Println(err)
		}
		return
	}

	_, err = connect.Exec(`
		CREATE TABLE IF NOT EXISTS example (
			country_code FixedString(2),
			os_id        UInt8,
			browser_id   UInt8,
			categories   Array(Int16),
			action_day   Date,
			action_time  DateTime
		) engine=Memory
	`)

	if err != nil {
		log.Fatal(err)
	}
	var (
		tx, _   = connect.Begin()
		stmt, _ = tx.Prepare("INSERT INTO example (country_code, os_id, browser_id, categories, action_day, action_time) VALUES (?, ?, ?, ?, ?, ?)")
	)
	defer stmt.Close()

	for i := 0; i < 100; i++ {
		if _, err := stmt.Exec(
			"RU",
			10+i,
			100+i,
			clickhouse.Array([]int16{1, 2, 3}),
			time.Now(),
			time.Now(),
		); err != nil {
			log.Fatal(err)
		}
	}

	if err := tx.Commit(); err != nil {
		log.Fatal(err)
	}

	rows, err := connect.Query("SELECT country_code, os_id, browser_id, categories, action_day, action_time FROM example")
	if err != nil {
		log.Fatal(err)
	}
	defer rows.Close()

	for rows.Next() {
		var (
			country               string
			os, browser           uint8
			categories            []int16
			actionDay, actionTime time.Time
		)
		if err := rows.Scan(&country, &os, &browser, &categories, &actionDay, &actionTime); err != nil {
			log.Fatal(err)
		}
		log.Printf("country: %s, os: %d, browser: %d, categories: %v, action_day: %s, action_time: %s", country, os, browser, categories, actionDay, actionTime)
	}

	if err := rows.Err(); err != nil {
		log.Fatal(err)
	}

	if _, err := connect.Exec("DROP TABLE example"); err != nil {
		log.Fatal(err)
	}
}

Use sqlx

package main

import (
	"log"
	"time"

	"github.com/jmoiron/sqlx"
	_ "github.com/ClickHouse/clickhouse-go"
)

func main() {
	connect, err := sqlx.Open("clickhouse", "tcp://127.0.0.1:9000?debug=true")
	if err != nil {
		log.Fatal(err)
	}
	var items []struct {
		CountryCode string    `db:"country_code"`
		OsID        uint8     `db:"os_id"`
		BrowserID   uint8     `db:"browser_id"`
		Categories  []int16   `db:"categories"`
		ActionTime  time.Time `db:"action_time"`
	}

	if err := connect.Select(&items, "SELECT country_code, os_id, browser_id, categories, action_time FROM example"); err != nil {
		log.Fatal(err)
	}

	for _, item := range items {
		log.Printf("country: %s, os: %d, browser: %d, categories: %v, action_time: %s", item.CountryCode, item.OsID, item.BrowserID, item.Categories, item.ActionTime)
	}
}

External tables support

package main

import (
	"database/sql"
    "database/sql/driver"
	"fmt"
    "github.com/ClickHouse/clickhouse-go/lib/column"
	"log"
	"time"

	"github.com/ClickHouse/clickhouse-go"
)

func main() {
	connect, err := sql.Open("clickhouse", "tcp://127.0.0.1:9000?debug=true")
	if err != nil {
		log.Fatal(err)
	}
	if err := connect.Ping(); err != nil {
		if exception, ok := err.(*clickhouse.Exception); ok {
			fmt.Printf("[%d] %s \n%s\n", exception.Code, exception.Message, exception.StackTrace)
		} else {
			fmt.Println(err)
		}
		return
	}

	_, err = connect.Exec(`
		CREATE TABLE IF NOT EXISTS example (
			country_code FixedString(2),
			os_id        UInt8,
			browser_id   UInt8,
			categories   Array(Int16),
			action_day   Date,
			action_time  DateTime
		) engine=Memory
	`)

	if err != nil {
		log.Fatal(err)
	}
	var (
		tx, _   = connect.Begin()
		stmt, _ = tx.Prepare("INSERT INTO example (country_code, os_id, browser_id, categories, action_day, action_time) VALUES (?, ?, ?, ?, ?, ?)")
	)
	defer stmt.Close()

	for i := 0; i < 100; i++ {
		if _, err := stmt.Exec(
			"RU",
			10+i,
			100+i,
			clickhouse.Array([]int16{1, 2, 3}),
			time.Now(),
			time.Now(),
		); err != nil {
			log.Fatal(err)
		}
	}

	if err := tx.Commit(); err != nil {
		log.Fatal(err)
	}

	col, err := column.Factory("country_code", "String", nil)
	if err != nil {
		log.Fatal(err)
	}
	countriesExternalTable := clickhouse.ExternalTable{
		Name: "countries",
		Values: [][]driver.Value{
			{"RU"},
		},
		Columns: []column.Column{col},
	}
	
    rows, err := connect.Query("SELECT country_code, os_id, browser_id, categories, action_day, action_time "+
            "FROM example WHERE country_code IN ?", countriesExternalTable)
	if err != nil {
		log.Fatal(err)
	}
	defer rows.Close()

	for rows.Next() {
		var (
			country               string
			os, browser           uint8
			categories            []int16
			actionDay, actionTime time.Time
		)
		if err := rows.Scan(&country, &os, &browser, &categories, &actionDay, &actionTime); err != nil {
			log.Fatal(err)
		}
		log.Printf("country: %s, os: %d, browser: %d, categories: %v, action_day: %s, action_time: %s", country, os, browser, categories, actionDay, actionTime)
	}

	if err := rows.Err(); err != nil {
		log.Fatal(err)
	}

	if _, err := connect.Exec("DROP TABLE example"); err != nil {
		log.Fatal(err)
	}
}
Comments
  • Flush batch cause panic

    Flush batch cause panic

    Issue description

    Using Flush() method in clickhouse.Batch causes panic

    Error log

    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x895d0b]
    
    goroutine 12 [running]:
    github.com/ClickHouse/ch-go/proto.(*Buffer).PutUInt8(...)
    	/go/pkg/mod/github.com/!click!house/[email protected]/proto/buffer.go:97
    github.com/ClickHouse/ch-go/proto.(*Buffer).PutByte(...)
    	/go/pkg/mod/github.com/!click!house/[email protected]/proto/buffer.go:82
    github.com/ClickHouse/clickhouse-go/v2.(*connect).sendData(0xc000146270, 0xc00011e1c0?, {0x0, 0x0})
    	/go/pkg/mod/github.com/!click!house/clickhouse-go/[email protected]/conn.go:171 +0xcb
    github.com/ClickHouse/clickhouse-go/v2.(*batch).Flush(0xc00014e280)
    	/go/pkg/mod/github.com/!click!house/clickhouse-go/[email protected]/conn_batch.go:165 +0x79
    

    Configuration

    OS: Alpine Linux 3.16

    Interface: native

    Go version: 1.19

    ClickHouse Server version: 22.9.3.18

    bug investigate P1 
    opened by Qjawko 34
  • Error driver: bad connection

    Error driver: bad connection

    1. I'll be prompted when the table I'm querying doesn't exist: Table xxx doesn't exist.
    2. When the table I'm requesting again exists, I prompt driver: Bad connection problem Use in a go web environment
    investigate 
    opened by y761350477 23
  • Concurrency bug in read for cancel

    Concurrency bug in read for cancel

    From https://github.com/ClickHouse/clickhouse-go/issues/727#issuecomment-1239665912

    We had some interesting scenario as more background for your investigation. This may or may not lead to the problem we had above. Here is the observation for one request (with the v1 driver):

    we set the timeout limit to 90s, and sent the query to clickhouse-go client.
    from CH server query_log, we found the query arrived the server (QueryStart event) about 89s later.
    we assumed that clickhouse-go driver was waiting for an open connection (during relatively busy time, all connections were used).
    the clickhouse-go client timed out after 90s, and went away (the CH server noticed that event).
    the CH server finished the query in 6+s, and tried to send the results back, and found the client left already, resulting in 210 error code.
    

    The reason I'd like to describe this scenario is that there might be a possibility in the v2 driver, the timed-out connection could be reused by a future request, while the connection didn't get "reset" when it timed out by the client earlier. Is it possible? Thanks.

    https://github.com/ClickHouse/clickhouse-go/issues/727#issuecomment-1251261325

    bug P1 stale 
    opened by gingerwizard 20
  • Safety: error out on unsupported server versions

    Safety: error out on unsupported server versions

    Is your feature request related to a problem? Please describe. we use server version 21.1.8, and upgrade to version 22.3 was delayed. In order to take advantage of performance improvement of client driver v2, we went ahead and upgraded from v1.5.4 to v2.2, without knowing it's not supported for the server version. Our testing didn't catch any problems except for the default conn_open_strategy incompatible change we reported earlier. It was almost 3 weeks after deployment that some problems manifested then it took a village for us to hunt down the potential causes along the long chain of services. Now we realized that this server and client version incompatibility could have caused the silent bugs (we had proof what went wrong - contact me if you need to know details). (Sorry that I'm from commercial software background, and it was hard for me to think the server and client software released within about a year from each other won't work together. So we missed that unsupported list we should have paid attention!)

    Describe the solution you'd like the client driver starts with hand-shake with the server, on unsupported server versions, to prevent accidents, it should error out, instead of continuing.

    Describe alternatives you've considered A server and client version compatibility matrix page would help users for safe upgrade path.

    Additional context Appreciate your understanding and support!!

    bug enhancement investigate 
    opened by ns-gzhang 17
  • HTTP Part 2

    HTTP Part 2

    Issue https://github.com/ClickHouse/clickhouse-go/issues/597

    Changes

    • Add HTTP url options
    • Add HTTP auth options
    • Add HTTP async insert

    cc @gingerwizard

    enhancement 
    opened by ortyomka 17
  • numInput returns wrong number if query contains '@'.

    numInput returns wrong number if query contains '@'.

    I run the following statement:

    INSERT INTO `dbr_people` (`id`,`name`,`email`) VALUES (258,'jonathan','[email protected]')
    

    Expected result: it completes successful. Actual Result: it fails with error: sql: expected 1 arguments, got 0

    It occurs because numInput counts @uservoice as variable and returns 1 as number of input parameters. :(

    investigate 
    opened by bgaifullin 17
  • New type Object('JSON') is not supported

    New type Object('JSON') is not supported

    Hi,

    New type Object('JSON') has appeared recently in ClickHouse 22.3.2.1. It would be super cool to add it in clickhouse-go. New feature allows you to significantly speed up a request(search) for json data type.

    https://clickhouse.com/docs/en/whats-new/changelog/#223 Experimental Feature

    New data type Object(<schema_format>), which supports storing of semi-structured data (for now JSON only). Data is written to such types as string. Then all paths are extracted according to format of semi-structured data and written as separate columns in most optimal types, that can store all their values. Those columns can be queried by names that match paths in source data. E.g data.key1.key2 or with cast operator data.key1.key2::Int64.

    Support of dynamic subcolumns (JSON data type) #23932 https://github.com/ClickHouse/ClickHouse/pull/23932

    opened by Astlol 16
  • SIGSEGV on multiple prepared statements

    SIGSEGV on multiple prepared statements

    Hello, I have an application which can make many concurrent INSERT statements. My code is working fine, but when the INSERT statements number increase, touching some hundred of concurrents, I have this panic:

    panic: runtime error: invalid memory address or nil pointer dereference
    panic: runtime error: invalid memory address or nil pointer dereference
    [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4c848e]
    goroutine 340 [running]:
    database/sql.(*Stmt).Close(0x0, 0x0, 0x0)
    /usr/lib/go-1.11/src/database/sql/sql.go:2545 +0x2e
    panic(0x6918c0, 0x8d8730)
    /usr/lib/go-1.11/src/runtime/panic.go:513 +0x1b9
    database/sql.(*Stmt).ExecContext(0x0, 0x72aa20, 0xc0000be010, 0xc00a6a5f20, 0x6, 0x6, 0x0, 0x0, 0x0, 0x0)
    /usr/lib/go-1.11/src/database/sql/sql.go:2301 +0x4a
    database/sql.(*Stmt).Exec(0x0, 0xc00a6a5f20, 0x6, 0x6, 0x10, 0x0, 0x0, 0xc001f44000)
    /usr/lib/go-1.11/src/database/sql/sql.go:2330 +0x65
    main.insertIP(0xc0002b2cc0, 0xc00b614090, 0xb, 0xc0001bd770, 0x34, 0xc0001bd7b4, 0xd, 0xc0001bd7a5, 0x2, 0xc00b6140a0)
    /root/project/MessageHandler.go:158 +0x546
    created by main.handleMessage
    /root/project/MessageHandler.go:378 +0x269 
    

    Each statement is prepared with about 150 values, and the entire cycle (Begin, Prepare, Exec and Commit) is done inside a goroutine. There are multiple goroutines running at the same time, and they share the same connection.

    bug 
    opened by AlessandroSechi 16
  • ClickHouse can produce blocks bigger than 1 mil rows.

    ClickHouse can produce blocks bigger than 1 mil rows.

    ClickHouse can produce blocks bigger than 1 mil rows, in case you have GROUP BY in your query.

    https://github.com/ClickHouse/clickhouse-go/blob/e9d187591f80acb3da4d24adf8ff61c2231d39ee/lib/proto/block.go#L144

    bug 
    opened by UnamedRus 15
  • Decimal: read value not valid

    Decimal: read value not valid

    Hi i am trying to read decimal value from clickhouse but it returns wrong value. clickhouse-server: 18.16.0 Code to reproduce

    package main
    
    import (
    	"database/sql"
    	"fmt"
    	"log"
    
    	"github.com/kshvakov/clickhouse"
    )
    
    func main() {
    	connect, err := sql.Open("clickhouse", "tcp://127.0.0.1:9000?debug=true")
    	checkErr(err)
    	if err := connect.Ping(); err != nil {
    		if exception, ok := err.(*clickhouse.Exception); ok {
    			fmt.Printf("[%d] %s \n%s\n", exception.Code, exception.Message, exception.StackTrace)
    		} else {
    			fmt.Println(err)
    		}
    		return
    	}
    
    	_, err = connect.Exec(`
    		CREATE TABLE IF NOT EXISTS example (
    			v Decimal(18,10)
    		) engine=Memory
    	`)
    
    	checkErr(err)
    	tx, err := connect.Begin()
    	checkErr(err)
    	stmt, err := tx.Prepare("INSERT INTO example (v) VALUES (?)")
    	checkErr(err)
    
    	if _, err := stmt.Exec(
    		0.08,
    	); err != nil {
    		log.Fatal(err)
    	}
    
    	checkErr(tx.Commit())
    	rows, err := connect.Query("SELECT v FROM example")
    	checkErr(err)
    	for rows.Next() {
    		var (
    			v               float64
    		)
    		checkErr(rows.Scan(&v))
    		log.Printf("v: %f", v)
    	}
    
    	if _, err := connect.Exec("DROP TABLE example"); err != nil {
    		log.Fatal(err)
    	}
    }
    
    func checkErr(err error) {
    	if err != nil {
    		log.Fatal(err)
    	}
    }
    

    it would print 800000000.000000

    bug 
    opened by minaevmike 15
  • Queries on new / idle Connections timeout and fail

    Queries on new / idle Connections timeout and fail

    I'm having connection issues on new / idle connections. I open the database and ping it on instance startup to make sure that my instances is healthy. The problem is that the first time I execute QueryContext after starting the instance, the query always fails with a 1 min timeout. Subsequent calls succeed and work perfectly.

    If I leave the connection idle for a while (maybe an hour), the first query I run times out after 1 min, the same as when the instance first loads.

    Any suggestions on how to debug this?

    bug 
    opened by derekperkins 14
  • Support Datadog trace

    Support Datadog trace

    Is your feature request related to a problem? Please describe. For better integration with Datadog, see as doc description: https://pkg.go.dev/gopkg.in/DataDog/dd-trace-go.v1/contrib/database/sql It would be nice to have an exposed Driver{}

    Describe the solution you'd like Expose https://github.com/ClickHouse/clickhouse-go/blob/main/clickhouse_std.go#L138 driver for tracing, like:

    sqltrace.Register("clickhouse", &clickhouse.StdDriver{}, sqltrace.WithServiceName("clickHouse"))
    conn, err := sql.Open("clickhouse", fmt.Sprintf("http://%s:%d", host, port))
    
    opened by Accoral 0
  • Bump github.com/testcontainers/testcontainers-go from 0.14.0 to 0.17.0

    Bump github.com/testcontainers/testcontainers-go from 0.14.0 to 0.17.0

    Bumps github.com/testcontainers/testcontainers-go from 0.14.0 to 0.17.0.

    Release notes

    Sourced from github.com/testcontainers/testcontainers-go's releases.

    v0.17.0

    What's Changed

    ⚠️ Breaking Changes

    Given the amount of issues after #476, causing consumers of this library to update their dependencies with multiple replace directives in their go.mod files, we have moved compose code to a separate module. Therefore the majority of the users of the library will only need to replace Docker dependency with the one used in this library, which is simpler in terms of usage. Please see Install instructions for further information.

    replace (
    	github.com/docker/docker => github.com/docker/docker v20.10.3-0.20221013203545-33ab36d6b304+incompatible // 22.06 branch
    )
    

    On the other hand, users of native Docker Compose code will still need all the replace directives, as described in the Compose docs.

    🚀 Features

    🐛 Bug Fixes

    • fix: avoid panics when checking container state and container.raw is nil (#635) @​mdelapenya

    📖 Documentation

    🧹 Housekeeping

    ... (truncated)

    Commits
    • 10c899c chore: move compose code to a separate module (#650)
    • 18a119b docs: refine onboarding process with quickstart guide (#706)
    • 593da80 chore: move redis-specific tests to the example module (#701)
    • 574e1ae chore: bump transitive dependencies (#527)
    • e9fa657 chore: reduce concurrent builds (#702)
    • bb03057 chore: add mysql example (#700)
    • 2de9fb8 chore(deps): bump google.golang.org/api from 0.104.0 to 0.105.0 (#699)
    • 71461a9 chore(deps): bump google.golang.org/api in /examples/firestore (#683)
    • f6b4131 chore(deps): bump cloud.google.com/go/spanner in /examples/spanner (#688)
    • 099b181 chore(deps): bump google.golang.org/api in /examples/pubsub (#685)
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    dependencies 
    opened by dependabot[bot] 0
  • increased memory usage in native v2 client

    increased memory usage in native v2 client

    Issue description

    Our system of ClickHouse inserters have been using the native v2.2.0 client with success for several weeks. After upgrading to v2.4.3 a few days ago, we noticed that the memory and CPU usage increased markedly. Memory usage was roughly 7-10x more, and CPU usage was 6-9x more on our k8 pods.

    Example code

    Our inserters collect in memory ~250k messages, morph them each into ClickHouse row batches and call client.WriteBatch() after checking to see if a lock table doesn't have any rows. The columns contain array(string), array(uint8) uint8, and string types. And some of the array(string) average about 500 elements.

    Error log

    didn't see any insert error logs

    Configuration

    OS: pod images are built using golang:1.19

    I think this is the OS:

    cat /etc/*-release
    ...
    Debian GNU/Linux 11
    ...
    

    Interface: E.g. native, database/sql native

    Driver version: v.2.4.3

    Go version: run go version in your console 1.19.4

    ClickHouse Server version: 22.8.5.29

    discuss performance 
    opened by mettledrum 2
  • Expose DialStrategy function to user for custom connection routing

    Expose DialStrategy function to user for custom connection routing

    Expose a way to have a customized way how connections created: (the result of dial function)

    opts.DialStrategy = func(ctx context.Context, connID int, opts *clickhouse.Options, dial clickhouse.Dial) (clickhouse.DialResult, error) {
    	return dial(ctx, "server.host:5678", opts)
    }
    

    Default DialStrategy has following implementation clickhouse.DefaultDialStrategy:

    func DefaultDialStrategy(ctx context.Context, connID int, opt *Options, dial Dial) (r DialResult, err error) {
    	for i := range opt.Addr {
    		var num int
    		switch opt.ConnOpenStrategy {
    		case ConnOpenInOrder:
    			num = i
    		case ConnOpenRoundRobin:
    			num = (int(connID) + i) % len(opt.Addr)
    		}
    
    		if r, err = dial(ctx, opt.Addr[num], opt); err == nil {
    			return r, nil
    		}
    	}
    
    	if err == nil {
    		err = ErrAcquireConnNoAddress
    	}
    
    	return r, err
    }
    

    clickhouse.DialStrategy function receives a clickhouse.Dial function that needs to be triggered by the user if a new connection is needed. It returns a dedicated DialResult that wraps over internal and coupled connect struct. It's a hacky solution, as connection and pool management are highly coupled in the library. Until we find a better design in v3, we need to have a trade-off.

    Resolves #772

    opened by jkaflik 0
Releases(v2.4.3)
  • v2.4.3(Nov 29, 2022)

    Bug Fixes

    • Fix in batch concurrency - batch could panic if used in separate go routines.

    The issue was originally detected due to the use of a batch in a go routine and Abort being called after the connection was released on the batch. This would invalidate the connection which had been subsequently reassigned.

    This issue could occur as soon as the conn is released (this can happen in a number of places e.g. after Send or an Append error), and it potentially returns to the pool for use in another go routine. Subsequent releases could then occur e.g., the user calls Abort mainly but also Send would do it. The result is the connection being closed in the release function while another batch or query potentially used it.

    This release includes a guard to prevent release from being called more than once on a batch. It assumes that batches are not thread-safe - they aren't (only connections are).

    Source code(tar.gz)
    Source code(zip)
  • v2.4.2(Nov 24, 2022)

    Bug Fixes

    • Don't panic on Send() on batch after invalid Append - https://github.com/ClickHouse/clickhouse-go/pull/830
    • Fix JSON issue with nil if column order is inconsisent - https://github.com/ClickHouse/clickhouse-go/pull/824
    Source code(tar.gz)
    Source code(zip)
  • v2.4.1(Nov 22, 2022)

    Bug Fixes

    • Patch release to fix "Regression - escape character was not considered when comparing column names" - https://github.com/ClickHouse/clickhouse-go/issues/828
    Source code(tar.gz)
    Source code(zip)
  • v2.4.0(Nov 21, 2022)

    New Features

    1. Support for Nullables in Tuples - https://github.com/ClickHouse/clickhouse-go/pull/821 https://github.com/ClickHouse/clickhouse-go/pull/817
    2. Use headers for auth and not url if SSL https://github.com/ClickHouse/clickhouse-go/pull/811
    3. Support additional headers - https://github.com/ClickHouse/clickhouse-go/pull/811
    4. Support int64 for DateTime https://github.com/ClickHouse/clickhouse-go/pull/807
    5. Support inserting Enums as int8/int16/int https://github.com/ClickHouse/clickhouse-go/pull/802
    6. Print error if unsupported server https://github.com/ClickHouse/clickhouse-go/pull/792
    7. Allow block buffer size to tuned for performance - see BlockBufferSize - https://github.com/ClickHouse/clickhouse-go/pull/776
    8. Support custom datetime in Scan - https://github.com/ClickHouse/clickhouse-go/pull/767
    9. Support insertion of an orderedmap - https://github.com/ClickHouse/clickhouse-go/pull/763

    Bug Fixes

    1. Decompress errors over HTTP - https://github.com/ClickHouse/clickhouse-go/pull/792
    2. Use timezone vs timeZone so we work on older versions - https://github.com/ClickHouse/clickhouse-go/pull/781
    3. Ensure only columns specified in INSERT are required in batch - https://github.com/ClickHouse/clickhouse-go/pull/79
    4. Respect order of columns in insert for batch - https://github.com/ClickHouse/clickhouse-go/pull/790
    5. Handle double pointers for Nullable columns when batch inserting - https://github.com/ClickHouse/clickhouse-go/pull/77
    6. Use nil for LowCardinality(Nullable(X)) - https://github.com/ClickHouse/clickhouse-go/pull/768

    Breaking Changes

    1. Align timezone handling with spec - https://github.com/ClickHouse/clickhouse-go/pull/766, specifically:
      1. if parsing strings for datetime, datetime64 or dates we assume the locale is Local (i.e. the client) if not specified in the string.
      2. The server (or column tz) is used for datetime and datetime64 rendering. For date/date32, these have no tz info in the server. For now, they will be rendered as UTC - consistent with the clickhouse-client
      3. Addresses bind when no location is set
    Source code(tar.gz)
    Source code(zip)
  • v2.3.0(Aug 22, 2022)

    • Support for uuid types via Scanner/Stringer https://github.com/ClickHouse/clickhouse-go/pull/720
    • Fix copylock when use sync.Pool in clickhouse.Pool https://github.com/ClickHouse/clickhouse-go/pull/724
    • Remove 128mb limit on block size for compression https://github.com/ClickHouse/clickhouse-go/issues/721
    • Fix Insertion into quoted table https://github.com/ClickHouse/clickhouse-go/issues/715
    Source code(tar.gz)
    Source code(zip)
  • v2.3.0-beta.2(Aug 1, 2022)

    v2.3.0-beta.2

    Notes

    • The release is beta as we only have support for Go 1.18. Will be promoted to GA on the release of Go 1.19.
    • Utilizes ch-go v0.47.2.

    New Features

    • Support flush (experimental) on batch - allows multiple blocks per batch - https://github.com/ClickHouse/clickhouse-go/pull/704
    • Respect ReadTimeout for native - https://github.com/ClickHouse/clickhouse-go/pull/699
    • Support username/password passed as parameters in DSN - https://github.com/ClickHouse/clickhouse-go/pull/691
    • Support []byte for string - https://github.com/ClickHouse/clickhouse-go/pull/688
    • Support for sql.Null* types on equivalent ClickHouse types - https://github.com/ClickHouse/clickhouse-go/pull/686
    • Support for typed maps and slices for tuples (assuming identical sub-column types) - https://github.com/ClickHouse/clickhouse-go/pull/685
    • Default addresses for native and HTTP - localhost:9000 and localhost:8123, respectively. Handles nil Options. - https://github.com/ClickHouse/clickhouse-go/pull/683
    • Brotli compression support (for http) - https://github.com/ClickHouse/clickhouse-go/pull/671
    • Deflate compression support (for http) - https://github.com/ClickHouse/clickhouse-go/pull/667
    • Gzip compression support (for http) - https://github.com/ClickHouse/clickhouse-go/pull/664
    • Allow compression algorithm to be specified in DSN - https://github.com/ClickHouse/clickhouse-go/pull/668
    • Support for Array Sets in parameter binding - https://github.com/ClickHouse/clickhouse-go/pull/659

    Bug Fixes

    • Fix handling of empty response over HTTP - https://github.com/ClickHouse/clickhouse-go/pull/705
    • Handle empty maps - https://github.com/ClickHouse/clickhouse-go/pull/694
    • Immutable options - https://github.com/ClickHouse/clickhouse-go/pull/675
    • Prevent maps from being used with unnamed tuples - https://github.com/ClickHouse/clickhouse-go/pull/665
    • Handle special chars in column names - https://github.com/ClickHouse/clickhouse-go/pull/660
    Source code(tar.gz)
    Source code(zip)
  • v2.3.0-beta.1(Jul 11, 2022)

    • Adds http support via the std interface https://github.com/ClickHouse/clickhouse-go/pull/622 https://github.com/ClickHouse/clickhouse-go/pull/642
    • Low level encoding/decoding performed by ch-go (1.18 golang support only) https://github.com/ClickHouse/clickhouse-go/pull/646
    • Zstd compression support https://github.com/ClickHouse/clickhouse-go/pull/656
    • Support for HTTP compression https://github.com/ClickHouse/clickhouse-go/pull/650
    • Support duration in int64 https://github.com/ClickHouse/clickhouse-go/pull/635

    This version only supports Golang 1.18+.

    Source code(tar.gz)
    Source code(zip)
  • v2.2.0(Jun 27, 2022)

    • Fix Uint128, UInt256 integer overflow https://github.com/ClickHouse/clickhouse-go/pull/629
    • Support for JSON in std interface
    • Support for Nested flatten_nested=0
    • JSON support v2 - still experimental.
    • Named tuples can not be marshalled into a map
    Source code(tar.gz)
    Source code(zip)
  • v2.1.0(Jun 17, 2022)

    • Experimental JSON Support - https://github.com/ClickHouse/clickhouse-go/pull/590
    • Allow writing Tuple data as a pointer - https://github.com/ClickHouse/clickhouse-go/pull/608
    Source code(tar.gz)
    Source code(zip)
  • v2.0.15(Jun 7, 2022)

    • Error if no address is provided https://github.com/ClickHouse/clickhouse-go/pull/606
    • Fix Decimal on negatives https://github.com/ClickHouse/clickhouse-go/pull/605
    • Allow ipv4 to be passed in ipv6 https://github.com/ClickHouse/clickhouse-go/pull/604
    • Support maps in exec https://github.com/ClickHouse/clickhouse-go/pull/603
    • Explicit support and testing
    • Failure over connections properly https://github.com/ClickHouse/clickhouse-go/pull/591
    • Support string decoding into BinaryUnmarshaler https://github.com/ClickHouse/clickhouse-go/pull/586
    • Support positional arguments https://github.com/ClickHouse/clickhouse-go/pull/577
    • Support uint128 https://github.com/ClickHouse/clickhouse-go/pull/568
    • Support string in datetime64 https://github.com/ClickHouse/clickhouse-go/pull/571
    • Fix race condition on structMap cache https://github.com/ClickHouse/clickhouse-go/pull/567
    Source code(tar.gz)
    Source code(zip)
  • v2.0.14(May 4, 2022)

  • v2.0.13(May 4, 2022)

    IMPORTANT THIS RELEASE IS BROKEN. Please use v2.0.14 which is identical with respect to the following changes.

    Changes included this in release:

    1. Fix map of maps https://github.com/ClickHouse/clickhouse-go/pull/554
    2. Increase row limit to 1b - https://github.com/ClickHouse/clickhouse-go/pull/559
    3. Fix deadlock on Close() https://github.com/ClickHouse/clickhouse-go/pull/548
    4. Rename external to ext https://github.com/ClickHouse/clickhouse-go/pull/530
    Source code(tar.gz)
    Source code(zip)
  • v2.0.12(Feb 21, 2022)

    What's Changed

    • Bump go.opentelemetry.io/otel/trace from 1.4.0 to 1.4.1 by @dependabot in https://github.com/ClickHouse/clickhouse-go/pull/503

    Full Changelog: https://github.com/ClickHouse/clickhouse-go/compare/v2.0.11...v2.0.12

    Source code(tar.gz)
    Source code(zip)
  • v2.0.11(Feb 17, 2022)

    What's Changed

    • Bump go.opentelemetry.io/otel/trace from 1.3.0 to 1.4.0 by @dependabot in https://github.com/ClickHouse/clickhouse-go/pull/492
    • Fix nullable column for batch insertion by @grizio in https://github.com/ClickHouse/clickhouse-go/pull/498
    • v2: Add ability to pass as query arguments any type based on string. by @f0mster in https://github.com/ClickHouse/clickhouse-go/pull/496

    New Contributors

    • @grizio made their first contribution in https://github.com/ClickHouse/clickhouse-go/pull/498

    Full Changelog: https://github.com/ClickHouse/clickhouse-go/compare/v2.0.10...v2.0.11

    Source code(tar.gz)
    Source code(zip)
  • v2.0.10(Feb 12, 2022)

    What's Changed

    • Batch: add abort method https://github.com/ClickHouse/clickhouse-go/pull/469

    Full Changelog: https://github.com/ClickHouse/clickhouse-go/compare/v2.0.9...v2.0.10

    Source code(tar.gz)
    Source code(zip)
  • v2.0.9(Feb 8, 2022)

    What's Changed

    • Nested data types https://github.com/ClickHouse/clickhouse-go/issues/483

    • Bump github.com/pierrec/lz4/v4 from 4.1.12 to 4.1.13 by @dependabot in https://github.com/ClickHouse/clickhouse-go/pull/489

    Full Changelog: https://github.com/ClickHouse/clickhouse-go/compare/v2.0.8...v2.0.9

    Source code(tar.gz)
    Source code(zip)
  • v2.0.8(Feb 5, 2022)

    What's Changed

    • Add DialContext option by @slaskis in https://github.com/ClickHouse/clickhouse-go/pull/487

    New Contributors

    • @slaskis made their first contribution in https://github.com/ClickHouse/clickhouse-go/pull/487

    Full Changelog: https://github.com/ClickHouse/clickhouse-go/compare/v2.0.7...v2.0.8

    Source code(tar.gz)
    Source code(zip)
  • v2.0.0-beta.17(Feb 1, 2022)

  • v2.0.0-beta.16(Feb 1, 2022)

  • v2.0.0-beta.15(Jan 31, 2022)

  • v2.0.0-beta.14(Jan 31, 2022)

  • v2.0.0-beta.13(Jan 30, 2022)

Owner
ClickHouse
ClickHouse
Qmgo - The Go driver for MongoDB. It‘s based on official mongo-go-driver but easier to use like Mgo.

Qmgo English | 简体中文 Qmgo is a Go driver for MongoDB . It is based on MongoDB official driver, but easier to use like mgo (such as the chain call). Qmg

Qiniu Cloud 1k Dec 28, 2022
Go driver for PostgreSQL over SSH. This driver can connect to postgres on a server via SSH using the local ssh-agent, password, or private-key.

pqssh Go driver for PostgreSQL over SSH. This driver can connect to postgres on a server via SSH using the local ssh-agent, password, or private-key.

mattn 52 Nov 6, 2022
Jaeger ClickHouse storage plugin implementation

This is implementation of Jaeger's storage plugin for ClickHouse.

Jaeger - Distributed Tracing Platform 150 Jan 4, 2023
Uptrace - Distributed tracing backend using OpenTelemetry and ClickHouse

Distributed tracing backend using OpenTelemetry and ClickHouse Uptrace is a dist

Rohan 0 Mar 8, 2022
Firebird RDBMS sql driver for Go (golang)

firebirdsql (Go firebird sql driver) Firebird RDBMS http://firebirdsql.org SQL driver for Go Requirements Firebird 2.5 or higher Golang 1.13 or higher

Hajime Nakagami 186 Dec 20, 2022
Lightweight Golang driver for ArangoDB

Arangolite Arangolite is a lightweight ArangoDB driver for Go. It focuses on pure AQL querying. See AranGO for a more ORM-like experience. IMPORTANT:

Fabien Herfray 73 Sep 26, 2022
Golang MySQL driver

Install go get github.com/vczyh/go-mysql-driver Usage import _ "github.com/vczyh

Zhang,Yuheng 0 Jan 27, 2022
Mirror of Apache Calcite - Avatica Go SQL Driver

Apache Avatica/Phoenix SQL Driver Apache Calcite's Avatica Go is a Go database/sql driver for the Avatica server. Avatica is a sub-project of Apache C

The Apache Software Foundation 103 Nov 3, 2022
Microsoft ActiveX Object DataBase driver for go that using exp/sql

go-adodb Microsoft ADODB driver conforming to the built-in database/sql interface Installation This package can be installed with the go get command:

mattn 132 Dec 30, 2022
Microsoft SQL server driver written in go language

A pure Go MSSQL driver for Go's database/sql package Install Requires Go 1.8 or above. Install with go get github.com/denisenkom/go-mssqldb . Connecti

null 1.7k Dec 26, 2022
Oracle driver for Go using database/sql

go-oci8 Description Golang Oracle database driver conforming to the Go database/sql interface Installation Install Oracle full client or Instant Clien

mattn 598 Dec 30, 2022
sqlite3 driver for go using database/sql

go-sqlite3 Latest stable version is v1.14 or later not v2. NOTE: The increase to v2 was an accident. There were no major changes or features. Descript

mattn 6.3k Jan 8, 2023
GO DRiver for ORacle DB

Go DRiver for ORacle godror is a package which is a database/sql/driver.Driver for connecting to Oracle DB, using Anthony Tuininga's excellent OCI wra

null 409 Jan 5, 2023
Go Sql Server database driver.

gofreetds Go FreeTDS wrapper. Native Sql Server database driver. Features: can be used as database/sql driver handles calling stored procedures handle

minus5 108 Dec 16, 2022
PostgreSQL driver and toolkit for Go

pgx - PostgreSQL Driver and Toolkit pgx is a pure Go driver and toolkit for PostgreSQL. pgx aims to be low-level, fast, and performant, while also ena

Jack Christensen 6.5k Jan 4, 2023
Pure Go Postgres driver for database/sql

pq - A pure Go postgres driver for Go's database/sql package Install go get github.com/lib/pq Features SSL Handles bad connections for database/sql S

null 7.8k Jan 2, 2023
Go language driver for RethinkDB

RethinkDB-go - RethinkDB Driver for Go Go driver for RethinkDB Current version: v6.2.1 (RethinkDB v2.4) Please note that this version of the driver on

RethinkDB 1.6k Dec 24, 2022
goriak - Go language driver for Riak KV

goriak Current version: v3.2.1. Riak KV version: 2.0 or higher, the latest version of Riak KV is always recommended. What is goriak? goriak is a wrapp

Gustav Westling 29 Nov 22, 2022
Mongo Go Models (mgm) is a fast and simple MongoDB ODM for Go (based on official Mongo Go Driver)

Mongo Go Models Important Note: We changed package name from github.com/Kamva/mgm/v3(uppercase Kamva) to github.com/kamva/mgm/v3(lowercase kamva) in v

kamva 607 Jan 2, 2023