Fluent SQL generation for golang

Overview

Stability: Maintenance

Squirrel is "complete".

Bug fixes will still be merged (slowly). Bug reports are welcome, but I will not necessarily respond to them. If another fork (or substantially similar project) actively improves on what Squirrel does, let me know and I may link to it here.

Squirrel - fluent SQL generator for Go

import "github.com/Masterminds/squirrel"

GoDoc Build Status

Squirrel is not an ORM. For an application of Squirrel, check out structable, a table-struct mapper

Squirrel helps you build SQL queries from composable parts:

import sq "github.com/Masterminds/squirrel"

users := sq.Select("*").From("users").Join("emails USING (email_id)")

active := users.Where(sq.Eq{"deleted_at": nil})

sql, args, err := active.ToSql()

sql == "SELECT * FROM users JOIN emails USING (email_id) WHERE deleted_at IS NULL"
sql, args, err := sq.
    Insert("users").Columns("name", "age").
    Values("moe", 13).Values("larry", sq.Expr("? + 5", 12)).
    ToSql()

sql == "INSERT INTO users (name,age) VALUES (?,?),(?,? + 5)"

Squirrel can also execute queries directly:

stooges := users.Where(sq.Eq{"username": []string{"moe", "larry", "curly", "shemp"}})
three_stooges := stooges.Limit(3)
rows, err := three_stooges.RunWith(db).Query()

// Behaves like:
rows, err := db.Query("SELECT * FROM users WHERE username IN (?,?,?,?) LIMIT 3",
                      "moe", "larry", "curly", "shemp")

Squirrel makes conditional query building a breeze:

if len(q) > 0 {
    users = users.Where("name LIKE ?", fmt.Sprint("%", q, "%"))
}

Squirrel wants to make your life easier:

// StmtCache caches Prepared Stmts for you
dbCache := sq.NewStmtCache(db)

// StatementBuilder keeps your syntax neat
mydb := sq.StatementBuilder.RunWith(dbCache)
select_users := mydb.Select("*").From("users")

Squirrel loves PostgreSQL:

psql := sq.StatementBuilder.PlaceholderFormat(sq.Dollar)

// You use question marks for placeholders...
sql, _, _ := psql.Select("*").From("elephants").Where("name IN (?,?)", "Dumbo", "Verna").ToSql()

/// ...squirrel replaces them using PlaceholderFormat.
sql == "SELECT * FROM elephants WHERE name IN ($1,$2)"


/// You can retrieve id ...
query := sq.Insert("nodes").
    Columns("uuid", "type", "data").
    Values(node.Uuid, node.Type, node.Data).
    Suffix("RETURNING \"id\"").
    RunWith(m.db).
    PlaceholderFormat(sq.Dollar)

query.QueryRow().Scan(&node.id)

You can escape question marks by inserting two question marks:

SELECT * FROM nodes WHERE meta->'format' ??| array[?,?]

will generate with the Dollar Placeholder:

SELECT * FROM nodes WHERE meta->'format' ?| array[$1,$2]

FAQ

  • How can I build an IN query on composite keys / tuples, e.g. WHERE (col1, col2) IN ((1,2),(3,4))? (#104)

    Squirrel does not explicitly support tuples, but you can get the same effect with e.g.:

    sq.Or{
      sq.Eq{"col1": 1, "col2": 2},
      sq.Eq{"col1": 3, "col2": 4}}
    WHERE (col1 = 1 AND col2 = 2) OR (col1 = 3 AND col2 = 4)

    (which should produce the same query plan as the tuple version)

  • Why doesn't Eq{"mynumber": []uint8{1,2,3}} turn into an IN query? (#114)

    Values of type []byte are handled specially by database/sql. In Go, byte is just an alias of uint8, so there is no way to distinguish []uint8 from []byte.

  • Some features are poorly documented!

    This isn't a frequent complaints section!

  • Some features are poorly documented?

    Yes. The tests should be considered a part of the documentation; take a look at those for ideas on how to express more complex queries.

License

Squirrel is released under the MIT License.

Comments
  • Add UNION operator

    Add UNION operator

    I want to express query SELECT a FROM b WHERE col>? UNION SELECT a FROM c WHERE col<? but squirrel is not supporting UNION.

    I think, The following example code is good design for support UNION.

    q, attrs, err := sq.Union(
            sq.Select("*").From("a").Where("col > ?", hoge),
            sq.Select("*").From("b").Where("col < ?", piyo)).ToSql()
    
    q == "SELECT a FROM b WHERE col>? UNION SELECT a FROM c WHERE col<?"
    attrs == []interface{hoge, piyo}
    

    What do you think so? If you agree on this idea, I'll create patch and send PR. Thanks!

    opened by umisama 14
  • Make OR where queries

    Make OR where queries

    I can't find a way to express queries of the form SELECT * FROM table WHERE a OR b. According to the documentation (and the code I read so far), all the call of the Where method are put together using AND. It would be useful to be able to do OR too.

    Here is what I came up with while thinking about what the cleanest syntax would be:

    squirrel.Select("*").From("table").Where(squirrel.Or{
        conditionA,
        conditionB,
    })
    

    Here, the Where method is simply a variadic function taking a number of Or{} and And{} struct, which are merely slices of interfaces of the same kind than the current parameters of Where. That way would make complex imbrication of conditions quite easy.

    What do you think about it ? If you agree with that system, would you like some help to make it ? Cheers.

    opened by elwinar 12
  • Support of CASE operator in SELECT query

    Support of CASE operator in SELECT query

    Hi there! :) Here's a proposal for support of SELECT CASE statements (http://dev.mysql.com/doc/refman/5.0/en/control-flow-functions.html#operator_case). The commit contains test which shows an idea which is better to discuss before writing any real code. The interface is a matter of discussion. Rationale of this proposal: In current version of squirrel we have to pregenerate a subquery with SELECT CASE, like this, for example:

        var values map[string]string = loadValues()
        args = make([]interface{}, len(values)*2)
    
        buf.WriteString("(SELECT CASE")
        for k, v := range values {
            buf.WriteString(" WHEN value = ? THEN ?")
            args[i*2], args[i*2+1] = value, i
        }
        buf.WriteString(" END AS col1)")
    
        qb := squirrel.Select(buf.String()).From("sometable")
    

    But in this case nothing stops developer from making mistakes like forgotten escaped value. Also we can use squirrel to pregenerate a separate subquery, but the code is still messy. Support of SELECT CASE can be used to protect the developer :)

        var values map[string]string = loadValues()
        qb := squirrel.Select("").
            From("sometable").
            Case(nil)
    
        for k, v := range values {
            qb = qb.WhenThen(squirrel.Eq{"value": value}, i)
        }
    
        qb = qb.EndCase("col1")
    

    What do you think about this?

    Regards, Ivan.

    opened by elgris 11
  • ToSql() alternative?

    ToSql() alternative?

    Hi,

    I've been playing with your library (feels great!) to handle complex queries based on user input.

    I've had the idea of implementing my own intermediary "builder" methods that would help finalize the SQL query and thought about using "test examples" to quickly test the feature.

    // builders_test.go, run with `go test -v`
    
    package main
    
    import (
    	"fmt"
    
    	sq "gopkg.in/Masterminds/squirrel.v1"
    )
    
    func mustPrintSQL(builder sq.SelectBuilder) {
    	sql, params, err := builder.ToSql()
    	if err != nil {
    		panic(err)
    	}
    	fmt.Print(sql + " ")
    	fmt.Println(params)
    }
    
    func ExampleBuildSearchWithNoInput() {
    	builder := sq.Select("*").From("posts")
    	builder = buildSearch(builder, "", "")
    	mustPrintSQL(builder)
    	// Output: SELECT * FROM posts []
    }
    
    func ExampleBuildSearchWithWildcard() {
    	builder := sq.Select("*").From("posts")
    	builder = buildSearch(builder, "title", "*hello*")
    	mustPrintSQL(builder)
    	// Output: SELECT * FROM posts WHERE title ILIKE ? [%hello%]
    }
    
    func ExampleBuildSearchWithoutWildcard() {
    	builder := sq.Select("*").From("posts")
    	builder = buildSearch(builder, "title", "Hello!")
    	mustPrintSQL(builder)
    	// Output: SELECT * FROM posts WHERE title = ? [Hello!]
    }
    

    I understand the query parameters need to get passed separately to the db driver, but I was wondering if it was possible to inspect the final output, for debugging or testing, especially for very complex queries.

    So instead of:

    SELECT * FROM posts WHERE id = ? [1]
    

    I would see

    SELECT * FROM posts WHERE id = 1
    

    Maybe a ToSQLUnsafe(), DumpSQL() or DebugSQL() function?

    Any thoughts?

    opened by benjamin-thomas 10
  • add support for >,>=,<,<=

    add support for >,>=,<,<=

    This commit adds support for doing less than, less than or equal to, greater than, and greater than or equal to. I added tests as well.

    Please give feedback if not merged.

    opened by collinglass 10
  • DISTINCT ON (postgres only)

    DISTINCT ON (postgres only)

    The API currently includes DISTINCT SQL command, however postgres supports a DISTINCT ON command which allows you to define a column to compute distinctiveness, but still select columns as you usually would.

    Is there any plans for this, or, if I submit a PM, would this be an accepted feature?

    opened by jsimnz 9
  • Support subquery in FROM operator

    Support subquery in FROM operator

    I want to create query like SELECT * FROM (SELECT Max(someval), modified FROM sometable group by someval ) WHERE modified>?, but I can't find any way because squirrel is not supporting subquery in FROM operator.

    I have a idea to support subquery, see following code.

    q, _, _ := sq.Select("*").FromSubQuery(
            sq.Select("Max(someval)", "modified").
                    From("sometable").
                    GroupBy("someval")).
            Where("modifyed > ?", t)
    

    define:

    func (b SelectBuilder) FromSubQuery(SelectBuilder) SelectBuilder
    

    What do you think so? If you agree on this idea, I'll create patch and send PR. Thanks!

    opened by umisama 9
  • Please update license so GitHub will recognize its type in the API

    Please update license so GitHub will recognize its type in the API

    I am working on a license checker using the GitHub API and it comes back with NOASSERTION for this repo rather than MIT.

    Would it be possible for you to assert your license type in the Github Admin U so the API response would say MIT?

    https://docs.github.com/en/communities/setting-up-your-project-for-healthy-contributions/adding-a-license-to-a-repository

    Thank you in advance or considering.

    opened by mikeschinkel 7
  • Add `SuffixExpr` function to `SelectBuilder`

    Add `SuffixExpr` function to `SelectBuilder`

    I would like to do something like this:

    sq.
    	Select("col").
    	From("table").
    	LimitSelect(sq.
    		Select().
    		Column(sq.Case().
    			When("x = y", 10).
    			Else(0).From("table"),
    		),
    	)
    

    The SQL that would be generated by this would look something like:

    SELECT col FROM table 
    LIMIT (
      SELECT CASE WHEN x = y THEN 10 ELSE 0 END FROM table
    )
    

    This is valid for at least Postgres, as I'm able to run a query like this, however I am not sure about other flavors.

    I can sort of work around this with .Suffix but it makes me rather sad to have to do it that way 😢

    opened by slnt 7
  • Support *Context methods from database/sql in Go 1.8

    Support *Context methods from database/sql in Go 1.8

    New API in Go 1.8 allows passing a context.Context to Exec, Query and QueryRow (via the ExecContext, QueryContext and QueryRowContext methods, respectively.)

    Drivers can take advantage of this in numerous ways - we would particularly enjoy distributed tracing in Lightstep.

    opened by cv 7
  • Support FROM clause in UpdateBuilder

    Support FROM clause in UpdateBuilder

    Trying to construct an update statement like the following in squirrel is proving impossible:

    -- Update value in table a for all rows where the associated table b row has specified value in test_column
    UPDATE table_a
    SET table_a.value_column = $1
    FROM table_b
    WHERE table_b.a_id = table_a.id 
    and table_b.test_column =  $2
    

    I think UpdateBuilder should support a FROM clause to be able to leverage the abilities of postgres updates syntax: https://www.postgresql.org/docs/current/static/sql-update.html

    opened by scags9876 7
  • Misplaced params when using windows or subqueries

    Misplaced params when using windows or subqueries

    hi Team,

    We've been using Squirrel for a while now and (frankly) love it. The thing we're coming up against is there's not a lot of great support for subqueries. We're using sq.Alias to give a Window on a Column, but when params are passed in Squirrel seems to place them using the initial index despite passing Expr's around in what looks like a supported fashion.

    We've had to revert a code change that we found in testing here: https://github.com/autobrr/autobrr/pull/599

    This also seems to go wrong when using the WHERE IN clause, but admittedly that seems to be less supported (although greatly desired).

    Thoughts, directions, patches for us to test are greatly appreciated.

    opened by KyleSanderson 0
  • Sql doesn't match for SignleStore

    Sql doesn't match for SignleStore

    I'm trying to use this library for SingleStore MySQL DB. By Insertbuilder, I've got this query string.

    pgQb().Insert("hello").Columns("user_id", "name").Values(
    	 	3, "test"
    )
    

    INSERT INTO hello(user_id, name) VALUES ($1,$2) [3 "test"]

    Here is a valid query in SingleStore. INSERT INTO hello(user_id,name) VALUES (2, 'test323'); If I run this query in the SingleStore query field, it shows an error.

    Looking forward to hearing the solution soon. Best.

    opened by DedicatedDev 0
  • query.OrderBy working fine for DESC but not for ASC

    query.OrderBy working fine for DESC but not for ASC

    I have a Quotes table, there is a created_at time, in the format of DATETIME, I am using the line

    query = query.OrderBy(Quotes.created_at DESC")

    Which totally works fine, but when I use it as

    query = query.OrderBy(Quotes.created_at ASC")

    It is throwing error, I need it in both order as it is necessary, can anybody recommend a fix or something that will help? Thanks!

    opened by huzaifamk 0
  • No way to add options between INTO and VALUES on INSERT (likely MS SQL specific)

    No way to add options between INTO and VALUES on INSERT (likely MS SQL specific)

    MS SQL has a specific way for definition of returning values. It is similar to Postgres' RETURNING ID but should be placed in the middle of the query, so suffix is not applicable.

    Example (from stackoverflow):

    INSERT INTO MyTable(Name, Address, PhoneNo)
    OUTPUT INSERTED.ID
    VALUES ('Yatrix', '1234 Address Stuff', '1112223333')
    
    opened by dmakushin 1
  • how to use InsertBatch and UpdateBatch to exec muliti sql

    how to use InsertBatch and UpdateBatch to exec muliti sql

    liek this

    func (m *customLabelModel) InsertBatch(rows []*Label) error {
    
    	var labelInsertRows = strings.Join(stringx.Remove(labelFieldNames, "`id`"), ",")
    
    	insert := squirrel.Insert("").Into(m.table).Columns(labelInsertRows)
    	for _, row := range rows {
    		insert = insert.Values(row.Id, row.Title, row.PicUrl, row.Sort, row.CreateTime, row.UpdateTime, row.IsDel)
    	}
    	query, values, err := insert.ToSql()
    	if err != nil {
    		return err
    	}
    	_, err = m.conn.Exec(query, values...)
    	if err != nil {
    		return err
    	}
    	return nil
    }
    
    
    opened by jsonMark 0
Releases(v1.5.3)
Owner
null
Parses a file and associate SQL queries to a map. Useful for separating SQL from code logic

goyesql This package is based on nleof/goyesql but is not compatible with it any more. This package introduces support for arbitrary tag types and cha

null 0 Oct 20, 2021
Go-sql-reader - Go utility to read the externalised sql with predefined tags

go-sql-reader go utility to read the externalised sql with predefined tags Usage

null 0 Jan 25, 2022
a golang library for sql builder

Gendry gendry is a Go library that helps you operate database. Based on go-sql-driver/mysql, it provides a series of simple but useful tools to prepar

DiDi 1.4k Dec 26, 2022
convert sql to elasticsearch DSL in golang(go)

_____ _ _ ____ _____ ___ ____ ____ ___ _ | ____| | / \ / ___|_ _|_ _|/ ___|/ ___| / _ \ | | | _| | | / _ \ \___ \ |

Xargin 1k Jan 7, 2023
GoTSQL : A Better Way to Organize SQL codebase using Templates in Golang

GoTSQL - A Better Way to Organize SQL codebase using Templates in Golang Installation through Go Get command $ go get github.com/migopsrepos/gotsql In

null 17 Aug 17, 2022
SQL transaction wrapper on golang

TxWrapper TxWrapper is a sql transaction wrapper. It helps to exclude writing code for rollback and commit commands. Usage import ( "context"

Dmitry Stoletoff 1 Mar 14, 2022
Simple SQL escape and format for golang

sqlstring Simple SQL escape and format Escaping sql values //Format sql := sqlstring.Format("select * from users where name=? and age=? limit ?,?", "t

solar 9 Sep 4, 2022
BigQuery database/sql golang driver

BigQuery SQL Driver This library is compatible with Go 1.17+ Please refer to CHA

Viant, Inc 7 Dec 7, 2022
Spansqlx - Spanner sql pkgs with golang

spansqlx spanner sql pkgs install go get github.com/reiot777/spansqlx usage Bel

reiot 0 Jan 15, 2022
Golang REST Layer SQL Storage Handler

This REST Layer resource storage backend stores data in a SQL Database using database/sql.

Hyper38 GmbH 0 Feb 15, 2022
Go package for sharding databases ( Supports every ORM or raw SQL )

Octillery Octillery is a Go package for sharding databases. It can use with every OR Mapping library ( xorm , gorp , gorm , dbr ...) implementing data

BlasTrain Co., Ltd. 176 Dec 16, 2022
Prep finds all SQL statements in a Go package and instruments db connection with prepared statements

Prep Prep finds all SQL statements in a Go package and instruments db connection with prepared statements. It allows you to benefit from the prepared

Max Chechel 33 Dec 10, 2022
pggen - generate type safe Go methods from Postgres SQL queries

pggen - generate type safe Go methods from Postgres SQL queries pggen is a tool that generates Go code to provide a typesafe wrapper around Postgres q

Joe Schafer 207 Jan 3, 2023
🐳 A most popular sql audit platform for mysql

?? A most popular sql audit platform for mysql

Henry Yee 7.3k Jan 6, 2023
sqlx is a library which provides a set of extensions on go's standard database/sql library

sqlx is a library which provides a set of extensions on go's standard database/sql library. The sqlx versions of sql.DB, sql.TX, sql.Stmt, et al. all leave the underlying interfaces untouched, so that their interfaces are a superset on the standard ones. This makes it relatively painless to integrate existing codebases using database/sql with sqlx.

Jason Moiron 13.1k Jan 7, 2023
A tool to run queries in defined frequency and expose the count as prometheus metrics. Supports MongoDB and SQL

query2metric A tool to run db queries in defined frequency and expose the count as prometheus metrics. Why ? Product metrics play an important role in

S Santhosh Nagaraj 19 Jul 1, 2022
Dumpling is a fast, easy-to-use tool written by Go for dumping data from the database(MySQL, TiDB...) to local/cloud(S3, GCP...) in multifarious formats(SQL, CSV...).

?? Dumpling Dumpling is a tool and a Go library for creating SQL dump from a MySQL-compatible database. It is intended to replace mysqldump and mydump

PingCAP 267 Nov 9, 2022
Universal command-line interface for SQL databases

usql A universal command-line interface for PostgreSQL, MySQL, Oracle Database, SQLite3, Microsoft SQL Server, and many other databases including NoSQ

XO 7.8k Jan 9, 2023
auto generate sql from gorm model struct

gorm2sql: auto generate sql from gorm model struct A Swiss Army Knife helps you generate sql from gorm model struct. Installation go get github.com/li

Danieliu 116 Dec 22, 2022