simply SQL Parser for Go ( powered by vitess and TiDB )

Overview

vitess-sqlparser

Simply SQL and DDL parser for Go (powered by vitess and TiDB ) this library inspired by https://github.com/xwb1989/sqlparser

(original source : https://github.com/youtube/vitess/tree/master/go/vt/sqlparser)

Why

xwb1989/sqlparser is famous sql parser in Go.
But it cannot parse some query (like offset or bulk insert...) because it customizes vitess's sql parser.

Also, some libraries use from vitess sql parser directly. But vitess's sql parser only partial supports DDL parsing.

We want to perfectly support parsing for SQL and DDL.
Therefore we use vitess sql parser directly and also use TiDB parser for DDL parsing.

Compare SQL parser libraries in Go

library supports offset (or other complexity) query supports DDL
xwb1989/sqlparser
zhenjl/sqlparser
knocknote/vitess-sqlparser

Installation

[NOTE] Required Go version more than 1.9

go get -u github.com/knocknote/vitess-sqlparser

Examples

package main

import (
 	"fmt"
	"github.com/blastrain/vitess-sqlparser/sqlparser"
)

func main() {
	stmt, err := sqlparser.Parse("select * from user_items where user_id=1 order by created_at limit 3 offset 10")
	if err != nil {
		panic(err)
	}
	fmt.Printf("stmt = %+v\n", stmt)
}
Issues
  • add Formatter for CreateTable statement

    add Formatter for CreateTable statement

    Add formatter for CreateTable statement, also add TableOption and ColumnOption structure for it. But still not support PARTITION option.

    DEMO

    package main
    
    import (
    	"fmt"
    
    	"github.com/knocknote/vitess-sqlparser/sqlparser"
    )
    
    func main() {
    	stmt, err := sqlparser.Parse(`
    CREATE TABLE partition_example (
      id bigint(20) unsigned NOT NULL AUTO_INCREMENT,
      created_at datetime NOT NULL,
      PRIMARY KEY (id, created_at)
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8
    PARTITION BY RANGE COLUMNS(created_at)
    (PARTITION p201809 VALUES LESS THAN ('2018-10-01') ENGINE = InnoDB,
     PARTITION p201810 VALUES LESS THAN ('2018-11-01') ENGINE = InnoDB,
     PARTITION p201811 VALUES LESS THAN ('2018-12-01') ENGINE = InnoDB,
     PARTITION p201812 VALUES LESS THAN ('2019-01-01') ENGINE = InnoDB,
     PARTITION p201901 VALUES LESS THAN ('2019-02-01') ENGINE = InnoDB,
     PARTITION p201902 VALUES LESS THAN ('2019-03-01') ENGINE = InnoDB,
     PARTITION p201903 VALUES LESS THAN ('2019-04-01') ENGINE = InnoDB,
     PARTITION p201904 VALUES LESS THAN ('2019-05-01') ENGINE = InnoDB,
     PARTITION p201905 VALUES LESS THAN ('2019-06-01') ENGINE = InnoDB,
     PARTITION p201906 VALUES LESS THAN ('2019-07-01') ENGINE = InnoDB,
     PARTITION p201907 VALUES LESS THAN ('2019-08-01') ENGINE = InnoDB,
     PARTITION p201908 VALUES LESS THAN ('2019-09-01') ENGINE = InnoDB,
     PARTITION p201909 VALUES LESS THAN ('2019-10-01') ENGINE = InnoDB,
     PARTITION p201910 VALUES LESS THAN ('2019-11-01') ENGINE = InnoDB,
     PARTITION p201911 VALUES LESS THAN ('2019-12-01') ENGINE = InnoDB);
    `)
    	if err != nil {
    		panic(err)
    	}
    	fmt.Println(sqlparser.String(stmt))
    	/*
    		       output the following
    
    			   CREATE TABLE `partition_example` (
    			           `id` bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT,
    			           `created_at` datetime NOT NULL,
    			           PRIMARY KEY  (`id`, `created_at`)
    			   ) ENGINE=InnoDB DEFAULT CHARSET=utf8
    	*/
    }
    
    opened by goccy 2
  • How to get 'table name'?

    How to get 'table name'?

    use demo

    stmt, err := sqlparser.Parse fmt.Printf("stmt = %+v\n", stmt) The output "stmt = &{Cache: Comments:[] Distinct: Hints: SelectExprs:[0xc420088e10] From:[0xc420089110 0xc420089140] Where:0xc42000a5c0 GroupBy:[] Having: OrderBy:[] Limit: Lock:}"

    How do I get the value in the From please help me

    opened by feloxx 2
  • New to sqlparser, how could I quickly understand AST

    New to sqlparser, how could I quickly understand AST

    Hi All, I think you guys must be expert on sql parser, how could I understand AST as well as the whole design on this parser easily? Even fetch the tableName looks pretty heavy, I am pretty confused somehow.

    tableName := stmt.(*sqlparser.Select).From[0].(*sqlparser.AliasedTableExpr).Expr.(sqlparser.TableName).Name.String()
    
    opened by skywhat 1
  • add vendor directory

    add vendor directory

    resolve https://github.com/knocknote/vitess-sqlparser/issues/7

    Already fixed dependencies in this repository. To install by go get (-u) github.com/knocknote/vitess-sqlparser , add vendor directory generated by glide

    opened by goccy 1
  • support truncate table query

    support truncate table query

    vitess's sqlparser doesn't support such as REPAIR, OPTIMIZE, or TRUNCATE statement. TiDB parser supports these statements. Therefore use this and supports truncate table query.

    opened by goccy 1
  • add TiDB parser for parsing DDL

    add TiDB parser for parsing DDL

    vitess parser has not support fully parsing for DDL. So use https://github.com/ruiaylin/sqlparser for parsing DDL. Add directly dependency built package because ruiaylin/sqlparser must compile after downloaded.

    support like the following query

    CREATE TABLE sample (
      id bigint(20) unsigned NOT NULL,
      other_id bigint(20) unsigned NOT NULL,
      enum_column enum('a','b','c','d') DEFAULT NULL,
      int_column int(10) DEFAULT '0',
      PRIMARY KEY (id)
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
    

    parsing results

    &sqlparser.CreateTable{
      DDL: &sqlparser.DDL{
        Action: "create",
        Table:  sqlparser.TableName{
          Name: sqlparser.TableIdent{
            v: "",
          },
          Qualifier: sqlparser.TableIdent{
            v: "",
          },
        },
        NewName: sqlparser.TableName{
          Name: sqlparser.TableIdent{
            v: "sample",
          },
          Qualifier: sqlparser.TableIdent{
            v: "",
          },
        },
        IfExists: false,
      },
      Columns: []*sqlparser.ColumnDef{
        &sqlparser.ColumnDef{
          Name:  "id",
          Type:  "bigint",
          Elems: []string{},
        },
        &sqlparser.ColumnDef{
          Name:  "other_id",
          Type:  "bigint",
          Elems: []string{},
        },
        &sqlparser.ColumnDef{
          Name:  "enum_column",
          Type:  "enum",
          Elems: []string{
            "a",
            "b",
            "c",
            "d",
          },
        },
        &sqlparser.ColumnDef{
          Name:  "int_column",
          Type:  "int",
          Elems: []string{},
        },
      },
    }
    
    opened by goccy 1
  • ports from vitess sqlparser

    ports from vitess sqlparser

    [changes summary]

    • replace github.com/youtube/vitess/xxx to github.com/knocknote/vitess-sqlparser
    • import source from https://github.com/youtube/vitess/tree/master/go/vt/sqlparser and it's dependencies

    original source revision : https://github.com/youtube/vitess/commit/a564f7a7bc48d68b697dbf154af2b78f6bb5b8a3

    opened by goccy 1
  • Can't install

    Can't install

    go get github.com/knocknote/vitess-sqlparser can't load package: package github.com/knocknote/vitess-sqlparser: no Go files in /home/kh/go/src/github.com/knocknote/vitess-sqlparser

    opened by Kaizhe 0
Owner
BlasTrain Co., Ltd.
BlasTrain Co., Ltd.
Vitess is a database clustering system for horizontal scaling of MySQL.

Vitess Vitess is a database clustering system for horizontal scaling of MySQL through generalized sharding. By encapsulating shard-routing logic, Vite

Vitess 14.3k Jun 29, 2022
Dumpling is a fast, easy-to-use tool written by Go for dumping data from the database(MySQL, TiDB...) to local/cloud(S3, GCP...) in multifarious formats(SQL, CSV...).

?? Dumpling Dumpling is a tool and a Go library for creating SQL dump from a MySQL-compatible database. It is intended to replace mysqldump and mydump

PingCAP 260 Jun 10, 2022
PgSQL compatible on distributed database TiDB

TiDB for PostgreSQL Introduction TiDB for PostgreSQL is an open source launched by Digital China Cloud Base to promote and integrate into the open sou

DigitalChinaOpenSource 334 Jun 25, 2022
A tool for finding corrupted data rows in TiDB

tidb-bad-rows A tool for finding corrupted data rows in TiDB. It scans the target table and using a divide and conquer paradigm to locate all corrupte

Wenxuan 1 Nov 17, 2021
SQL Parser implemented in Go

sqlparser Go package for parsing MySQL SQL queries. Notice The backbone of this repo is extracted from vitessio/vitess. Inside vitessio/vitess there i

Wenbin Xiao 1.1k Jun 22, 2022
Simple SQL parser

gosqlparser gosqlparser is a simple SQL parser. Installation As simple as: go get github.com/krasun/gosqlparser Usage ... Supported Statements CREATE

Dmytro Krasun 32 May 7, 2022
Parses a file and associate SQL queries to a map. Useful for separating SQL from code logic

goyesql This package is based on nleof/goyesql but is not compatible with it any more. This package introduces support for arbitrary tag types and cha

null 0 Oct 20, 2021
write APIs using direct SQL queries with no hassle, let's rethink about SQL

SQLer SQL-er is a tiny portable server enables you to write APIs using SQL query to be executed when anyone hits it, also it enables you to define val

Mohammed Al Ashaal 2k Jun 24, 2022
Go-sql-reader - Go utility to read the externalised sql with predefined tags

go-sql-reader go utility to read the externalised sql with predefined tags Usage

null 0 Jan 25, 2022
Generate applications powered by your database.

DBCore (ALPHA) DBCore is a code generator build around database schemas and an API specification. Included with DBCore are templates for generating a

Phil Eaton 508 Jun 15, 2022
Goproxy4mysql - Kingshard- a high-performance proxy for MySQL powered by Go

kingshard 中文主页 Fork from github.com/flike/kingshard Overview kingshard is a high

null 1 Jan 7, 2022
A tool to run queries in defined frequency and expose the count as prometheus metrics. Supports MongoDB and SQL

query2metric A tool to run db queries in defined frequency and expose the count as prometheus metrics. Why ? Product metrics play an important role in

S Santhosh Nagaraj 18 Apr 21, 2022
OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.

OctoSQL OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases, streaming sources and file formats using

Jacob Martin 3.1k Jun 27, 2022
WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

null 2.2k Jun 28, 2022
null 3 Mar 7, 2022
PostgreSQL style Parser splitted from CockroachDB

What's this PostgreSQL style Parser splitted from CockroachDB See: Complex SQL format example

auxten 133 Jun 27, 2022
Prep finds all SQL statements in a Go package and instruments db connection with prepared statements

Prep Prep finds all SQL statements in a Go package and instruments db connection with prepared statements. It allows you to benefit from the prepared

Max Chechel 31 Jan 23, 2022
SQL Optimizer And Rewriter

文档 | FAQ | 变更记录 | 路线图 | English SOAR SOAR(SQL Optimizer And Rewriter) 是一个对 SQL 进行优化和改写的自动化工具。 由小米人工智能与云平台的数据库团队开发与维护。 功能特点 跨平台支持(支持 Linux, Mac 环境,Wind

Xiaomi 7.9k Jun 22, 2022
Additions to Go's database/sql for super fast performance and convenience.

gocraft/dbr (database records) gocraft/dbr provides additions to Go's database/sql for super fast performance and convenience. $ go get -u github.com/

null 1.7k Jun 29, 2022