PostgreSQL style Parser splitted from CockroachDB

Overview

What's this

PostgreSQL style Parser splitted from CockroachDB

See: Complex SQL format example

I tried to import github.com/cockroachdb/cockroach/pkg/sql/parser, but the dependencies is too complex to make it work.

To make things easy, I did these things:

  1. Copy all the pkg/sql/parser, pkg/sql/lex and simplify the dependencies
  2. Simplify the Makefile to just generate the goyacc stuff
  3. Add the goyacc generated files in parser and lex to make go get work easily, see the .gitignore files
  4. Trim the etcd dependency, see the go.mod
  5. Rename all test file except some pkg/sql/parser tests
  6. Add all necessary imports to vendor
  7. Remove the panic of meeting unregistried functions, see the WrapFunction
  8. Other nasty things make the parser just work that I forgot :p

Features

  • Pure golang implementation
  • Almost full support of PostgreSQL (cockroachdb style PostgreSQL)

SQL Standard Compliance

The code is derived from CockroachDB v20.1.11 which supports most of the major features of SQL:2011. See:

🚧 🚧 🚧 still under construction 🚧 🚧 🚧

Progress

  • 2020-02-16 github.com/auxten/postgresql-parser/pkg/sql/parser Unit tests works now!

Todo

  • Fix more unit tests
  • Make built-in function parsing work
Comments
  • Update dependencies to latest version

    Update dependencies to latest version

    I updated all the dependencies to the latest versions and removed the replaced dependency.

    @auxten Please let me know if this is not OK and if you based the project on certain dependencies. Then we can pin those dependencies. Also, I am thinking about various ways to contribute to this project. Please let me know of the next steps or create tickets, so we can talk about various aspects of it.

    opened by mostafa 5
  • cleaning up Makefile and various bug fixes inside the build processes

    cleaning up Makefile and various bug fixes inside the build processes

    This PR removes most of the unused/dead Makefile code relevant to the CockroachDB project. This PR also fixes some small issues inside the build processes such as the import paths defined inside sql.y and the non-alphanumeric build constraint.

    All generated files could be generated using make generate. Make generate does not yet ensure that all the necessary tools are installed. This will be included in a future PR.

    NOTE: the vendor directory inside the project is currently outdated. I would like to suggest removing the vendor directory since all import issues have been resolved inside this PR.

    opened by jeroenrinzema 2
  • run make all get error   no /pkg/cmd/prereqs

    run make all get error no /pkg/cmd/prereqs

    (echo "// Code generated by goyacc. DO NOT EDIT.";
    echo "// GENERATED FILE DO NOT EDIT";
    cat pkg/sql/parser/gen/sql.go.tmp |
    sed -E 's/^const ([A-Z][_A-Z0-9]) =.$/const \1 = lex.\1/g') > pkg/sql/parser/sql.go.tmp || rm pkg/sql/parser/sql.go.tmp mv -f pkg/sql/parser/sql.go.tmp pkg/sql/parser/sql.go goimports -w pkg/sql/parser/sql.go (echo "// Code generated by make. DO NOT EDIT.";
    echo "// GENERATED FILE DO NOT EDIT";
    echo;
    echo "package lex";
    echo;
    grep '^const [A-Z][_A-Z0-9]* ' pkg/sql/parser/gen/sql.go.tmp) > pkg/sql/lex/tokens.go.tmp || rm pkg/sql/lex/tokens.go.tmp mv -f pkg/sql/lex/tokens.go.tmp pkg/sql/lex/tokens.go go install -v ./pkg/cmd/prereqs can't load package: package ./pkg/cmd/prereqs: cannot find package "." in:

    opened by finch039 2
  • Dependency security warning

    Dependency security warning

    I am getting a high severity security warning, can we update this dependency?

    Improper Input Validation in GoGo Protobuf

    Package Affected versions Patched version
    github.com/gogo/protobuf(Go) < 1.3.2 1.3.2

    An issue was discovered in GoGo Protobuf before 1.3.2. plugin/unmarshal/unmarshal.go lacks certain index validation, aka the "skippy peanut butter" issue.

    opened by a631807682 1
  • hlc_clock_device_linux.go

    hlc_clock_device_linux.go

    github.com/auxten/postgresql-parser/pkg/util/hlc ../../auxten/postgresql-parser/pkg/util/hlc/hlc_clock_device_linux.go:51:11: cannot use ctx (type context.Context) as type string in argument to logrus.Infof

    Using arch. Commented out that single log write on line 51 and 'log' import and the example worked.

    Thanks!

    opened by 5k3105 1
  • build: upgrade dep version

    build: upgrade dep version

    Desc

    Upgrade github.com/gogo/protobuf version close #17

    Note

    I just upgrade package version and i don't know how to test it. I only tried example to be executable.

    opened by a631807682 0
  • Removed the unused storage engine protobuffers

    Removed the unused storage engine protobuffers

    I am not sure if you are planning to use this code for future features. I would like to suggest to remove the storage package if there are no plans made for this package.

    opened by jeroenrinzema 0
  • Move examples into separate directories

    Move examples into separate directories

    go vet currently throws the error that main is redeclared inside the example directory. This PR fixes this issue by moving the examples into separate directories.

    opened by jeroenrinzema 0
  • Switch

    Switch "STRING" for "TEXT" string type by default.

    CockroachDB uses "STRING" (but accepts "TEXT" for compatibility) and PostgreSQL uses "TEXT" always.

    Fixes https://github.com/auxten/postgresql-parser/issues/15

    opened by MFAshby 0
  • Data type STRING not recognized

    Data type STRING not recognized

    Hi,

    I'm experimenting with postgresql-parser to intercept and rewrite some queries to a postgresql database.

    One problem I have encountered is that the formatter will output "STRING" by default for string data type. PostgreSQL does not support this, instead the data type name is "TEXT". CockroachDB supports "TEXT" as an alias for "STRING" [1]

    In order to make query-rewriting work, I've patched postgresql-parser to output "TEXT" data type by default instead of "STRING". I think this could be a reasonable default since CockroachDB supports both.

    [1] https://www.cockroachlabs.com/docs/stable/string.html

    opened by MFAshby 0
  • Data type STRING not recognized

    Data type STRING not recognized

    Hi,

    I'm experimenting with postgresql-parser to intercept and rewrite some queries to a postgresql database.

    One problem I have encountered is that the formatter will output "STRING" by default for string data type. PostgreSQL does not support this, instead the data type name is "TEXT". CockroachDB supports "TEXT" as an alias for "STRING" [1]

    In order to make query-rewriting work, I've patched postgresql-parser to output "TEXT" data type by default instead of "STRING". I think this could be a reasonable default since CockroachDB supports both.

    [1] https://www.cockroachlabs.com/docs/stable/string.html

    opened by MFAshby 0
  • 无法解析带有类型转换符

    无法解析带有类型转换符 "::" 的语句

    解析此种类型的sql失败:

    SELECT COUNT ( * ) FROM pg_class C LEFT JOIN pg_namespace n ON n.oid = C.relnamespace WHERE C.relkind = ANY ( '{r,v,m}' :: CHAR [] ) UNION SELECT COUNT ( * ) FROM pg_attribute A JOIN pg_class C ON A.attrelid = C.oid JOIN pg_namespace n ON C.relnamespace = n.oid JOIN pg_type tp ON tp.typelem = A.atttypid WHERE A.attnum > 0 UNION SELECT COUNT ( * ) FROM information_schema.routines

    出错的位置在 "::" 操作符

    opened by Aaron3S 1
Releases(v1.0.1)
  • v1.0.1(Aug 3, 2022)

    What's Changed

    • Update README.md by @wule61 in https://github.com/auxten/postgresql-parser/pull/10
    • build: upgrade dep version by @a631807682 in https://github.com/auxten/postgresql-parser/pull/18
    • Remove vendored dependencies by @mostafa in https://github.com/auxten/postgresql-parser/pull/19

    New Contributors

    • @wule61 made their first contribution in https://github.com/auxten/postgresql-parser/pull/10
    • @a631807682 made their first contribution in https://github.com/auxten/postgresql-parser/pull/18
    • @mostafa made their first contribution in https://github.com/auxten/postgresql-parser/pull/19

    Full Changelog: https://github.com/auxten/postgresql-parser/compare/v1.0.0...v1.0.1

    Source code(tar.gz)
    Source code(zip)
  • v0.4(Aug 4, 2021)

  • v0.2(Mar 8, 2021)

Owner
auxten
For freedom
auxten
pg_timetable: Advanced scheduling for PostgreSQL

pg_timetable: Advanced scheduling for PostgreSQL pg_timetable is an advanced job scheduler for PostgreSQL, offering many advantages over traditional s

CYBERTEC PostgreSQL International GmbH 787 Sep 21, 2022
Cross-platform client for PostgreSQL databases

pgweb Web-based PostgreSQL database browser written in Go. Overview Pgweb is a web-based database browser for PostgreSQL, written in Go and works on O

Dan Sosedoff 7.5k Sep 21, 2022
pREST (PostgreSQL REST), simplify and accelerate development, ⚡ instant, realtime, high-performance on any Postgres application, existing or new

pREST pREST (PostgreSQL REST), simplify and accelerate development, instant, realtime, high-performance on any Postgres application, existing or new P

pREST 3.4k Sep 20, 2022
Enhanced PostgreSQL logical replication

pgcat - Enhanced postgresql logical replication Why pgcat? Architecture Build from source Install Run Conflict handling Table mapping Replication iden

jinhua luo 357 Sep 10, 2022
Interactive client for PostgreSQL and MySQL

dblab Interactive client for PostgreSQL and MySQL. Overview dblab is a fast and lightweight interactive terminal based UI application for PostgreSQL a

Daniel Omar Vergara Pérez 544 Sep 28, 2022
WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

null 2.3k Sep 28, 2022
Worker failover support for PostgreSQL Citus extension using pg_auto_failover.

citus-failover Worker failover support for citus community version using pg_auto_failover. What is this? This is a simple service to monitor changes i

Navid Zarepak 22 Aug 3, 2022
Interactive terminal user interface and CLI for database connections. MySQL, PostgreSQL. More to come.

?? dbui dbui is the terminal user interface and CLI for database connections. It provides features like, Connect to multiple data sources and instance

Kanan Rahimov 105 Sep 14, 2022
Modify orca-zhang/borm in order to use in PostgreSQL

borm ??️ 针对 orca-zhang/borm 进行了修改,暂时只能兼容PostgreSQL 原因 在b站时候用过borm,用起来感觉非常简洁 自己学校里用PostgreSQL比较多 可变条件真的非常好用 问题 首先需要注意的是,这是写给PG的 PG 根本不存在某些 MySQL 独有的函数

null 14 Aug 24, 2022
Cross-platform client for PostgreSQL databases

pgweb Web-based PostgreSQL database browser written in Go. Overview Pgweb is a web-based database browser for PostgreSQL, written in Go and works on O

Dan Sosedoff 7.5k Sep 24, 2022
PolarDB Cluster Manager is the cluster management component of PolarDB for PostgreSQL, responsible for topology management, high availability, configuration management, and plugin extensions.

What is PolarDB Cluster Manager PolarDB Cluster Manager is the cluster management component of PolarDB for PostgreSQL, responsible for topology manage

null 8 Dec 15, 2021
A simple Golang-based application that queries a PostgreSQL database

Qwik-E-Mart Demo App A simple Golang-based application that queries a PostgreSQL database named qwikemart to read and return customer data stored in t

Joe Garcia 0 Nov 6, 2021
CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables.

The open-source cloud asset inventory backed by SQL. CloudQuery extracts, transforms, and loads your cloud assets into normalized PostgreSQL tables. C

CloudQuery 2.5k Sep 22, 2022
Implemented PostgreSQL with Golang

Customer Information Web Api Implemented PostgreSQL with Golang docker run --name postgresql-container -p 5432:5432 -e POSTGRES_PASSWORD=Password! -d

bseyhan 2 Nov 15, 2021
Typescript type declaration to PostgreSQL CREATE TABLE converter

ts2psql NOTE: This is WIP. Details in this readme are ideal state. Current usage: go build && ./ts2psql (or go build && ts2psql if on Windows OS). A s

null 1 Jan 13, 2022
An experimental toolkit for injecting alternate authentication strategies into a PostgreSQL-compatible wire format.

PG Auth Proxy This is an experimental toolkit for injecting alternate authentication strategies into a PostgreSQL-compatible wire format. This is a pr

Cockroach Labs 1 Jan 20, 2022
Thin clones of PostgreSQL to build powerful development, test, QA, staging environments

Database Lab Engine (DLE) ⚡ Blazing-fast cloning of PostgreSQL databases ?? Thin clones of PostgreSQL to build powerful development, test, QA, staging

Postgres.ai 1.4k Sep 23, 2022
Dugopg - PostgreSQL tool For Golang

⚡️ DuGoPG Installation go get -u github.com/durudex/dugopg Example import (

Durudex 5 May 9, 2022
📊 parallel black box PostgreSQL unit tests run against a real database.

?? psql-docker-tests-example Parallel black box PostgreSQL unit tests run against a real database. Consider reading the Medium Story first. This packa

Adrian Brad 3 Sep 15, 2022