"Go SQL DB" is a relational database that supports SQL queries for research purposes

Related tags

Database go-sqldb
Overview

Go SQL DB

中文

"Go SQL DB" is a relational database that supports SQL queries for research purposes. The main goal is to show the basic principles and key design of a relational database to database enthusiasts. Therefore, to easily understand, a lot of tricks but not very rigorous design was adopted, and the amount of code was controlled at about 2000 lines (including 400 lines of unit tests).

Features

  1. Pure Golang implementation, does not rely on any third-party packages. Goconvey was only introduced in unit tests
  2. Unit test coverage ≈ 73.5%

Storage Engine

  1. Special Thanks to Let's Build a Simple Database
  2. Data retrieval structure based on B+Tree
  3. Disk persistence engine based on 4KB paging
  4. Close to POD (Plain Old Data) speed serialization & deserialization

SQL Parser

  1. Tokenizer is implemented based on text/scanner
  2. Support simple SELECT, INSERT syntax
    1. SELECT supports WHERE of numeric type
    2. Support LIMIT, but not support ORDER BY temporarily
  3. If you want to know how the SQL Parser that can be used in the production environment is implemented, please refer to the SQL Parser that I stripped from CockroachDB and supports the SQL-2011 standard

Execution Planner

  1. Select Implementation based on Volcano Model
  2. HTTP-based query and insert interface

Known Issues

  1. No DDL is implemented for the time being, only a fixed Schema
    struct Row {
        Id uint32
        Sex byte
        Age uint8
        Username [32]byte
        Email [128]byte
        Phone [64]byte
    }
  2. For limited support for SQL syntax, see Test Cases
  3. Tokenizer is based on a clever implementation of the Golang language itself, there will be problems with the support of special characters in some strings, which can be solved by quoting strings with "

How to run

  1. Run

    go run . test.db
  2. INSERT

    Execute INSERT INTO table (id, username, email) VALUES (10, auxten, "auxtenwpc_gmailcom")

    BY accessing: http://localhost:8080/exec?q=INSERT%20INTO%20table%20(id,%20username,%20email)%20VALUES%20(10,%20auxten,%20%22auxtenwpc_gmailcom%22)

  3. SELECT

    Query SELECT * FROM table WHERE id > 3 LIMIT 10

    BY accessing: http://localhost:8080/query?q=SELECT%20*%20FROM%20table%20WHERE%20id%20%3E%203%20LIMIT%2010

Thanks

  1. SQL-2011 SQL Parser
  2. Marshal/Unmarshal Code generation
  3. Document-oriented, embedded SQL database: genji
  4. CockroachDB
  5. Let's Build a Simple Database
Issues
  • Support for DDL

    Support for DDL

    Hey @auxten! First of all congratulations on launching this project!! I'm actually excited to crate the first Issue!! I was thinking if I could help out in implementing the DDL. I've myself implemented a simple DB in Python and although it did not support DDL, it was possible to create custom tables(all columns are strings) and the start. You can check it out here NaiveDB. I'm a newbie to the world of Golang, and therefore would like some guidance from you on how i would go about helping you out.

    opened by rajat-008 2
  • Question of execution plan

    Question of execution plan

    感谢你开源的项目啊,对于学习数据库很有帮助。 请教一个问题:我看测试用例 SELECT id, username, FROM table WHERE id > 5 AND id < 7 LIMIT 3 查询的时候,过程是,先遍历所有的row,再在内存中过滤,最后返回满足要求的row。 我理解的是应该会先根据where条件在B+树种过滤好再返回吧,而不是在内存中,而且也只会返回 id, username 其他的字段不会返回(这个字段过滤是在内存中吗?还是说在存储层就已经过滤了?)。还是我哪里理解错了?谢谢。

    Translation of the question:

    Thank you for your open source project, it is very helpful for learning database theory. Ask a question: I see the test case SELECT id, username, FROM table WHERE id> 5 AND id <7 LIMIT 3 When querying, the process is to first traverse all rows, then filter in memory, and finally return the row that meets the requirements. . What I understand is that it should be filtered in the B-Tree based on where conditions before returning, not in memory, and only id will be returned, and other fields of username will not be returned (is this field filtered in memory? Or Say it has been filtered at the storage layer?). Or am I getting it wrong? Thank you.

    opened by yiippee 1
  • error running examples (firefox only?)

    error running examples (firefox only?)

    when trying example i get error.

    image

    this is due to firefox being a json browser and the examples are returning plain/text but saying they are returning json.

    writer.Header().Set("Content-Type", "application/json")
    q := request.URL.Query()
    query := q.Get("q")
    if query != "" {
    	var (
    		ast *parser.SelectTree
    	)
    
    	p := &parser.Parser{}
    	if p.GetSQLType(query) != parser.SELECT {
    		_, _ = fmt.Fprintf(writer, "not a SELECT statement")
    		return
    	}
    
    

    for all the error printing lines, this will fix;

    example; http.Error(writer, "not a SELECT statement", http.StatusBadRequest) from _, _ = fmt.Fprintf(writer, "not a SELECT statement")

    but the data written will also have to be converted to json.

    opened by splace 0
Owner
auxten
a Database Guy
auxten
The lightweight, distributed relational database built on SQLite.

rqlite is a lightweight, distributed relational database, which uses SQLite as its storage engine. Forming a cluster is very straightforward, it grace

rqlite 10.6k Jun 26, 2022
A MySQL-compatible relational database with a storage agnostic query engine. Implemented in pure Go.

go-mysql-server is a SQL engine which parses standard SQL (based on MySQL syntax) and executes queries on data sources of your choice. A simple in-memory database and table implementation are provided, and you can query any data source you want by implementing a few interfaces.

DoltHub 816 Jun 26, 2022
Go reproduction of Bustub--a simple relational database system.

Bustub in Golang Bustub is the course project of CMU15-445 Database System, which is a simple relational database system. This repo is a golang reprod

Zhang Each 1 Dec 18, 2021
☄ The golang convenient converter supports Database to Struct, SQL to Struct, and JSON to Struct.

Gormat - Cross platform gopher tool The golang convenient converter supports Database to Struct, SQL to Struct, and JSON to Struct. 中文说明 Features Data

永林 271 Jun 21, 2022
Export output from pg_stat_activity and pg_stat_statements from Postgres into a time-series database that supports the Influx Line Protocol (ILP).

pgstat2ilp pgstat2ilp is a command-line program for exporting output from pg_stat_activity and pg_stat_statements (if the extension is installed/enabl

Zikani Nyirenda Mwase 4 Dec 15, 2021
Owl is a db manager platform,committed to standardizing the data, index in the database and operations to the database, to avoid risks and failures.

Owl is a db manager platform,committed to standardizing the data, index in the database and operations to the database, to avoid risks and failures. capabilities which owl provides include Process approval、sql Audit、sql execute and execute as crontab、data backup and recover .

null 35 Jun 17, 2022
This is a simple graph database in SQLite, inspired by "SQLite as a document database".

About This is a simple graph database in SQLite, inspired by "SQLite as a document database". Structure The schema consists of just two structures: No

Denis Papathanasiou 1.1k Jun 21, 2022
Hard Disk Database based on a former database

Hard Disk Database based on a former database

null 0 Nov 1, 2021
Simple key value database that use json files to store the database

KValDB Simple key value database that use json files to store the database, the key and the respective value. This simple database have two gRPC metho

Francisco Santos 0 Nov 13, 2021
CockroachDB - the open source, cloud-native distributed SQL database.

CockroachDB is a cloud-native SQL database for building global, scalable cloud services that survive disasters. What is CockroachDB? Docs Quickstart C

CockroachDB 25k Jun 23, 2022
A decentralized, trusted, high performance, SQL database with blockchain features

中文简介 CovenantSQL(CQL) is a Byzantine Fault Tolerant relational database built on SQLite: ServerLess: Free, High Availabile, Auto Sync Database Service

CovenantSQL 1.3k Jun 21, 2022
LBADD: An experimental, distributed SQL database

LBADD Let's build a distributed database. LBADD is an experimental distributed SQL database, written in Go. The goal of this project is to build a dat

Tom Arrell 379 May 22, 2022
A course to build the SQL layer of a distributed database.

TinySQL TinySQL is a course designed to teach you how to implement a distributed relational database in Go. TinySQL is also the name of the simplifed

TiDB Incubator 979 Jun 26, 2022
Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository.

Dolt is a SQL database that you can fork, clone, branch, merge, push and pull just like a git repository. Connect to Dolt just like any MySQL database to run queries or update the data using SQL commands. Use the command line interface to import CSV files, commit your changes, push them to a remote, or merge your teammate's changes.

DoltHub 12.2k Jun 28, 2022
DonutDB: A SQL database implemented on DynamoDB and SQLite

DonutDB: A SQL database implemented on DynamoDB and SQLite

Peter Sanford 96 Jun 15, 2022
This is a simple Golang application that executes SQL commands to clean up a mirror node's database.

This is a simple Golang application that executes SQL commands to clean up a mirror node's database.

Tom 1 Jan 24, 2022
A simple, fast, embeddable, persistent key/value store written in pure Go. It supports fully serializable transactions and many data structures such as list, set, sorted set.

NutsDB English | 简体中文 NutsDB is a simple, fast, embeddable and persistent key/value store written in pure Go. It supports fully serializable transacti

徐佳军 2.3k Jun 26, 2022
An embedded key/value database for Go.

bbolt bbolt is a fork of Ben Johnson's Bolt key/value store. The purpose of this fork is to provide the Go community with an active maintenance and de

etcd-io 5.7k Jun 30, 2022
BuntDB is an embeddable, in-memory key/value database for Go with custom indexing and geospatial support

BuntDB is a low-level, in-memory, key/value store in pure Go. It persists to disk, is ACID compliant, and uses locking for multiple readers and a sing

Josh Baker 3.8k Jun 24, 2022