SQL Optimizer And Rewriter

Overview

SOAR

Gitter License Go Report Card Build Status GoDoc

文档 | FAQ | 变更记录 | 路线图 | English

SOAR

SOAR(SQL Optimizer And Rewriter) 是一个对 SQL 进行优化和改写的自动化工具。 由小米人工智能与云平台的数据库团队开发与维护。

功能特点

  • 跨平台支持(支持 Linux, Mac 环境,Windows 环境理论上也支持,不过未全面测试)
  • 目前只支持 MySQL 语法族协议的 SQL 优化
  • 支持基于启发式算法的语句优化
  • 支持复杂查询的多列索引优化(UPDATE, INSERT, DELETE, SELECT)
  • 支持 EXPLAIN 信息丰富解读
  • 支持 SQL 指纹、压缩和美化
  • 支持同一张表多条 ALTER 请求合并
  • 支持自定义规则的 SQL 改写

快速入门

交流与反馈

  • 欢迎通过 Github Issues 提交问题报告与建议
  • QQ 群:779359816(未满) 758940447(已满)
  • Gitter 推荐

xiaomi_sa

License

Apache License 2.0.

Issues
  • heuristic rules that ref blingbling

    heuristic rules that ref blingbling

    Feature Description

    Reference blingbling, SOAR should add these heuristic rules.

    Use Case(s)

    --rule-allow-delete-has-join            是否允许DELETE语句中使用JOIN. 默认: false
    --rule-allow-delete-has-sub-clause      是否允许DELETE语句中使用子查询. 默认: false
    --rule-allow-delete-many-table          是否允许同时删除多个表数据. 默认: false
    --rule-allow-full-text                  是否允许使用全文索引. 默认: false
    --rule-allow-update-has-join            是否允许UPDATE语句中使用JOIN. 默认: false
    --rule-allow-update-has-sub-clause      是否允许UPDATE语句中使用子查询. 默认: false
    --rule-collate string                   通用允许的collate, 默认(多个用逗号隔开) (default "utf8_general_ci,utf8mb4_general_ci")
    --rule-insert-rows int                  每批允许 insert 的行数 (default 1000)
    --rule-text-type-column-count int       允许使用text/blob字段个数. 如果在rule-not-allow-column-type相关text字段.该参数将不其作用 (default 2)
    --rule-not-allow-column-type string     不允许的字段类型, 至此的类型: decimal, tinyint, smallint, int, float, double, timestamp, bigint, mediumint, date, time, datetime, year, newdate, varchar, bit, json, newdecimal, enum, set, tinyblob, mediumblob, longblob, blob, tinytext, mediumtext, longtext, text, geometry (default "tinytext,mediumtext,logtext,tinyblob,mediumblob,longblob")
    

    Thanks to @daiguadaidai

    feature request 
    opened by martianzhang 7
  • -test-dsn配置

    -test-dsn配置

    请问配置文件中关于test-dsn的配置,端口号是可以自己随意指定吗,另外schema、user、password和online-dsn中的对应参数需要一样吗

    test-dsn: addr: "" schema: information_schema user: "" password: '********' charset: utf8mb4 disable: true

    help wanted 
    opened by Linkyang953 6
  • Sampling Error,You  have an error in your SQL syntax;

    Sampling Error,You have an error in your SQL syntax;

    • 获取优化报告时,数据采样提示语法错误,错误信息:
    2018/10/29 15:24:59.806 [D] [mysql.go:76] Execute SQL with DSN(mysql:3306/optimizer_ix4i4k7wMDiYHo1G) : CREATE TABLE `sql_workflow` (
      `id` int(11) NOT NULL AUTO_INCREMENT,
      `workflow_name` varchar(50) NOT NULL ,
      `engineer` varchar(50) NOT NULL,
      `review_man` varchar(50) NOT NULL,
      `create_time` datetime(6) NOT NULL,
      `finish_time` datetime(6) DEFAULT NULL,
      `status` varchar(50) NOT NULL,
      `is_backup` varchar(20) NOT NULL,
      `review_content` longtext NOT NULL,
      `cluster_name` varchar(50) NOT NULL,
      `reviewok_time` datetime(6) DEFAULT NULL,
      `sql_content` longtext NOT NULL,
      `execute_result` longtext NOT NULL,
      `is_manual` int(11) NOT NULL,
      `audit_remark` varchar(200) NOT NULL,
      PRIMARY KEY (`id`)
    ) ENGINE=InnoDB AUTO_INCREMENT=72 DEFAULT CHARSET=utf8
    2018/10/29 15:24:59.829 [D] [env.go:397] createTable, Start Sampling data from archer.sql_workflow to optimizer_ix4i4k7wMDiYHo1G.sql_workflow ...
    2018/10/29 15:24:59.830 [D] [mysql.go:76] Execute SQL with DSN(mysql:3306/archer) : show table status where name = 'sql_workflow'
    2018/10/29 15:24:59.832 [D] [mysql.go:76] Execute SQL with DSN(mysql:3306/archer) : show table status where name = 'sql_workflow'
    2018/10/29 15:24:59.833 [D] [sampling.go:99] SamplingData, tableRows: 66, wantRowsCount: 30000, factor: 454.545455
    2018/10/29 15:24:59.833 [D] [sampling.go:118] Sampling data execute: select DATA_TYPE from information_schema.COLUMNS where TABLE_SCHEMA='optimizer_ix4i4k
    7wMDiYHo1G' and TABLE_NAME = 'sql_workflow'
    2018/10/29 15:24:59.839 [E] [sampling.go:227] doSampling Error from optimizer_ix4i4k7wMDiYHo1G.sql_workflow: Received #1064 error from MySQL server: "You 
    have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '0_0_0'", "None", "0", "
    "], [2, "CHECKED", 0, "Audit completed", "None", "create ' at line 1"
    2018/10/29 15:24:59.839 [D] [sampling.go:215] 70 rows sampling out
    2018/10/29 15:24:59.840 [D] [index.go:84] Enter: NewAdvisor(), Caller: main.main
    
    • 语句和数据信息(表结构上面的错误日志有)
    select * from sql_workflow;
    
    INSERT INTO `sql_workflow` (`id`,`workflow_name`,`engineer`,`review_man`,`create_time`,`finish_time`,`status`,`is_backup`,`review_content`,`cluster_name`,`reviewok_time`,`sql_content`,`execute_result`,`is_manual`,`audit_remark`) VALUES ('1','创建用户信息表','archer','["auditor", ""]','2018-06-07 16:34:41','2018-06-07 16:35:28','已正常结束','是','[[1, "CHECKED", 0, "Audit completed", "None", "use archer_test", 0, "'0_0_0'", "None", "0", ""], [2, "CHECKED", 0, "Audit completed", "None", "create table users(\r\n  id bigint unsigned auto_increment comment 'id',\r\n  username varchar(20) not null default '' comment '\u7528\u6237\u540d',\r\n  nickname varchar(20) not null default '' comment '\u59d3\u540d',\r\n  phone varchar(20) not null default '' comment '\u624b\u673a\u53f7',\r\n  email varchar(50) not null default '' comment '\u90ae\u7bb1',\r\n  id_number varchar(18) not null default '' comment '\u8eab\u4efd\u8bc1\u53f7\u7801',\r\n  primary key(id)\r\n) engine=innodb,charset utf8mb4,comment '\u7528\u6237\u4fe1\u606f\u8868'", 0, "'0_0_1'", "mysql_3306_archer_test", "0", ""]]','archer','2018-06-07 16:35:28','use archer_test; create table users( id bigint unsigned auto_increment comment 'id', username varchar(20) not null default '' comment '用户名', nickname varchar(20) not null default '' comment '姓名', phone varchar(20) not null default '' comment '手机号', email varchar(50) not null default '' comment '邮箱', id_number varchar(18) not null default '' comment '身份证号码', primary key(id) ) engine=innodb,charset utf8mb4,comment '用户信息表';','[[1, "RERUN", 0, "Execute Successfully", "None", "use archer_test", 0, "'1528360528_136_0'", "None", "0.000", ""], [2, "EXECUTED", 0, "Execute Successfully", "None", "create table users(\r\n  id bigint unsigned auto_increment comment 'id',\r\n  username varchar(20) not null default '' comment '\u7528\u6237\u540d',\r\n  nickname varchar(20) not null default '' comment '\u59d3\u540d',\r\n  phone varchar(20) not null default '' comment '\u624b\u673a\u53f7',\r\n  email varchar(50) not null default '' comment '\u90ae\u7bb1',\r\n  id_number varchar(18) not null default '' comment '\u8eab\u4efd\u8bc1\u53f7\u7801',\r\n  primary key(id)\r\n) engine=innodb,charset utf8mb4,comment '\u7528\u6237\u4fe1\u606f\u8868'", 0, "'1528360528_136_1'", "mysql_3306_archer_test", "0.010", ""]]','0','');
    
    • 配置信息和版本信息
     ./soar -version
    Version: 2018-10-23 16:27:51 +0800 @7519019
    Branch: master
    Compile: 2018-10-23 18:26:17 +0800 by go version go1.10.4 linux/amd64
    GitDirty: 105
    
    ./soar -print-config
    allow-online-as-test: false
    drop-test-temporary: true
    only-syntax-check: false
    sampling-statistic-target: 100
    sampling: true
    profiling: false
    trace: false
    explain: true
    conn-time-out: 3
    query-time-out: 30
    delimiter: ;
    log-level: 7
    log-output: /opt/archery/downloads/log/soar.log
    report-type: html
    report-css: ""
    report-javascript: ""
    report-title: SQL优化分析报告
    markdown-extensions: 94
    markdown-html-flags: 0
    ignore-rules:
    - ""
    rewrite-rules:
    - delimiter
    - orderbynull
    - groupbyconst
    - dmlorderby
    - having
    - star2columns
    - insertcolumns
    - distinctstar
    blacklist: /opt/soar.blacklist
    max-join-table-count: 5
    max-group-by-cols-count: 5
    max-distinct-count: 5
    max-index-cols-count: 5
    max-total-rows: 9999999
    max-query-cost: 9999
    spaghetti-query-length: 2048
    allow-drop-index: false
    max-in-count: 10
    max-index-bytes-percolumn: 767
    max-index-bytes: 3072
    table-allow-charsets:
    - utf8
    - utf8mb4
    table-allow-engines:
    - innodb
    max-index-count: 10
    max-column-count: 40
    index-prefix: idx_
    unique-key-prefix: uk_
    max-subquery-depth: 5
    max-varchar-length: 1024
    explain-sql-report-type: pretty
    explain-type: extended
    explain-format: traditional
    explain-warn-select-type:
    - ""
    explain-warn-access-type:
    - ALL
    explain-max-keys: 3
    explain-min-keys: 0
    explain-max-rows: 10000
    explain-warn-extra:
    - ""
    explain-max-filtered: 100
    explain-warn-scalability:
    - O(n)
    show-warnings: false
    show-last-query-cost: false
    query: ""
    list-heuristic-rules: false
    list-rewrite-rules: false
    list-test-sqls: false
    list-report-types: false
    verbose: true
    dry-run: true
    max-pretty-sql-length: 1024
    
    bug 
    opened by hhyo 6
  • make deps wrong with gometalinter.v2 can intall

    make deps wrong with gometalinter.v2 can intall "gas": executable file

    OS: macOS 10.14 Mojave SOAP Version: https://github.com/XiaoMi/soar.git (Tag v0.8.0)

    已確認 echo ${PATH} 路徑指定正確

    檢查相依性時使用 retool do gometalinter.v2 intall 產生錯誤

    ==================================================

    錯誤訊息如下:

    wnlin:soar wn.lin$ make deps
    Dependency check
    mysql found
    docker found
    git found
    go found
    govendor found
    retool found
    # The retool tools.json is setup from retool-install.sh
    retool sync
    retool do gometalinter.v2 intall
    WARNING: exec: "gas": executable file not found in $PATH
    retool: fatal err: failed on 'gometalinter.v2 intall': exit status 2
    make: *** [deps] Error 1
    

    從 gometalinter.v2 的觀察也沒有 gas 能重編後安裝

    wnlin:gometalinter.v2 wn.lin$ pwd
    /Users/wn.lin/go/src/gopkg.in/alecthomas/gometalinter.v2
    
    wnlin:gometalinter.v2 wn.lin$ find . | grep gas | wc -l
           0
    

    Thanks.

    help wanted 
    opened by heavenruler 6
  • linux alpine镜像兼容性问题

    linux alpine镜像兼容性问题

    你好,我拉了仓库源码,执行了make release命令,用编译好的 ./release/soar.linux-amd64可执行文件在docker alpine镜像下运行会报错。

    以下是报错信息:

    bash-5.0# /webser/data/soar.linux-amd64
    bash: /webser/data/soar.linux-amd64: No such file or directory
    

    以下是我的系统信息:

    /webser/www # cat /proc/version
    Linux version 3.10.0-1062.12.1.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Tue Feb 4 23:02:59 UTC 2020
    /webser/www # cat /etc/issue
    Welcome to Alpine Linux 3.10
    Kernel \r on an \m (\l)
    /webser/www # uname -a
    Linux 875a68b80e0e 3.10.0-1062.12.1.el7.x86_64 #1 SMP Tue Feb 4 23:02:59 UTC 2020 x86_64 Linux
    

    以下是soar版本信息:

    Version: 2020-06-13 12:16:39 +0800 0da8710
    Branch: master
    Compile: 2020-06-17 19:24:44 +0800 by go version go1.14.2 linux/amd64
    GitDirty: 1
    0.11.0
    

    期待得到解决,谢谢!

    opened by liuyuanxiang 4
  • 对于权限不够的检查,会卡死

    对于权限不够的检查,会卡死

    2018/11/09 12:00:16.920 [W] [env.go:308] createDatabase, Error : Received #1044 error from MySQL server: "Access denied for user 'xxxx_all'@'%' to database 'optimizer_up2msgndl2tfxorl'" 2018/11/09 12:00:16.920 [E] [env.go:281] BuildVirtualEnv Error : Received #1044 error from MySQL server: "Access denied for user 'xxxx_all'@'%' to database 'optimizer_up2msgndl2tfxorl'"

    权限不够时,会导致卡住,没有任何输出,建议优化下

    feature request 
    opened by wangmin19861205 4
  • zsh: exec format error: ./soar 我不是很確定這是否是bug?

    zsh: exec format error: ./soar 我不是很確定這是否是bug?

    您好! 我在直接下載安裝,執行過程中,遇到以下問題. 但並不確定這是程序的問題或是我系統本身問題. 我在網上搜索,似乎很多軟件都遇到過同類情況.但沒有人給出本質原因和解決方案.

    OSX zsh 5.5 (x86_64-apple-darwin17.3.0) 我是用wget安裝 v0.8.1版本,然後按照文檔chmod 之後執行測試命令. 結果是zsh給我報了以下錯誤:

    → echo 'select * from film' | ./soar
    zsh: exec format error: ./soar
    

    嘗試直接運行,也會出現同樣的問題:

    → ./soar -m
    zsh: exec format error: ./soar
    

    如果這不是項目本身問題,也請回復一下我. 如果各位有人知道什麼原理,也請回復一下.

    help wanted 
    opened by wizarot 4
  • 添加QQ群没反应,gitter交流群没人回复

    添加QQ群没反应,gitter交流群没人回复

    Issues on GitHub are intended to be related to bugs or feature requests, so we recommend using our other community resources instead of asking here.

    • SOAR Doc
    • Any other questions can be asked in the community Gitter
    opened by edidada 3
  • RemoveSQLComments 字符串转义引号后跟着的#号会替换错误

    RemoveSQLComments 字符串转义引号后跟着的#号会替换错误

    https://github.com/XiaoMi/soar/blob/ee6cfd0c103c23961b9a68baf80a9976612f14a5/database/mysql.go#L260 RemoveSQLComments中如果SQL中的字符串带有#号,会被替换错误 如: insert into tbl (id,a) values(1,''#a'');

    not a bug 
    opened by chrobin 3
  • support dsn with ssh tunnel ?

    support dsn with ssh tunnel ?

    Issues on GitHub are intended to be related to bugs or feature requests, so we recommend using our other community resources instead of asking here.

    • SOAR Doc
    • Any other questions can be asked in the community Gitter
    no-plan 
    opened by somnus-L 3
  • Cant handle Create TEMPORAEY

    Cant handle Create TEMPORAEY

    Please answer these questions before submitting your issue. Thanks!

    1. What did you do? If possible, provide a recipe for reproducing the error.

    soar -query 1.sql -only-syntax-check At SQL 1 : line 1 column 16 near "CREATE TEMPORARY TABLE IF NOT EXISTS table2 AS (SELECT * FROM table1)" (total length 69)

    1. What did you expect to see? i want to handle create temporary correctly.

    2. What did you see instead?

    i get an error.

    1. What version of are you using (soar -version)?

    soar -version Version: 2019-01-21 16:54:09 +0800 0.11.0-16-gc12ae96 Branch: master Compile: 2019-02-18 16:14:56 +0800 by go version go1.11.5 darwin/amd64 GitDirty: 0

    dependency bug 
    opened by lolspider 3
  • Can NOT run `make tidb`

    Can NOT run `make tidb`

    Please answer these questions before submitting your issue. Thanks!

    1. What did you do? If possible, provide a recipe for reproducing the error. make tidb

    2. What did you expect to see? 可以成功执行。 install.md 中介绍,可以执行make tidb 更新依赖(make tidb 升级TiDB Parser依赖),但是实际并不可以?

    3. What did you see instead? make: *** No rule to make targettidb'. Stop.`

    4. What version of are you using (soar -version)?

    $ git lg -n 1
    * f720930 - (HEAD, origin/dev, origin/HEAD, dev) fix some test cases after pr merged (4 months ago) <zhangliang032>
    
    opened by shawn0915 0
  • 新增配置项用于控制是否允许测试数据库版本小于线上数据库

    新增配置项用于控制是否允许测试数据库版本小于线上数据库

    新增配置项用于控制是否允许测试数据库版本小于线上数据库

    Log: 新增配置项用于控制是否允许测试数据库版本小于线上数据库

    What problem does this PR solve?

    What is changed and how it works?

    Check List

    Tests

    • Unit test
    • Integration test
    • Manual test (add detailed scripts or steps below)
    • No code

    Code changes

    • Has exported function/method change
    • Has exported variable/fields change
    • Has interface methods change
    • Has persistent data change

    Side effects

    • Possible performance regression
    • Increased code complexity
    • Breaking backward compatibility
    opened by LiaoPuJian 0
  • [Bug]在macOs版本12.1和go版本1.17.2下, 无法运行

    [Bug]在macOs版本12.1和go版本1.17.2下, 无法运行

    16:32:59 in ~/GithubProject took 25s
    ➜ uname -a
                        'c.          [email protected]
                     ,xNMM.          ----------------------
                   .OMMMMo           OS: macOS 12.1 21C52 x86_64
                   OMMM0,            Host: MacBookPro11,4
         .;loddo:' loolloddol;.      Kernel: 21.2.0
       cKMMMMMMMMMMNWMMMMMMMMMM0:    Uptime: 3 days, 3 hours, 45 mins
     .KMMMMMMMMMMMMMMMMMMMMMMMWd.    Packages: 303 (brew)
     XMMMMMMMMMMMMMMMMMMMMMMMX.      Shell: zsh 5.8
    ;MMMMMMMMMMMMMMMMMMMMMMMM:       Resolution: [email protected]
    :MMMMMMMMMMMMMMMMMMMMMMMM:       DE: Aqua
    .MMMMMMMMMMMMMMMMMMMMMMMMX.      WM: Quartz Compositor
     kMMMMMMMMMMMMMMMMMMMMMMMMWd.    WM Theme: Blue (Light)
     .XMMMMMMMMMMMMMMMMMMMMMMMMMMk   Terminal: iTerm2
      .XMMMMMMMMMMMMMMMMMMMMMMMMK.   Terminal Font: MesloLGSForPowerline-Regular 13
        kMMMMMMMMMMMMMMMMMMMMMMd     CPU: Intel i7-4770HQ (8) @ 2.20GHz
         ;KMMMMMMMWXXWMMMMMMMk.      GPU: Intel Iris Pro
           .cooc,.    .,coo:.        Memory: 10677MiB / 16384MiB
    
    
    
    
    
    
    16:33:02 in ~/GithubProject
    ➜ wget -c https://ghproxy.fsou.cc/https://github.com/XiaoMi/soar/releases/download/0.11.0/soar.darwin-amd64 -O soar
    --2021-12-25 16:33:15--  https://ghproxy.fsou.cc/https://github.com/XiaoMi/soar/releases/download/0.11.0/soar.darwin-amd64
    正在解析主机 ghproxy.fsou.cc (ghproxy.fsou.cc)... 130.211.45.23
    正在连接 ghproxy.fsou.cc (ghproxy.fsou.cc)|130.211.45.23|:443... 已连接。
    已发出 HTTP 请求,正在等待回应... 200 OK
    长度:16447536 (16M) [application/octet-stream]
    正在保存至: “soar”
    
    soar                                        100%[==========================================================================================>]  15.69M  6.27MB/s  用时 2.5s
    
    2021-12-25 16:33:21 (6.27 MB/s) - 已保存 “soar” [16447536/16447536])
    
    
    16:33:21 in ~/GithubProject took 6s
    ➜ chmod a+x soar
    
    16:33:28 in ~/GithubProject
    ➜ ./soar -h
    fatal error: runtime: bsdthread_register error
    
    runtime stack:
    runtime.throw(0x175e7b5, 0x21)
    	/usr/local/go/src/runtime/panic.go:616 +0x81 fp=0x7ff7bfeff148 sp=0x7ff7bfeff128 pc=0x10290b1
    runtime.goenvs()
    	/usr/local/go/src/runtime/os_darwin.go:129 +0x83 fp=0x7ff7bfeff178 sp=0x7ff7bfeff148 pc=0x1026c33
    runtime.schedinit()
    	/usr/local/go/src/runtime/proc.go:501 +0xd6 fp=0x7ff7bfeff1e0 sp=0x7ff7bfeff178 pc=0x102b9a6
    runtime.rt0_go(0x7ff7bfeff218, 0x2, 0x7ff7bfeff218, 0x0, 0x1000000, 0x2, 0x7ff7bfeff430, 0x7ff7bfeff437, 0x0, 0x7ff7bfeff43a, ...)
    	/usr/local/go/src/runtime/asm_amd64.s:252 +0x1f4 fp=0x7ff7bfeff1e8 sp=0x7ff7bfeff1e0 pc=0x1051a64
    
    16:33:36 in ~/GithubProject
    ➜ go version
    go version go1.17.2 darwin/amd64
    
    opened by wilbur-yu 0
  • 文档及交流

    文档及交流

    soar即使在文档并不完美的情况下,获得7.4k的star,说明其表现不错。为你们开源点赞

    有如下建议希望官方支持:

    • 请不要将一个优秀的开源库的交流推荐到封闭的社交群里。社交群能做到@即达,但实时响应并不能保证,在GitHub进行讨论问题是比较好的方式,能做到广而告之。

    • 请麻烦完善文档及文档展示,即使只支持中文。再好的开源库,文档是第一步,对于使用者来说,甚至这就是最后一步。

    感谢聆听

    opened by BeatsKitano 0
Releases(0.11.0)
  • 0.11.0(Jan 21, 2019)

    New Features:

    • add mysql environment verbose info
    • add JSONFind function, which support JSON iterate
    • add new test database world_x
    • SplitStatement support optimizer hint /*+xxx */
    • include bats bash auto test framework
    • replace mysql database driver mymysql with go-sql-driver
    • add new -report-type [ast-json, tiast-json]
    • command line dsn args support '@', '/', ':' in password
    • add new heuristic rule RES.009, "SELECT * FROM tbl WHERE col = col = 'abc'"
    • add new heuristic rule RuleColumnNotAllowType COL.018
    • add string escape function for security

    Fixed BUGs:

    • fix #173 with JSONFind WHERE col = col = '' and col1 = 'xx'
    • fix #184 table status field datatype overflow
    • fix explain result with multi rows error
    • fix #178 JSON datatype only support utf8mb4
    • fix #122 single table select * don't auto-complete table name
    • fix #171 support socket access type
    • fix #58 sampling not deal with NULL able string
    • fix #172 compatible with mysql 5.1, which explain has no Index_Comment column
    • fix #163 column.Tp may be nil, which may raise panic
    • fix #151 bit type not config as int, when two columns compare will give ARG.003 suggestion.
    Source code(tar.gz)
    Source code(zip)
    soar.darwin-amd64(15.68 MB)
    soar.linux-amd64(14.02 MB)
    soar.windows-amd64(14.11 MB)
  • 0.10.0(Dec 5, 2018)

    2018-11 change log

    • add all third-party lib into vendor
    • support -report-type chardet
    • add more heuristic rules: TBL.008, KEY.010, ARG.012, KWR.004
    • add -cleanup-test-database command-line arg
    • add -check-config parameter
    • fix #146 pretty cause syntax error
    • fix #140 COL.012, COL.015 NULL type about TEXT/BLOB
    • fix #141 empty output when query execute failed on mysql
    • fix #89 index advisor give wrong database name, optimizer_xx
    • fix #121 RemoveSQLComment trim space
    • fix #120 trimspace before check single line comment
    • fix mac os stdout print buffer truncate
    • fix -config arg load file error
    • fix #116 SplitStatement check if single comment line is in multi-line sql.
    • fix #112 multi-line comment will cause line counter error, when -report-type=lint
    • fix #110 remove bom before auditing
    • fix #104 case insensitive regex @ CLA.009
    • fix #87 RuleImplicitConversion value type mistach check bug
    • fix #38 always true where condition check
    • abandon stdin terminal interactive mod, which may seems like hangup
    Source code(tar.gz)
    Source code(zip)
    soar.darwin-amd64(15.19 MB)
    soar.linux-amd64(13.56 MB)
    soar.windows-amd64(13.63 MB)
  • 0.9.0(Nov 10, 2018)

Owner
Xiaomi
Xiaomi
write APIs using direct SQL queries with no hassle, let's rethink about SQL

SQLer SQL-er is a tiny portable server enables you to write APIs using SQL query to be executed when anyone hits it, also it enables you to define val

Mohammed Al Ashaal 2k Aug 1, 2022
Go-sql-reader - Go utility to read the externalised sql with predefined tags

go-sql-reader go utility to read the externalised sql with predefined tags Usage

null 0 Jan 25, 2022
A tool to run queries in defined frequency and expose the count as prometheus metrics. Supports MongoDB and SQL

query2metric A tool to run db queries in defined frequency and expose the count as prometheus metrics. Why ? Product metrics play an important role in

S Santhosh Nagaraj 19 Jul 1, 2022
OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases and file formats using SQL.

OctoSQL OctoSQL is a query tool that allows you to join, analyse and transform data from multiple databases, streaming sources and file formats using

Jacob Martin 3.9k Aug 9, 2022
WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

WAL-G is an archival restoration tool for PostgreSQL, MySQL/MariaDB, and MS SQL Server (beta for MongoDB and Redis).

null 2.3k Aug 11, 2022
null 3 Mar 7, 2022
Prep finds all SQL statements in a Go package and instruments db connection with prepared statements

Prep Prep finds all SQL statements in a Go package and instruments db connection with prepared statements. It allows you to benefit from the prepared

Max Chechel 31 Jan 23, 2022
Additions to Go's database/sql for super fast performance and convenience.

gocraft/dbr (database records) gocraft/dbr provides additions to Go's database/sql for super fast performance and convenience. $ go get -u github.com/

null 1.7k Aug 11, 2022
A Go SQL query builder and struct mapper.

godb - a Go query builder and struct mapper godb is a simple Go query builder and struct mapper, not a full-featured ORM. godb does not manage relatio

Samuel GAY 716 Aug 8, 2022
CLI tool that can execute SQL queries on CSV, LTSV, JSON and TBLN. Can output to various formats.

trdsql CLI tool that can execute SQL queries on CSV, LTSV, JSON and TBLN. It is a tool like q, textql and others. The difference from these tools is t

Noboru Saito 1.2k Aug 2, 2022
sqlbench measures and compares the execution time of one or more SQL queries.

sqlbench sqlbench measures and compares the execution time of one or more SQL queries. The main use case is benchmarking simple CPU-bound query varian

Felix Geisendörfer 351 Jul 8, 2022
Use SQL to query host, DNS and exploit information using Shodan. Open source CLI. No DB required.

Shodan Plugin for Steampipe Query Shodan with SQL Use SQL to query host, DNS and exploit information using Shodan. For example: select * from shod

Turbot 23 May 10, 2022
Query and Provision Cloud Infrastructure using an extensible SQL based grammar

Deploy, Manage and Query Cloud Infrastructure using SQL [Documentation] [Developer Guide] Cloud infrastructure coding using SQL InfraQL allows you to

InfraQL 22 Apr 5, 2022
Scan database/sql rows directly to structs, slices, and primitive types

Scan Scan standard lib database rows directly to structs or slices. For the most comprehensive and up-to-date docs see the godoc Examples Multiple Row

Brett Jones 296 Aug 11, 2022
A flexible and powerful SQL string builder library plus a zero-config ORM.

SQL builder for Go Install Usage Basic usage Pre-defined SQL builders Build SQL for MySQL, PostgreSQL or SQLite Using Struct as a light weight ORM Nes

Huan Du 817 Aug 9, 2022
Use SQL to instantly query file, domain, URL and IP scanning results from VirusTotal.

VirusTotal Plugin for Steampipe Use SQL to query file, domain, URL and IP scanning results from VirusTotal. Get started → Documentation: Table definit

Turbot 23 Jun 22, 2022
Use SQL to instantly query users, groups, applications and more from Okta. Open source CLI. No DB required.

Okta Plugin for Steampipe Use SQL to query infrastructure including users, groups, applications and more from Okta. Get started → Documentation: Table

Turbot 4 Dec 15, 2021
simply SQL Parser for Go ( powered by vitess and TiDB )

vitess-sqlparser Simply SQL and DDL parser for Go (powered by vitess and TiDB ) this library inspired by https://github.com/xwb1989/sqlparser (origina

BlasTrain Co., Ltd. 289 Jul 30, 2022
Use SQL to instantly query instances, networks, databases, and more from Scaleway. Open source CLI. No DB required.

Scaleway Plugin for Steampipe Use SQL to query infrastructure servers, networks, databases and more from your Scaleway project. Get started → Document

Turbot 6 Jul 16, 2022