Skip to content

Commit 95c8b61

Browse files
SKaltSKalt
committed
chore: sync the wip-cst branch with the main branch
1 parent 5ba4a55 commit 95c8b61

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

62 files changed

+5698
-1163
lines changed

‎.github/dependabot.yml

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
version: 2
2+
updates:
3+
- package-ecosystem: cargo
4+
directory: "/"
5+
schedule:
6+
interval: daily
7+
open-pull-requests-limit: 10
8+
- package-ecosystem: cargo
9+
directory: "/sqlparser_bench"
10+
schedule:
11+
interval: daily
12+
open-pull-requests-limit: 10

‎.github/workflows/rust.yml

+70-22
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,78 @@
11
name: Rust
22

3-
on: [push]
3+
on: [push, pull_request]
44

55
jobs:
6-
build:
6+
7+
codestyle:
8+
runs-on: ubuntu-latest
9+
steps:
10+
- name: Set up Rust
11+
uses: hecrj/setup-rust-action@v1
12+
with:
13+
components: rustfmt
14+
# Note that `nightly` is required for `license_template_path`, as
15+
# it's an unstable feature.
16+
rust-version: nightly
17+
- uses: actions/checkout@v2
18+
- run: cargo +nightly fmt -- --check --config-path <(echo 'license_template_path = "HEADER"')
19+
20+
lint:
21+
runs-on: ubuntu-latest
22+
steps:
23+
- name: Set up Rust
24+
uses: hecrj/setup-rust-action@v1
25+
with:
26+
components: clippy
27+
- uses: actions/checkout@v2
28+
- run: cargo clippy --all-targets --all-features -- -D warnings
29+
30+
compile:
731
runs-on: ubuntu-latest
32+
steps:
33+
- name: Set up Rust
34+
uses: hecrj/setup-rust-action@v1
35+
- uses: actions/checkout@master
36+
- run: cargo check --all-targets --all-features
837

38+
test:
39+
strategy:
40+
matrix:
41+
rust: [stable, beta, nightly]
42+
runs-on: ubuntu-latest
943
steps:
10-
- uses: actions/checkout@v1
1144
- name: Setup Rust
12-
run: |
13-
rustup toolchain install nightly --profile default
14-
rustup toolchain install stable
15-
rustup override set stable
16-
# Clippy must be run first, as its lints are only triggered during
17-
# compilation. Put another way: after a successful `cargo build`, `cargo
18-
# clippy` is guaranteed to produce no results. This bug is known upstream:
19-
# https://github.com/rust-lang/rust-clippy/issues/2604.
20-
# - name: Clippy
21-
# run: cargo clippy -- --all-targets --all-features -- -D warnings
22-
- name: Check formatting
23-
run: |
24-
cargo +nightly fmt -- --check --config-path <(echo 'license_template_path = "HEADER"')
25-
- name: Build
26-
run: cargo build --verbose
27-
- name: Run tests
28-
run: cargo test --verbose
29-
- name: Run tests for all features
30-
run: cargo test --verbose -- all-features
45+
uses: hecrj/setup-rust-action@v1
46+
with:
47+
rust-version: ${{ matrix.rust }}
48+
- name: Install Tarpaulin
49+
uses: actions-rs/[email protected]
50+
with:
51+
crate: cargo-tarpaulin
52+
version: 0.14.2
53+
use-tool-cache: true
54+
- name: Checkout
55+
uses: actions/checkout@v2
56+
- name: Test
57+
run: cargo test --all-features
58+
- name: Coverage
59+
if: matrix.rust == 'stable'
60+
run: cargo tarpaulin -o Lcov --output-dir ./coverage
61+
- name: Coveralls
62+
if: matrix.rust == 'stable'
63+
uses: coverallsapp/github-action@master
64+
with:
65+
github-token: ${{ secrets.GITHUB_TOKEN }}
66+
67+
publish-crate:
68+
if: startsWith(github.ref, 'refs/tags/v0')
69+
runs-on: ubuntu-latest
70+
needs: [test]
71+
steps:
72+
- name: Set up Rust
73+
uses: hecrj/setup-rust-action@v1
74+
- uses: actions/checkout@v2
75+
- name: Publish
76+
shell: bash
77+
run: |
78+
cargo publish --token ${{ secrets.CRATES_TOKEN }}

‎.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Generated by Cargo
22
# will have compiled files and executables
33
/target/
4+
/sqlparser_bench/target/
45

56
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
67
# More information here http://doc.crates.io/guide.html#cargotoml-vs-cargolock

‎CHANGELOG.md

+72-1
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,74 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
Given that the parser produces a typed AST, any changes to the AST will technically be breaking and thus will result in a `0.(N+1)` version. We document changes that break via addition as "Added".
77

88
## [Unreleased]
9-
Check https://github.com/andygrove/sqlparser-rs/commits/master for undocumented changes.
9+
Check https://github.com/ballista-compute/sqlparser-rs/commits/main for undocumented changes.
10+
11+
12+
## [0.8.0] 2020-02-20
13+
14+
### Added
15+
* Introduce Hive QL dialect `HiveDialect` and syntax (#235) - Thanks @hntd187!
16+
* Add `SUBSTRING(col [FROM <expr>] [FOR <expr>])` syntax (#293)
17+
* Support parsing floats without leading digits `.01` (#294)
18+
* Support parsing multiple show variables (#290) - Thanks @francis-du!
19+
* Support SQLite `INSERT OR [..]` syntax (#281) - Thanks @zhangli-pear!
20+
21+
## [0.7.0] 2020-12-28
1022

1123
### Changed
24+
- Change the MySQL dialect to support `` `identifiers` `` quoted with backticks instead of the standard `"double-quoted"` identifiers (#247) - thanks @mashuai!
25+
- Update bigdecimal requirement from 0.1 to 0.2 (#268)
26+
27+
### Added
28+
- Enable dialect-specific behaviours in the parser (`dialect_of!()`) (#254) - thanks @eyalleshem!
29+
- Support named arguments in function invocations (`ARG_NAME => val`) (#250) - thanks @eyalleshem!
30+
- Support `TABLE()` functions in `FROM` (#253) - thanks @eyalleshem!
31+
- Support Snowflake's single-line comments starting with '#' or '//' (#264) - thanks @eyalleshem!
32+
- Support PostgreSQL `PREPARE`, `EXECUTE`, and `DEALLOCATE` (#243) - thanks @silathdiir!
33+
- Support PostgreSQL math operators (#267) - thanks @alex-dukhno!
34+
- Add SQLite dialect (#248) - thanks @mashuai!
35+
- Add Snowflake dialect (#259) - thanks @eyalleshem!
36+
- Support for Recursive CTEs - thanks @rhanqtl!
37+
- Support `FROM (table_name) alias` syntax - thanks @eyalleshem!
38+
- Support for `EXPLAIN [ANALYZE] VERBOSE` - thanks @ovr!
39+
- Support `ANALYZE TABLE`
40+
- DDL:
41+
- Support `OR REPLACE` in `CREATE VIEW`/`TABLE` (#239) - thanks @Dandandan!
42+
- Support specifying `ASC`/`DESC` in index columns (#249) - thanks @mashuai!
43+
- Support SQLite `AUTOINCREMENT` and MySQL `AUTO_INCREMENT` column option in `CREATE TABLE` (#234) - thanks @mashuai!
44+
- Support PostgreSQL `IF NOT EXISTS` for `CREATE SCHEMA` (#276) - thanks @alex-dukhno!
45+
46+
### Fixed
47+
- Fix a typo in `JSONFILE` serialization, introduced in 0.3.1 (#237)
48+
- Change `CREATE INDEX` serialization to not end with a semicolon, introduced in 0.5.1 (#245)
49+
- Don't fail parsing `ALTER TABLE ADD COLUMN` ending with a semicolon, introduced in 0.5.1 (#246) - thanks @mashuai
50+
51+
## [0.6.1] - 2020-07-20
52+
53+
### Added
54+
- Support BigQuery `ASSERT` statement (#226)
55+
56+
## [0.6.0] - 2020-07-20
57+
58+
### Added
59+
- Support SQLite's `CREATE TABLE (...) WITHOUT ROWID` (#208) - thanks @mashuai!
60+
- Support SQLite's `CREATE VIRTUAL TABLE` (#209) - thanks @mashuai!
61+
62+
## [0.5.1] - 2020-06-26
63+
This release should have been called `0.6`, as it introduces multiple incompatible changes to the API. If you don't want to upgrade yet, you can revert to the previous version by changing your `Cargo.toml` to:
64+
65+
sqlparser = "= 0.5.0"
66+
67+
68+
### Changed
69+
- **`Parser::parse_sql` now accepts a `&str` instead of `String` (#182)** - thanks @Dandandan!
1270
- Change `Ident` (previously a simple `String`) to store the parsed (unquoted) `value` of the identifier and the `quote_style` separately (#143) - thanks @apparebit!
1371
- Support Snowflake's `FROM (table_name)` (#155) - thanks @eyalleshem!
72+
- Add line and column number to TokenizerError (#194) - thanks @Dandandan!
73+
- Use Token::EOF instead of Option<Token> (#195)
74+
- Make the units keyword following `INTERVAL '...'` optional (#184) - thanks @maxcountryman!
75+
- Generalize `DATE`/`TIME`/`TIMESTAMP` literals representation in the AST (`TypedString { data_type, value }`) and allow `DATE` and other keywords to be used as identifiers when not followed by a string (#187) - thanks @maxcountryman!
76+
- Output DataType capitalized (`fmt::Display`) (#202) - thanks @Dandandan!
1477

1578
### Added
1679
- Support MSSQL `TOP (<N>) [ PERCENT ] [ WITH TIES ]` (#150) - thanks @alexkyllo!
@@ -21,9 +84,16 @@ Check https://github.com/andygrove/sqlparser-rs/commits/master for undocumented
2184
- Support basic forms of `CREATE SCHEMA` and `DROP SCHEMA` (#173) - thanks @alex-dukhno!
2285
- Support `NULLS FIRST`/`LAST` in `ORDER BY` expressions (#176) - thanks @houqp!
2386
- Support `LISTAGG()` (#174) - thanks @maxcountryman!
87+
- Support the string concatentation operator `||` (#178) - thanks @Dandandan!
88+
- Support bitwise AND (`&`), OR (`|`), XOR (`^`) (#181) - thanks @Dandandan!
89+
- Add serde support to AST structs and enums (#196) - thanks @panarch!
90+
- Support `ALTER TABLE ADD COLUMN`, `RENAME COLUMN`, and `RENAME TO` (#203) - thanks @mashuai!
91+
- Support `ALTER TABLE DROP COLUMN` (#148) - thanks @ivanceras!
92+
- Support `CREATE TABLE ... AS ...` (#206) - thanks @Dandandan!
2493

2594
### Fixed
2695
- Report an error for unterminated string literals (#165)
96+
- Make file format (`STORED AS`) case insensitive (#200) and don't allow quoting it (#201) - thanks @Dandandan!
2797

2898
## [0.5.0] - 2019-10-10
2999

@@ -138,3 +208,4 @@ We don't have a changelog for the changes made in 2018, but thanks to @crw5996,
138208

139209
## [0.1.0] - 2018-09-03
140210
Initial release
211+

‎Cargo.toml

+19-9
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
[package]
22
name = "sqlparser"
33
description = "Extensible SQL Lexer and Parser with support for ANSI SQL:2011"
4-
version = "0.5.1-alpha-0"
4+
version = "0.8.1-alpha.0"
55
authors = ["Andy Grove <[email protected]>"]
6-
homepage = "https://github.com/andygrove/sqlparser-rs"
6+
homepage = "https://github.com/ballista-compute/sqlparser-rs"
77
documentation = "https://docs.rs/sqlparser/"
88
keywords = [ "ansi", "sql", "lexer", "parser" ]
9-
repository = "https://github.com/andygrove/sqlparser-rs"
9+
repository = "https://github.com/ballista-compute/sqlparser-rs"
1010
license = "Apache-2.0"
1111
include = [
1212
"src/**/*.rs",
@@ -19,15 +19,25 @@ name = "sqlparser"
1919
path = "src/lib.rs"
2020

2121
[features]
22-
cst = ["rowan"] # Retain a concrete synatax tree, available as `parser.syntax()`
22+
cst = ["rowan"] # Retain a concrete synatax tree, available as `parser.syntax()`
23+
json_example = ["serde_json", "serde"] # Enable JSON output in the `cli` example:
2324

2425
[dependencies]
25-
bigdecimal = { version = "0.1.0", optional = true, features = ["serde"] }
26-
log = "0.4.5"
26+
bigdecimal = { version = "0.2", features = ["serde"], optional = true }
27+
log = "0.4"
2728
rowan = { version = "0.10.0", optional = true, features = ["serde1"] }
28-
serde = { version = "1.0.106", features = ["derive"] }
29-
serde_json = "1.0.52"
29+
serde = { version = "1.0", features = ["derive"], optional = true }
30+
# serde_json is only used in examples/cli, but we have to put it outside
31+
# of dev-dependencies because of
32+
# https://github.com/rust-lang/cargo/issues/1596
33+
serde_json = { version = "1.0", optional = true }
3034

3135
[dev-dependencies]
32-
simple_logger = "1.0.1"
36+
simple_logger = "1.9"
3337
matches = "0.1"
38+
39+
[package.metadata.release]
40+
# Instruct `cargo release` to not run `cargo publish` locally:
41+
# https://github.com/sunng87/cargo-release/blob/master/docs/reference.md#config-fields
42+
# See docs/releasing.md for details.
43+
disable-publish = true

‎README.md

+12-5
Original file line numberDiff line numberDiff line change
@@ -185,17 +185,17 @@ Remarks
185185

186186
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
187187
[![Version](https://img.shields.io/crates/v/sqlparser.svg)](https://crates.io/crates/sqlparser)
188-
[![Build Status](https://travis-ci.org/andygrove/sqlparser-rs.svg?branch=master)](https://travis-ci.org/andygrove/sqlparser-rs)
189-
[![Coverage Status](https://coveralls.io/repos/github/andygrove/sqlparser-rs/badge.svg?branch=master)](https://coveralls.io/github/andygrove/sqlparser-rs?branch=master)
188+
[![Build Status](https://github.com/ballista-compute/sqlparser-rs/workflows/Rust/badge.svg?branch=main)](https://github.com/ballista-compute/sqlparser-rs/actions?query=workflow%3ARust+branch%3Amain)
189+
[![Coverage Status](https://coveralls.io/repos/github/ballista-compute/sqlparser-rs/badge.svg?branch=main)](https://coveralls.io/github/ballista-compute/sqlparser-rs?branch=main)
190190
[![Gitter Chat](https://badges.gitter.im/sqlparser-rs/community.svg)](https://gitter.im/sqlparser-rs/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
191191

192192
The goal of this project is to build a SQL lexer and parser capable of parsing
193193
SQL that conforms with the [ANSI/ISO SQL standard][sql-standard] while also
194194
making it easy to support custom dialects so that this crate can be used as a
195195
foundation for vendor-specific parsers.
196196

197-
This parser is currently being used by the [DataFusion] query engine and
198-
[LocustDB].
197+
This parser is currently being used by the [DataFusion] query engine,
198+
[LocustDB], and [Ballista].
199199

200200
## Example
201201

@@ -223,6 +223,12 @@ This outputs
223223
AST: [Query(Query { ctes: [], body: Select(Select { distinct: false, projection: [UnnamedExpr(Identifier("a")), UnnamedExpr(Identifier("b")), UnnamedExpr(Value(Long(123))), UnnamedExpr(Function(Function { name: ObjectName(["myfunc"]), args: [Identifier("b")], over: None, distinct: false }))], from: [TableWithJoins { relation: Table { name: ObjectName(["table_1"]), alias: None, args: [], with_hints: [] }, joins: [] }], selection: Some(BinaryOp { left: BinaryOp { left: Identifier("a"), op: Gt, right: Identifier("b") }, op: And, right: BinaryOp { left: Identifier("b"), op: Lt, right: Value(Long(100)) } }), group_by: [], having: None }), order_by: [OrderByExpr { expr: Identifier("a"), asc: Some(false) }, OrderByExpr { expr: Identifier("b"), asc: None }], limit: None, offset: None, fetch: None })]
224224
```
225225

226+
## Command line
227+
To parse a file and dump the results as JSON:
228+
```
229+
$ cargo run --features json_example --example cli FILENAME.sql [--dialectname]
230+
```
231+
226232
## SQL compliance
227233

228234
SQL was first standardized in 1987, and revisions of the standard have been
@@ -300,9 +306,10 @@ resources.
300306

301307
[tdop-tutorial]: https://eli.thegreenplace.net/2010/01/02/top-down-operator-precedence-parsing
302308
[`cargo fmt`]: https://github.com/rust-lang/rustfmt#on-the-stable-toolchain
303-
[current issues]: https://github.com/andygrove/sqlparser-rs/issues
309+
[current issues]: https://github.com/ballista-compute/sqlparser-rs/issues
304310
[DataFusion]: https://github.com/apache/arrow/tree/master/rust/datafusion
305311
[LocustDB]: https://github.com/cswinter/LocustDB
312+
[Ballista]: https://github.com/ballista-compute/ballista
306313
[Pratt Parser]: https://tdop.github.io/
307314
[sql-2016-grammar]: https://jakewheat.github.io/sql-overview/sql-2016-foundation-grammar.html
308315
[sql-standard]: https://en.wikipedia.org/wiki/ISO/IEC_9075

‎docs/benchmarking.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Benchmarking
2+
3+
Run `cargo bench` in the project `sqlparser_bench` execute the queries.
4+
It will report results using the `criterion` library to perform the benchmarking.
5+
6+
The bench project lives in another crate, to avoid the negative impact on building the `sqlparser` crate.

‎docs/releasing.md

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Releasing
2+
3+
## Prerequisites
4+
Publishing to crates.io has been automated via GitHub Actions, so you will only
5+
need push access to the [ballista-compute GitHub repository](https://github.com/ballista-compute/sqlparser-rs)
6+
in order to publish a release.
7+
8+
We use the [`cargo release`](https://github.com/sunng87/cargo-release)
9+
subcommand to ensure correct versioning. Install via:
10+
11+
```
12+
$ cargo install cargo-release
13+
```
14+
15+
## Process
16+
17+
1. **Before releasing** ensure `CHANGELOG.md` is updated appropriately and that
18+
you have a clean checkout of the `main` branch of the sqlparser repository:
19+
```
20+
$ git fetch && git status
21+
On branch main
22+
Your branch is up to date with 'upstream/main'.
23+
24+
nothing to commit, working tree clean
25+
```
26+
* If you have the time, check that the examples in the README are up to date.
27+
28+
2. Using `cargo-release` we can publish a new release like so:
29+
30+
```
31+
$ cargo release minor --push-remote upstream
32+
```
33+
34+
You can add `--dry-run` to see what the command is going to do,
35+
or `--skip-push` to stop before actually publishing the release.
36+
37+
`cargo release` will then:
38+
39+
* Bump the minor part of the version in `Cargo.toml` (e.g. `0.7.1-alpha.0`
40+
-> `0.8.0`. You can use `patch` instead of `minor`, as appropriate).
41+
* Create a new tag (e.g. `v0.8.0`) locally
42+
* Push the new tag to the specified remote (`upstream` in the above
43+
example), which will trigger a publishing process to crates.io as part of
44+
the [corresponding GitHub Action](https://github.com/ballista-compute/sqlparser-rs/blob/main/.github/workflows/rust.yml).
45+
46+
Note that credentials for authoring in this way are securely stored in
47+
the (GitHub) repo secrets as `CRATE_TOKEN`.
48+
* Bump the crate version again (to something like `0.8.1-alpha.0`) to
49+
indicate the start of new development cycle.
50+
51+
3. Push the updates to the `main` branch upstream:
52+
```
53+
$ git push upstream
54+
```
55+
56+
4. Check that the new version of the crate is available on crates.io:
57+
https://crates.io/crates/sqlparser
58+

0 commit comments

Comments
 (0)