Skip to content

Commit

Permalink
fix: log dependencies and quote sql
Browse files Browse the repository at this point in the history
  • Loading branch information
vagetablechicken committed Apr 24, 2024
1 parent a1dcb7a commit bd20c1c
Show file tree
Hide file tree
Showing 5 changed files with 58 additions and 9 deletions.
16 changes: 15 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ addtable t1 a int, b int64

# validate in online request mode
valreq select count(*) over w1 from t1 window w1 as (partition by a order by b rows between unbounded preceding and current row);

# When there are single quotes in SQL (currently double quotes are not supported, only single quotes can be used), please enclose them in double quotes to avoid parsing missing quotes.
valreq "select * from t1 join t2 on t2.b='abc'"
```
If the test fails, it will print the SQL compilation error. If it passes, it will print `validate * success`. The entire process takes place in a virtual environment, without concerns about resource usage after table creation or any side effects. Any SQL that passes validation through `valreq` will definitely be deployable in a real cluster.

Expand Down Expand Up @@ -181,7 +184,7 @@ Note that if a table already exists, the creation of a new table will replace th

If you want to create tables without redundant indexes, you can use `genddl` to generate ddl from the query SQL.

Note that method `genDDL` in `openmldb-jdbc` does not support multiple databases yet, so we can't use this method to parse SQLs that have multiple dbs.
Note that the `genDDL` method in `openmldb-jdbc` does not yet support multiple databases. Therefore, we cannot use this method to parse SQL statements that involve multiple databases. When there are single quotes in the SQL (double quotes are not supported at the moment, only single quotes), please enclose the SQL in double quotes to avoid missing quotes during parsing. For example, use `genddl "select * from t1 join t2 on t2.c1 == '123';"`.

- Example1
```
Expand Down Expand Up @@ -265,3 +268,14 @@ t1;
### CLI Framework

We use `cliche` for the CLI interface. See [Manual](https://code.google.com/archive/p/cliche/wikis/Manual.wiki) and [source](https://github.com/budhash/cliche).

Due to the simplicity of the framework, when quotation marks are required in the SQL, only single quotation marks can be used, and the entire SQL must be enclosed in double quotation marks. Otherwise, the parsing will discard the double quotation marks in the SQL.

### Debugging

If you encounter any issues during usage, you can specify `log4j.properties` using `-Dlog4j.configuration=`. By default, the log level is set to WARN, but you can change it to print INFO and above logs to obtain more runtime information.

To test in the project:
```bash
java -jar target/emulator-1.1-SNAPSHOT.jar -Dlog4j.configuration=./src/test/resources/log4j.properties
```
20 changes: 17 additions & 3 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,14 @@ addtable t1 a int, b int64
你可以使用`val``valreq`分别进行在线批模式和在线请求模式(即服务部署上线)的OpenMLDB SQL验证。例如,我们测试一个SQL是否能被`DEPLOY`上线,使用`valreq`命令:

```sql
# table creations - t/addtable: create table
# table 创建 - t/addtable: create table
addtable t1 a int, b int64
addtable t2 a int, b string

# validate in online request mode
# 在线请求模式下验证SQL
valreq select count(*) over w1 from t1 window w1 as (partition by a order by b rows between unbounded preceding and current row);
# SQL中存在单引号时(目前不支持双引号,只能使用单引号),请用双引号括起来,避免解析遗漏引号
valreq "select * from t1 join t2 on t2.b='abc'"
```

如果测试不通过,将打印SQL编译错误;通过则打印`validate * success`。整个过程在虚拟环境中,无需担心建表后的资源占用,也没有任何副作用。只要`valreq`验证通过的 SQL,则一定能在真实集群中上线。
Expand Down Expand Up @@ -172,7 +175,7 @@ Emulator使用`openmldb-jdbc`进行验证,目前支持的OpenMLDB版本为:

#### `genddl <sql>`

可以帮助用户根据SQL直接生成最佳索引的建表语句,避免冗余索引(目前仅支持单数据库)。
可以帮助用户根据SQL直接生成最佳索引的建表语句,避免冗余索引(目前仅支持单数据库)。SQL中存在单引号时(目前不支持双引号,只能使用单引号),请用双引号括起来,避免解析遗漏引号,例如,`genddl "select * from t1 join t2 on t2.c1 == '123';"`

- 范例1
```
Expand Down Expand Up @@ -255,3 +258,14 @@ t1;
### CLI框架

我们使用`cliche`作为CLI框架,详见[操作手册](https://code.google.com/archive/p/cliche/wikis/Manual.wiki)[source](https://github.com/budhash/cliche)

由于框架简单,SQL中必须存在引号时,只能使用单引号,并将整个SQL用双引号括起来。否则,解析会丢掉SQL中的双引号。

### 调试

使用过程中如果出现问题,可以通过`-Dlog4j.configuration=`指定`log4j.properties`,默认是WARN日志,可以改为打印INFO及以上的日志,能获取更多运行信息。

在项目中测试:
```bash
java -jar target/emulator-1.1-SNAPSHOT.jar -Dlog4j.configuration=./src/test/resources/log4j.properties
```
15 changes: 15 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,21 @@
<artifactId>snakeyaml</artifactId>
<version>2.2</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.22</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>2.0.0-alpha1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.21</version>
</dependency>
</dependencies>

<build>
Expand Down
1 change: 1 addition & 0 deletions src/main/java/com/openmldb/emulator/App.java
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,7 @@ private boolean toydbSupport() {
public void genddl(@Param(name = "sql", description = "deployment sql") String... sqlParts)
throws SQLException {
String sql = Joiner.on(" ").join(sqlParts);
log.info("gen ddl: {}", sql);
List<String> ddl = SqlClusterExecutor.genDDL(sql, schemaMap);
System.out.println(Joiner.on("\n").join(ddl));
}
Expand Down
15 changes: 10 additions & 5 deletions src/test/java/com/openmldb/emulator/AppTest.java
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
package com.openmldb.emulator;

import static org.junit.Assert.assertThat;
import static org.junit.Assert.assertTrue;

import java.io.BufferedReader;
Expand All @@ -12,10 +11,6 @@

import com.google.common.base.Charsets;
import com.google.common.io.CharSource;
import com.google.common.io.CharStreams;
import com.google.common.io.Resources;
import com.openmldb.emulator.App;

import asg.cliche.CLIException;
import asg.cliche.Shell;
import asg.cliche.ShellFactory;
Expand Down Expand Up @@ -79,4 +74,14 @@ public void testYamlCases() throws SQLException, IOException {
App app = new App();
app.run(com.google.common.io.Resources.getResource("case.yaml").getFile());
}

@Test
public void testGenddl() throws SQLException {
App app = new App();
app.addtable("t1", "c1 int, c2 string, c3 string, c4 string");
app.addtable("t2", "c1 int, c2 string, c3 string, c4 string");
app.genddl("select * from t1 last join t2 t22 on t1.c1=t22.c1 and t1.c2=t22.c2 and t1.c3=t22.c3 and t22.c4='abc'");
app.genddl("select * from t1 last join t2 t22 on t1.c1=t22.c1 and t1.c2=t22.c2 and t1.c3=t22.c3 and t22.c4=abc");

}
}

0 comments on commit bd20c1c

Please sign in to comment.