Skip to content

Commit

Permalink
Bring back the "|" symbol and eliminate search rakes (#5436)
Browse files Browse the repository at this point in the history
This commit brings back the "|" symbol for pipeline separator
while retaining the "|>" as an option.  The latter is needed to
exit a SQL expression back into a pipe when a SQL expression
is immediately followed by a shortcut that begins with an identifier
(implied-yield expression or agg function).

For any valid query without shortcuts, there is never an ambiguity
between the bitwise OR "|" and pipe separator because keywords cannot
be used as identifiers.  The argument that such ambiguity is
present made in the Google SQL pipes paper presumably assumes a
one-token lookahead parser, which is not the case here.

Toward this end, we changed our grammar to disallow keywords as
identifier and added the backtick-string syntax to escape any
string as identifier.

Furthermore, we arranged for the expression grammar to include
"|" for bitwise OR in SQL expression but omit it in any
non-SQL pipe operator.  This means any valid SQL queries will
continue to be valid SuperSQL queries (inclusive of bitwise OR).
Moreover, expressions in pipeline operators do not have any
pipe-character ambiguity with shortcuts because bitwise-OR is omitted.
We can add bitwise functionality as named functions in place of the
archaic C-style syntax (which can work in both SQL and pipeline
operators).  Best practice will be to use these functions over
the old bitwise syntax.

We also removed search syntax as a shortcut and now require a "?"
(or "search" keyword) to signal the keyword search operator.
This means typos that happen to compile into unintended searches
are no longer accepted by the grammar thereby eliminating the so-called
search rake.

* disable legacy bitwise-OR syntax in SQL expressions

Ok as disccused, I'm pushing these changes to remove bitwise-OR
from the SQL expression syntax, thus removing the ambiguity with the
SQL shortcuts.  If/when users want to run SuperSQL queries on legacy
SQL with bitwise OR, we can put this grammar back in place but under
a flag to enable it only as needed.  In the meantime, wew ill introduce
bitwise_or(), bitwise_and(), etc named functions in subsequent PR.
  • Loading branch information
mccanne authored Nov 5, 2024
1 parent 3da0f83 commit 5a0232a
Show file tree
Hide file tree
Showing 182 changed files with 4,500 additions and 4,395 deletions.
4 changes: 2 additions & 2 deletions cmd/super/ztests/ast.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
script: |
super -C -c a
super -C -c "?a"
super -C -I query.zed
super -C -I query.zed -c a
inputs:
- name: query.zed
data: |
q
? q
outputs:
- name: stdout
Expand Down
2 changes: 1 addition & 1 deletion cmd/super/ztests/diropt1.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
script: super -f zeek -split out -c "*" -
script: super -f zeek -split out -

inputs:
- name: stdin
Expand Down
2 changes: 1 addition & 1 deletion cmd/super/ztests/diropt2.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
script: |
super -z -split out -o foo -c "*" -
super -z -split out -o foo -
inputs:
- name: stdin
Expand Down
2 changes: 1 addition & 1 deletion cmd/super/ztests/stop-on-error-1.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
script: |
! super -c "*" good.jsup bad.jsup
! super good.jsup bad.jsup
inputs:
- name: good.jsup
Expand Down
2 changes: 1 addition & 1 deletion cmd/super/ztests/stop-on-error-2.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
script: |
super -z -e=false -c "*" good.jsup bad.jsup
super -z -e=false good.jsup bad.jsup
inputs:
- name: good.jsup
Expand Down
2 changes: 1 addition & 1 deletion cmd/super/ztests/stop-on-error-3.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Second input has bad middle line (detection succeeds).
script: |
! super -z -e=false -c "*" good.jsup bad.jsup
! super -z -e=false good.jsup bad.jsup
inputs:
- name: good.jsup
Expand Down
Loading

0 comments on commit 5a0232a

Please sign in to comment.