over — traverse nested values as a lateral query
over <expr> [, <expr>...]
over <expr> [, <expr>...] [with <var>=<expr> [, ... <var>[=<expr>]] => ( <lateral> )
The over operator traverses complex values to create a new sequence
of derived values (e.g., the elements of an array) and either
(in the first form) sends the new values directly to its output or
(in the second form) sends the values to a scoped computation as indicated
by <lateral>, which may represent any SuperPipe subquery operating on the
derived sequence of values as this.
Each expression <expr> is evaluated in left-to-right order and derived sequences are
generated from each such result depending on its types:
- an array value generates each of its elements,
- a map value generates a sequence of records of the form
{key:<key>,value:<value>}for each entry in the map, and - all other values generate a single value equal to itself.
Records can be converted to maps with the flatten function
resulting in a map that can be traversed,
e.g., if this is a record, it can be traversed with over flatten(this).
The nested subquery depicted as <lateral> is called a lateral subquery.
Over evaluates each expression and emits it
echo null | super -z -c 'over 1,2,"foo"' -
=>
1
2
"foo"
The over clause is evaluated once per each input value
echo "null null" | super -z -c 'over 1,2' -
=>
1
2
1
2
Array elements are enumerated
echo null | super -z -c 'over [1,2],[3,4,5]' -
=>
1
2
3
4
5
Over traversing an array
echo '{a:[1,2,3]}' | super -z -c 'over a' -
=>
1
2
3
Filter the traversed values
echo '{a:[6,5,4]} {a:[3,2,1]}' | super -z -c 'over a |> this % 2 == 0' -
=>
6
4
2
Aggregate the traversed values
echo '{a:[1,2]} {a:[3,4,5]}' | super -z -c 'over a |> sum(this)' -
=>
15
Aggregate the traversed values in a lateral query
echo '{a:[1,2]} {a:[3,4,5]}' | super -z -c 'over a => ( sum(this) )' -
=>
3
12
Access the outer values in a lateral query
echo '{a:[1,2],s:"foo"} {a:[3,4,5],s:"bar"}' |
super -z -c 'over a with s => (sum(this) |> yield {s,sum:this})' -
=>
{s:"foo",sum:3}
{s:"bar",sum:12}
Traverse a record by flattening it
echo '{s:"foo",r:{a:1,b:2}} {s:"bar",r:{a:3,b:4}} ' |
super -z -c 'over flatten(r) with s => (yield {s,key:key[0],value})' -
=>
{s:"foo",key:"a",value:1}
{s:"foo",key:"b",value:2}
{s:"bar",key:"a",value:3}
{s:"bar",key:"b",value:4}