diff --git a/docs/img/demo.gif b/docs/img/demo.gif new file mode 100644 index 00000000..dda4aff8 Binary files /dev/null and b/docs/img/demo.gif differ diff --git a/docs/index.html b/docs/index.html index de66af34..6b494947 100644 --- a/docs/index.html +++ b/docs/index.html @@ -16,9 +16,9 @@
/user/index/surname("Johnson",<userID:int>)
+ /user/index/surname("Johnson",<userID:int>)
/user(:userID,...)
- /user(9323,"Timothy","Johnson",37)=nil
+ /user(9323,"Timothy","Johnson",37)=nil
/user(24335,"Andrew","Johnson",42)=nil
/user(33423,"Ryan","Johnson",0x0ffa83,42.2)=nil
FQL is an open
@@ -54,11 +54,8 @@ FQL
Using FQL
- - Command Line
-
- - Headless
- - Fullscreen
-
+ - Command
+ Line
- API (Layer)
Roadmap
@@ -68,12 +65,12 @@ Overview
href="https://github.com/janderland/fql/blob/main/syntax.ebnf">context-free
grammar. The queries look like key-values encoded using the
directory & tuple layers.
- /my/directory("my","tuple")=4000
+ /my/directory("my","tuple")=4000
FQL queries may define a single key-value to be written, as shown
above, or may define a set of key-values to be read, as shown
below.
- /my/directory("my","tuple")=<int>
- /my/directory("my","tuple")=4000
+ /my/directory("my","tuple")=<int>
+ /my/directory("my","tuple")=4000
The query above has a variable <int>
as its
value. Variables act as placeholders for any of the supported data elements. In this case, the variable
@@ -82,33 +79,33 @@
Overview
FQL queries can also perform range reads & filtering by
including a variable in the key’s tuple. The query below will return
all key-values which conform to the schema defined by the query.
- /my/directory(<>,"tuple")=nil
- /my/directory("your","tuple")=nil
+ /my/directory(<>,"tuple")=nil
+ /my/directory("your","tuple")=nil
/my/directory(42,"tuple")=nil
All key-values with a certain key prefix can be range read by
ending the key’s tuple with ...
.
- /my/directory("my","tuple",...)=<>
- /my/directory("my","tuple")=0x0fa0
+ /my/directory("my","tuple",...)=<>
+ /my/directory("my","tuple")=0x0fa0
/my/directory("my","tuple",47.3)=0x8f3a
/my/directory("my","tuple",false,0xff9a853c12)=nil
A query’s value may be omitted to imply a variable, meaning the
following query is semantically identical to the one above.
- /my/directory("my","tuple",...)
- /my/directory("my","tuple")=0x0fa0
+ /my/directory("my","tuple",...)
+ /my/directory("my","tuple")=0x0fa0
/my/directory("my","tuple",47.3)=0x8f3a
/my/directory("my","tuple",false,0xff9a853c12)=nil
Including a variable in the directory tells FQL to perform the read
on all directory paths matching the schema.
- /<>/directory("my","tuple")
- /my/directory("my","tuple")=0x0fa0
+ /<>/directory("my","tuple")
+ /my/directory("my","tuple")=0x0fa0
/your/directory("my","tuple")=nil
Key-values can be cleared by using the special clear
token as the value.
- /my/directory("my","tuple")=clear
+ /my/directory("my","tuple")=clear
The directory layer can be queried by only including a directory
path.
- /my/<>
- /my/directory
+ /my/<>
+ /my/directory
Data Elements
An FQL query contains instances of data elements. These are the
same types of elements found in the Data Elements
bint
support is not yet implemented.
Tuples & values may contain any of the data elements.
- /region/north_america(22.3,-8)=("rain","fog")
+ /region/north_america(22.3,-8)=("rain","fog")
/region/east_asia("japan",nil)=0xff
Strings are the only data element allowed in directories. If a
directory string only contains alphanumericals, underscores, dashes,
and periods then the quotes don’t need to be included.
- /quoteless-string_in.dir(true)=false
+ /quoteless-string_in.dir(true)=false
/"other ch@r@cters must be quoted!"(20)=32.3
Quoted strings may contain quotes via backslash escapes.
- /my/dir("I said \"hello\"")=nil
+ /my/dir("I said \"hello\"")=nil
Value Encoding
The directory and tuple layers are responsible for encoding the
data elements in the key. As for the value, FDB doesn’t provide a
@@ -245,33 +242,33 @@
Variables & Schemas
href="#data-elements">data element
may be represented with a
variable. Variables are specified as a list of element types,
separated by |
, wrapped in angled braces.
- <uint|str|uuid|bytes>
+ <uint|str|uuid|bytes>
The variable’s type list describes which data elements are allowed
at the variable’s position. A variable may be empty, including no
element types, meaning it represents all element types.
- /user(<int>,<str>,<>)=<>
- /user(0,"jon",0xffab0c)=nil
+ /user(<int>,<str>,<>)=<>
+ /user(0,"jon",0xffab0c)=nil
/user(20,"roger",22.3)=0xff
/user(21,"",nil)="nothing"
Before the type list, a variable can be given a name. This name is
used to reference the variable in subsequent queries, allowing for index indirection.
- /index("cars",<varName:int>)
+ /index("cars",<varName:int>)
/data(:varName,...)
- /user(33,"mazda")=nil
+ /user(33,"mazda")=nil
/user(320,"ford")=nil
/user(411,"chevy")=nil
Space & Comments
Whitespace and newlines are allowed within a tuple, between its
elements.
- /account/private(
+ /account/private(
<uint>,
<uint>,
<str>,
)=<int>
Comments start with a %
and continue until the end of
the line. They can be used to describe a tuple’s elements.
- % private account balances
+ % private account balances
/account/private(
<uint>, % user ID
<uint>, % group ID
@@ -293,7 +290,7 @@ Mutations
Mutation queries with a data element
as their value perform a write operation.
- /my/dir("hello","world")=42
+ /my/dir("hello","world")=42
db.Transact(func(tr fdb.Transaction) (interface{}, error) {
dir, err := directory.CreateOrOpen(tr, []string{"my", "dir"}, nil)
if err != nil {
@@ -309,7 +306,7 @@ Mutations
})
Mutation queries with the clear
token as their value
perform a clear operation.
- /my/dir("hello","world")=clear
+ /my/dir("hello","world")=clear
db.Transact(func(tr fdb.Transaction) (interface{}, error) {
dir, err := directory.Open(tr, []string{"my", "dir"}, nil)
if err != nil {
@@ -334,7 +331,7 @@ Reads
href="#variables">variable as the value, and are therefore read
queries.
- /my/dir(99.8,7dfb10d1-2493-4fb5-928e-889fdc6a7136)=<int|str>
+ /my/dir(99.8,7dfb10d1-2493-4fb5-928e-889fdc6a7136)=<int|str>
db.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {
dir, err := directory.Open(tr, []string{"my", "dir"}, nil)
if err != nil {
@@ -365,7 +362,7 @@ Reads
the key-value does not match the schema.
If the value is specified as an empty variable, then the raw bytes
are returned.
- /some/data(10139)=<>
+ /some/data(10139)=<>
db.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {
dir, err := directory.Open(tr, []string{"some", "data"}, nil)
if err != nil {
@@ -381,7 +378,7 @@ Reads
Queries with variables or the
...
token in their key (and optionally in their value)
result in a range of key-values being read.
- /people("coders",...)
+ /people("coders",...)
db.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {
dir, err := directory.Open(tr, []string{"people"}, nil)
if err != nil {
@@ -415,7 +412,7 @@ Directories
directory as a query. These queries can only perform reads. If the
directory path contains no variables, the query will read that single
directory.
- /root/<>/items
+ /root/<>/items
root, err := directory.Open(tr, []string{"root"}, nil)
if err != nil {
if errors.Is(err, directory.ErrDirNotExists) {
@@ -452,11 +449,11 @@ Filtering
Because filtering is performed on the client side, range reads may
stream a lot of data to the client while the client filters most of it
away. For example, consider the following query:
- /people(3392,<str|int>,<>)=(<uint>,...)
+ /people(3392,<str|int>,<>)=(<uint>,...)
In the key, the location of the first variable or ...
token determines the range read prefix used by FQL. For this
particular query, the prefix would be as follows:
- /people(3392)
+ /people(3392)
Foundation DB will stream all key-values with this prefix to the
client. As they are received, the client will filter out key-values
which don’t match the query’s schema. Below you can see a Go
@@ -539,25 +536,25 @@
Indirection
Suppose we have a large list of people, one key-value for each
person.
- /people(<id:uint>,<firstName:str>,<lastName:str>,<age:int>)=nil
+ /people(<id:uint>,<firstName:str>,<lastName:str>,<age:int>)=nil
If we wanted to read all records with the last name of “Johnson”,
we’d have to perform a linear search across the entire “people”
directory. To make this kind of search more efficient, we can store an
index of last names in a separate directory.
- /index/last_name(<lastName:str>,<id:uint>)=nil
+ /index/last_name(<lastName:str>,<id:uint>)=nil
FQL can forward the observed values of named variables from one
query to the next, allowing us to efficiently query for all people
with the last name of “Johnson”.
- /index/last_name("Johnson",<id:uint>)
+ /index/last_name("Johnson",<id:uint>)
/people(:id,...)
- /people(23,"Lenny","Johnson",22,"Mechanic")=nil
+ /people(23,"Lenny","Johnson",22,"Mechanic")=nil
/people(348,"Roger","Johnson",54,"Engineer")=nil
/people(2003,"Larry","Johnson",8,"N/A")=nil
The first query returned 3 key-values containing the IDs of 23,
348, & 2003 which were then fed into the second query resulting in
3 individual single reads.
- /index/last_name("Johnson",<id:uint>)
- /index/last_name("Johnson",23)=nil
+ /index/last_name("Johnson",<id:uint>)
+ /index/last_name("Johnson",23)=nil
/index/last_name("Johnson",348)=nil
/index/last_name("Johnson",2003)=nil
Aggregation
@@ -570,11 +567,11 @@ Aggregation
href="https://apple.github.io/foundationdb/blob.html">storing large
blobs, the data is usually split into 10 kB chunks stored in the
value. The respective key contain the byte offset of the chunk.
- /blob(
+ /blob(
"my file", % The identifier of the blob.
<offset:int>, % The byte offset within the blob.
)=<chunk:bytes> % A chunk of the blob.
- /blob("my file",0)=10e3_bytes
+ /blob("my file",0)=10e3_bytes
/blob("my file",10000)=10e3_bytes
/blob("my file",20000)=2.7e3_bytes
@@ -587,25 +584,26 @@ Aggregation
themselves. This can be done using aggregation queries.
FQL provides a pseudo data type named agg
which
performs the aggregation.
- /blob("my file",...)=<blob:agg>
- /blob("my file",...)=22.7e3_bytes
+ /blob("my file",...)=<blob:agg>
+ /blob("my file",...)=22.7e3_bytes
Aggregation queries always result in a single key-value. With
non-aggregation queries, variables & the ...
token
are resolved as actual data elements in the query results. For
aggregation queries, only aggregation variables are resolved.
A similar pseudo data type for summing integers could be provided
as well.
- /deltas("group A",<int>)
- /deltas("group A",20)=nil
+ /deltas("group A",<int>)
+ /deltas("group A",20)=nil
/deltas("group A",-18)=nil
/deltas("group A",3)=nil
- /deltas("group A",<sum>)
- /deltas("group A",5)=<>
+ /deltas("group A",<sum>)
+ /deltas("group A",5)=<>
Using FQL
FQL can be used for exploring a Foundation DB cluster in a CLI
environment or programmatically as a Foundation DB layer.
Command Line
+
Headless
FQL provides a CLI for performing queries from the command line. To
execute a query in “headless” mode (without fullscreen), you can use
@@ -627,7 +625,7 @@
Fullscreen
Foundation DB is maintained for the lifetime of the application.
Single queries may be executed in their own transactions and the
results are displayed in a scrollable list.
-
+
Currently, this environment is not very useful, but it lays the
groundwork for a fully-featured FQL frontend (accidental
alliteration). The final version of this environment will include the
@@ -640,6 +638,7 @@
Fullscreen
Customizable formatting of key-values
Restoring a session after restart
+
API (Layer)
TODO: Review this section.
When integrating SQL into other languages, there are usually two
@@ -726,7 +725,16 @@
Roadmap
diff --git a/docs/index.md b/docs/index.md
index 891670e4..af2fb9b9 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -4,11 +4,11 @@ title: FQL
# We include this intro via the 'include-before'
# metadata field so it's placed before the TOC.
include-before: |
- ```lang-fql {.query}
+ ```language-fql {.query}
/user/index/surname("Johnson",)
/user(:userID,...)
```
- ```lang-fql {.result}
+ ```language-fql {.result}
/user(9323,"Timothy","Johnson",37)=nil
/user(24335,"Andrew","Johnson",42)=nil
/user(33423,"Ryan","Johnson",0x0ffa83,42.2)=nil
@@ -29,7 +29,7 @@ grammar](https://github.com/janderland/fql/blob/main/syntax.ebnf).
The queries look like key-values encoded using the directory
& tuple layers.
-```lang-fql {.query}
+```language-fql {.query}
/my/directory("my","tuple")=4000
```
@@ -37,11 +37,11 @@ FQL queries may define a single key-value to be written, as
shown above, or may define a set of key-values to be read,
as shown below.
-```lang-fql {.query}
+```language-fql {.query}
/my/directory("my","tuple")=
```
-```lang-fql {.result}
+```language-fql {.result}
/my/directory("my","tuple")=4000
```
@@ -57,11 +57,11 @@ including a variable in the key's tuple. The query below
will return all key-values which conform to the schema
defined by the query.
-```lang-fql {.query}
+```language-fql {.query}
/my/directory(<>,"tuple")=nil
```
-```lang-fql {.result}
+```language-fql {.result}
/my/directory("your","tuple")=nil
/my/directory(42,"tuple")=nil
```
@@ -69,11 +69,11 @@ defined by the query.
All key-values with a certain key prefix can be range read
by ending the key's tuple with `...`.
-```lang-fql {.query}
+```language-fql {.query}
/my/directory("my","tuple",...)=<>
```
-```lang-fql {.result}
+```language-fql {.result}
/my/directory("my","tuple")=0x0fa0
/my/directory("my","tuple",47.3)=0x8f3a
/my/directory("my","tuple",false,0xff9a853c12)=nil
@@ -83,11 +83,11 @@ A query's value may be omitted to imply a variable, meaning
the following query is semantically identical to the one
above.
-```lang-fql {.query}
+```language-fql {.query}
/my/directory("my","tuple",...)
```
-```lang-fql {.result}
+```language-fql {.result}
/my/directory("my","tuple")=0x0fa0
/my/directory("my","tuple",47.3)=0x8f3a
/my/directory("my","tuple",false,0xff9a853c12)=nil
@@ -96,11 +96,11 @@ above.
Including a variable in the directory tells FQL to perform
the read on all directory paths matching the schema.
-```lang-fql {.query}
+```language-fql {.query}
/<>/directory("my","tuple")
```
-```lang-fql {.result}
+```language-fql {.result}
/my/directory("my","tuple")=0x0fa0
/your/directory("my","tuple")=nil
```
@@ -108,18 +108,18 @@ the read on all directory paths matching the schema.
Key-values can be cleared by using the special `clear` token
as the value.
-```lang-fql {.query}
+```language-fql {.query}
/my/directory("my","tuple")=clear
```
The directory layer can be queried by only including
a directory path.
-```lang-fql {.query}
+```language-fql {.query}
/my/<>
```
-```lang-fql {.result}
+```language-fql {.result}
/my/directory
```
@@ -151,7 +151,7 @@ Descriptions of these elements can be seen below.
Tuples & values may contain any of the data elements.
-```lang-fql {.query}
+```language-fql {.query}
/region/north_america(22.3,-8)=("rain","fog")
/region/east_asia("japan",nil)=0xff
```
@@ -161,14 +161,14 @@ a directory string only contains alphanumericals,
underscores, dashes, and periods then the quotes don't need
to be included.
-```lang-fql {.query}
+```language-fql {.query}
/quoteless-string_in.dir(true)=false
/"other ch@r@cters must be quoted!"(20)=32.3
```
Quoted strings may contain quotes via backslash escapes.
-```lang-fql {.query}
+```language-fql {.query}
/my/dir("I said \"hello\"")=nil
```
@@ -205,7 +205,7 @@ element](#data-elements) may be represented with a variable.
Variables are specified as a list of element types,
separated by `|`, wrapped in angled braces.
-```lang-fql
+```language-fql
```
@@ -214,11 +214,11 @@ allowed at the variable's position. A variable may be empty,
including no element types, meaning it represents all
element types.
-```lang-fql {.query}
+```language-fql {.query}
/user(,,<>)=<>
```
-```lang-fql {.result}
+```language-fql {.result}
/user(0,"jon",0xffab0c)=nil
/user(20,"roger",22.3)=0xff
/user(21,"",nil)="nothing"
@@ -229,12 +229,12 @@ name is used to reference the variable in subsequent
queries, allowing for [index
indirection](#index-indirection).
-```lang-fql {.query}
+```language-fql {.query}
/index("cars",)
/data(:varName,...)
```
-```lang-fql {.result}
+```language-fql {.result}
/user(33,"mazda")=nil
/user(320,"ford")=nil
/user(411,"chevy")=nil
@@ -245,7 +245,7 @@ indirection](#index-indirection).
Whitespace and newlines are allowed within a tuple, between
its elements.
-```lang-fql {.query}
+```language-fql {.query}
/account/private(
,
,
@@ -256,7 +256,7 @@ its elements.
Comments start with a `%` and continue until the end of the
line. They can be used to describe a tuple's elements.
-```lang-fql
+```language-fql
% private account balances
/account/private(
, % user ID
@@ -285,7 +285,7 @@ clearing a key-value.
Mutation queries with a [data element](#data-elements) as
their value perform a write operation.
-```lang-fql {.query}
+```language-fql {.query}
/my/dir("hello","world")=42
```
@@ -308,7 +308,7 @@ db.Transact(func(tr fdb.Transaction) (interface{}, error) {
Mutation queries with the `clear` token as their value
perform a clear operation.
-```lang-fql {.query}
+```language-fql {.query}
/my/dir("hello","world")=clear
```
@@ -341,7 +341,7 @@ the schema exists.
> [variable](#variables) as the value, and are therefore
> read queries.
-```lang-fql {.query}
+```language-fql {.query}
/my/dir(99.8,7dfb10d1-2493-4fb5-928e-889fdc6a7136)=
```
@@ -380,7 +380,7 @@ cannot be decoded, the key-value does not match the schema.
If the value is specified as an empty variable, then the raw
bytes are returned.
-```lang-fql {.query}
+```language-fql {.query}
/some/data(10139)=<>
```
@@ -403,7 +403,7 @@ Queries with [variables](#variables) or the `...` token in
their key (and optionally in their value) result in a range
of key-values being read.
-```lang-fql {.query}
+```language-fql {.query}
/people("coders",...)
```
@@ -445,7 +445,7 @@ a lone directory as a query. These queries can only perform
reads. If the directory path contains no variables, the
query will read that single directory.
-```lang-fql {.query}
+```language-fql {.query}
/root/<>/items
```
@@ -495,7 +495,7 @@ reads may stream a lot of data to the client while the
client filters most of it away. For example, consider the
following query:
-```lang-fql {.query}
+```language-fql {.query}
/people(3392,,<>)=(,...)
```
@@ -503,7 +503,7 @@ In the key, the location of the first variable or `...`
token determines the range read prefix used by FQL. For this
particular query, the prefix would be as follows:
-```lang-fql {.query}
+```language-fql {.query}
/people(3392)
```
@@ -600,7 +600,7 @@ also called "indirection".
Suppose we have a large list of people, one key-value for
each person.
-```lang-fql {.query}
+```language-fql {.query}
/people(,,,)=nil
```
@@ -610,7 +610,7 @@ entire "people" directory. To make this kind of search more
efficient, we can store an index of last names in a separate
directory.
-```lang-fql {.query}
+```language-fql {.query}
/index/last_name(,)=nil
```
@@ -618,11 +618,11 @@ FQL can forward the observed values of named variables from
one query to the next, allowing us to efficiently query for
all people with the last name of "Johnson".
-```lang-fql {.query}
+```language-fql {.query}
/index/last_name("Johnson",)
/people(:id,...)
```
-```lang-fql {.result}
+```language-fql {.result}
/people(23,"Lenny","Johnson",22,"Mechanic")=nil
/people(348,"Roger","Johnson",54,"Engineer")=nil
/people(2003,"Larry","Johnson",8,"N/A")=nil
@@ -632,10 +632,10 @@ The first query returned 3 key-values containing the IDs of
23, 348, & 2003 which were then fed into the second query
resulting in 3 individual [single reads](#single-reads).
-```lang-fql {.query}
+```language-fql {.query}
/index/last_name("Johnson",)
```
-```lang-fql {.result}
+```language-fql {.result}
/index/last_name("Johnson",23)=nil
/index/last_name("Johnson",348)=nil
/index/last_name("Johnson",2003)=nil
@@ -654,14 +654,14 @@ blobs](https://apple.github.io/foundationdb/blob.html), the
data is usually split into 10 kB chunks stored in the value.
The respective key contain the byte offset of the chunk.
-```lang-fql {.query}
+```language-fql {.query}
/blob(
"my file", % The identifier of the blob.
, % The byte offset within the blob.
)= % A chunk of the blob.
```
-```lang-fql {.result}
+```language-fql {.result}
/blob("my file",0)=10e3_bytes
/blob("my file",10000)=10e3_bytes
/blob("my file",20000)=2.7e3_bytes
@@ -679,11 +679,11 @@ queries.
FQL provides a pseudo data type named `agg` which performs
the aggregation.
-```lang-fql {.query}
+```language-fql {.query}
/blob("my file",...)=
```
-```lang-fql {.result}
+```language-fql {.result}
/blob("my file",...)=22.7e3_bytes
```
@@ -696,21 +696,21 @@ resolved.
A similar pseudo data type for summing integers could be
provided as well.
-```lang-fql {.query}
+```language-fql {.query}
/deltas("group A",)
```
-```lang-fql {.result}
+```language-fql {.result}
/deltas("group A",20)=nil
/deltas("group A",-18)=nil
/deltas("group A",3)=nil
```
-```lang-fql {.query}
+```language-fql {.query}
/deltas("group A",)
```
-```lang-fql {.result}
+```language-fql {.result}
/deltas("group A",5)=<>
```
@@ -722,6 +722,8 @@ a CLI environment or programmatically as a Foundation DB
## Command Line
+
+
### Headless
FQL provides a CLI for performing queries from the command
@@ -756,7 +758,7 @@ application. Single queries may be executed in their own
transactions and the results are displayed in a scrollable
list.
-![](../vhs/demo.gif)
+![](img/demo.gif)
Currently, this environment is not very useful, but it lays
the groundwork for a fully-featured FQL frontend (accidental
@@ -770,6 +772,8 @@ include the following features:
- Customizable formatting of key-values
- Restoring a session after restart
+
+
## API (Layer)
TODO: Review this section.
diff --git a/docs/index.tmpl b/docs/index.tmpl
index e9ef3150..416944c5 100644
--- a/docs/index.tmpl
+++ b/docs/index.tmpl
@@ -24,7 +24,16 @@
diff --git a/docs/js/fql.js b/docs/js/fql.js
index f6b2ad73..f6df45ad 100644
--- a/docs/js/fql.js
+++ b/docs/js/fql.js
@@ -195,7 +195,7 @@
NUMBER,
{ // Highlight lone bar for inline text.
scope: 'variable',
- begin: /|/,
+ begin: /\|/,
},
],
}));