Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Doc fetch from cursor to be non-blocking and blocking #23

Merged
merged 2 commits into from
Nov 11, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 18 additions & 12 deletions delivery/subscription.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -83,33 +83,39 @@ If you specify `FULL` instead of the `since_clause`, the subscription cursor sta
FETCH from cursor function is currently only supported in the PSQL simple query mode. If you are using components like JDBC that default to the extended query mode, please manually set the mode to simple query mode.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extended mode is also supported in risingwavelabs/risingwave#17821 starting from v2.0


</Info>
#### FETCH NEXT FROM cursor
#### Non-blocking data fetch

After creating a subscription cursor, you can fetch the data by the `FETCH NEXT FROM cursor_name` command. Then you will see a result like below:
```sql
FETCH NEXT/n FROM cursor_name;
```

Fetch the next row or up to N rows from the cursor. If fewer than N rows are available, it will return whatever is available immediately without waiting. This also means that if there are no rows available (i.e., the latest data has been reached), an empty result will be returned immediately.

```sql
FETCH NEXT FROM cur;

----RESULT
t1.v1 | t1.v2 | t1.v3 | t1.op | rw_timestamp
t1.v1 | t1.v2 | t1.v3 | t1.op | rw_timestamp
-------+-------+-------+--------------+---------------
1 | 1 | 1 | UpdateDelete | 1715669376304
1 | 1 | 1 | UpdateDelete | 1715669376304
(1 row)
```

The `op` column in the result stands for the change operations. It has four options: `insert`, `update_insert`, `delete`, and `update_delete`. For a single UPDATE statement, the subscription log will contain two separate rows: one with `update_insert` and another with `update_delete`. This is because RisingWave treats an UPDATE as a delete of the old value followed by an insert of the new value. As for `rw_timestamp`, it corresponds to the Unix timestamp in milliseconds when the data was written.
In the example above, the `op` column in the result indicates the type of change operations. There are four options: `insert`, `update_insert`, `delete`, and `update_delete`. For a single UPDATE statement, the subscription log will contain two separate rows: one with `update_insert` and another with `update_delete`. This is because RisingWave treats an UPDATE as a delete of the old value followed by an insert of the new value. As for `rw_timestamp`, it corresponds to the Unix timestamp in milliseconds when the data was written.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The four options are in camelCase instead of snake_case: Insert, Delete, UpdateInsert, UpdateDelete.


#### Blocking data fetch

Note that each time `FETCH NEXT FROM cursor_name` is called, it will return one row of incremental data from the subscribed table. It does not return all the incremental data at once, but requires the user to repeatedly call this statement to fetch the data.
```sql
FETCH NEXT/n FROM cursor_name WITH (timeout = 'xx');
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's give a concrete example for timeout: replace timeout = 'xx' to timeout = '1s'

```

This method is non-blocking. Even if the current table has no new incremental data, `FETCH NEXT FROM cursor_name` will not block, but will return an empty row. When new incremental data is generated, calling this statement again will return the latest row of data.
Fetch up to N rows from the cursor with a specified timeout. The `timeout` value should be a string in the interval format. In this case, the fetch statement will return when either N rows have been fetched or the timeout occurs. If the timeout occurs, whatever has been read up to that point will be returned. Here are two scenarios to trigger the timeout:

#### FETCH n FROM cursor
1. The cursor has reached the latest data and has been waiting too long for new data to arrive.

You also can fetch multiple rows at once from the cursor using the `FETCH n FROM cursor_name` command. `n` is the number of rows to fetch.
2. At least N rows are available for the cursor to read, but retrieving all of them takes an extended period.

```sql
FETCH n FROM cursor_name;
```
To avoid polling for new data frequently with the non-blocking `FETCH`, you can set a longer timeout to simulate a scenario where you want the `FETCH` to block until new data arrives.

#### Order of the fetched data

Expand Down