Skip to content

Commit

Permalink
Update docs on error handling output schema.
Browse files Browse the repository at this point in the history
  • Loading branch information
robertwb committed Oct 18, 2024
1 parent ef0fc8b commit eafe08b
Showing 1 changed file with 4 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ The `output` parameter is a name that must referenced as an input to
another transform that will process the errors (e.g. by writing them out).
For example,
the following code will write all "good" processed records to one file and
any "bad" records to a separate file.
any "bad" records, along with metadata about what error was encountered,
to a separate file.

```
pipeline:
Expand Down Expand Up @@ -77,6 +78,8 @@ for a robust pipeline).
Note also that the exact format of the error outputs is still being finalized.
They can be safely printed and written to outputs, but their precise schema
may change in a future version of Beam and should not yet be depended on.
Currently it has, at the very least, an `element` field which holds the element
that caused the error.

Some transforms allow for extra arguments in their error_handling config, e.g.
for Python functions one can give a `threshold` which limits the relative number
Expand Down

0 comments on commit eafe08b

Please sign in to comment.