Skip to content

Commit

Permalink
[yaml] various inline provider doc fixes (apache#33296)
Browse files Browse the repository at this point in the history
* [yaml] various inline provider doc fixes

Signed-off-by: Jeffrey Kinard <[email protected]>

* Update yaml_combine.py

---------

Signed-off-by: Jeffrey Kinard <[email protected]>
Co-authored-by: Robert Bradshaw <[email protected]>
  • Loading branch information
Polber and robertwb authored Dec 6, 2024
1 parent 89ae2c1 commit 0bf6f69
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 13 deletions.
6 changes: 6 additions & 0 deletions sdks/python/apache_beam/yaml/yaml_combine.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,12 @@ class PyJsYamlCombine(beam.PTransform):
See also the documentation on
[YAML Aggregation](https://beam.apache.org/documentation/sdks/yaml-combine/).
Args:
group_by: The field(s) to aggregate on.
combine: The aggregation function to use.
language: The language used to define (and execute) the
custom callables in `combine`. Defaults to generic.
"""
def __init__(
self,
Expand Down
29 changes: 16 additions & 13 deletions sdks/python/apache_beam/yaml/yaml_mapping.py
Original file line number Diff line number Diff line change
Expand Up @@ -428,19 +428,19 @@ class _StripErrorMetadata(beam.PTransform):
For example, in the following pipeline snippet::
- name: MyMappingTransform
type: MapToFields
input: SomeInput
config:
language: python
fields:
...
error_handling:
output: errors
- name: RecoverOriginalElements
type: StripErrorMetadata
input: MyMappingTransform.errors
- name: MyMappingTransform
type: MapToFields
input: SomeInput
config:
language: python
fields:
...
error_handling:
output: errors
- name: RecoverOriginalElements
type: StripErrorMetadata
input: MyMappingTransform.errors
the output of `RecoverOriginalElements` will contain exactly those elements
from SomeInput that failed to processes (whereas `MyMappingTransform.errors`
Expand All @@ -453,6 +453,9 @@ class _StripErrorMetadata(beam.PTransform):

_ERROR_FIELD_NAMES = ('failed_row', 'element', 'record')

def __init__(self):
super().__init__(label=None)

def expand(self, pcoll):
try:
existing_fields = {
Expand Down

0 comments on commit 0bf6f69

Please sign in to comment.