Skip to content

Commit

Permalink
Composite Decoder: ref models
Browse files Browse the repository at this point in the history
Signed-off-by: Artem Inzhyyants <[email protected]>
  • Loading branch information
artem1205 committed Dec 19, 2024
1 parent 17add9e commit 655ce35
Show file tree
Hide file tree
Showing 2 changed files with 19 additions and 1 deletion.
15 changes: 15 additions & 0 deletions airbyte_cdk/sources/declarative/declarative_component_schema.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2727,8 +2727,12 @@ definitions:
description: "(This is experimental, use at your own risk)"
type: object
required:
- type
- parser
properties:
type:
type: string
enum: [ CompositeRawDecoder ]
parser:
anyOf:
- "$ref": "#/definitions/GzipParser"
Expand All @@ -2737,7 +2741,13 @@ definitions:
# PARSERS
GzipParser:
type: object
required:
- type
- inner_parser
properties:
type:
type: string
enum: [ GzipParser ]
inner_parser:
anyOf:
- "$ref": "#/definitions/JsonLineParser"
Expand All @@ -2750,7 +2760,12 @@ definitions:
default: utf-8
CsvParser:
type: object
required:
- type
properties:
type:
type: string
enum: [ CsvParser ]
encoding:
type: string
default: utf-8
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1114,6 +1114,7 @@ class JsonLineParser(BaseModel):


class CsvParser(BaseModel):
type: Literal["CsvParser"]
encoding: Optional[str] = "utf-8"
delimiter: Optional[str] = ","

Expand Down Expand Up @@ -1498,7 +1499,8 @@ class RecordSelector(BaseModel):


class GzipParser(BaseModel):
inner_parser: Optional[Union[JsonLineParser, CsvParser]] = None
type: Literal["GzipParser"]
inner_parser: Union[JsonLineParser, CsvParser]


class Spec(BaseModel):
Expand Down Expand Up @@ -1532,6 +1534,7 @@ class CompositeErrorHandler(BaseModel):


class CompositeRawDecoder(BaseModel):
type: Literal["CompositeRawDecoder"]
parser: Union[GzipParser, JsonLineParser, CsvParser]


Expand Down

0 comments on commit 655ce35

Please sign in to comment.