Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
dolfinus authored Dec 16, 2023
1 parent c99ab54 commit ad72fa6
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,12 +47,12 @@ When reading files the API accepts several options:
* `inferSchema`: if `true`, attempts to infer an appropriate type for each resulting DataFrame column, like a boolean, numeric or date type. If `false`, all resulting columns are of string type. Default is `true`.
* `columnNameOfCorruptRecord`: The name of new field where malformed strings are stored. Default is `_corrupt_record`.

Note: if you pass `schema` explicitly, you should add `_corrupt_record` field to the schema, like this:
Note: this field should be present in the dataframe schema if it is passed explicitly, like this:
```python
schema = StructType([StructField("my_field", TimestampType()), StructField("_corrupt_record", StringType())])
spark.read.format("xml").options(rowTag='item').schema(schema).load("file.xml")
```
Otherwise the parsing corrupt record will lead to creating row with all `null` fields, and you cannot access the original xml string.
If schema is infered, this field is added automatically.
* `attributePrefix`: The prefix for attributes so that we can differentiate attributes and elements. This will be the prefix for field names. Default is `_`. Can be empty, but only for reading XML.
* `valueTag`: The tag used for the value when there are attributes in the element having no child. Default is `_VALUE`.
* `charset`: Defaults to 'UTF-8' but can be set to other valid charset names
Expand Down

0 comments on commit ad72fa6

Please sign in to comment.