Replace JSON Lines with JSON to simplify implementation, improve processing speed, and enhance extensibility

[filip26]

Hi,
I’d like to propose avoiding JSON Lines for the following reasons:

Added Complexity

• Supporting JSON Lines requires additional implementation effort to handle both standard JSON and JSON Lines parsing.
• Converting JSON Lines into standard JSON through pre-processing is inefficient, as it results in redundant parsing with no added value other than compatibility.

Limited Extensibility

• JSON Lines does not allow adding metadata, such as positions or links to subsequent chunks, etc.

Inefficient Processing

• Processing line-by-line in a streaming context is less efficient compared to handling chunks or pages.
• JSON Lines enforces sequential, linear history processing. Standard JSON Object with embedded links enables non-linear history processing.

Using JSON improves adoption, speeds up processing, and supports extensibility.

Please consider the outcome. Thank you.

JSON Lines were likely intended to serve as a replacement for CSV