Skip to content

Commit

Permalink
Free Saxon cannot stream, commercial can
Browse files Browse the repository at this point in the history
  • Loading branch information
Martijn Dirkse committed Jul 8, 2024
1 parent 395228e commit 02c660d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/source/advancedDevelopment/muchData/muchData.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Messages stored on disk are stored in directory ``${java.io.tmpdir}/${instance.n

**Streaming**

The Frank!Framework in general `streams` messages flowing through a pipeline. This means that the Frank!Framework only holds in memory the bytes it is currently working with, not entire messages. This helps the Frank!Framework to process large messages without getting out of memory, but there is a caveat. The caveat is related to XSLT processing. When XSLT version 2.0 or version 3.0 is being applied (attribute ``xsltVersion=2`` or ``xsltVersion=3``), then XSLT processor Saxon is used. This XSLT processor does not support streaming. This means that if some message is transformed using an XSLT transformation and if XSLT version 2.0 or 3.0 is used for this, then the entire message has to be stored in memory. This can cause an out of memory exception for large messages.
The Frank!Framework in general `streams` messages flowing through a pipeline. This means that the Frank!Framework only holds in memory the bytes it is currently working with, not entire messages. This helps the Frank!Framework to process large messages without getting out of memory, but there is a caveat. The caveat is related to XSLT processing. When XSLT version 2.0 or version 3.0 is being applied (attribute ``xsltVersion=2`` or ``xsltVersion=3``), then XSLT processor Saxon is used. The free version of this XSLT processor does not support streaming (the commercial one does, but is not used by the Frank!Framework). This means that if some message is transformed using an XSLT transformation and if XSLT version 2.0 or 3.0 is used for this, then the entire message has to be stored in memory. This can cause an out of memory exception for large messages.

This issue can be fixed by using ``xsltVersion=1``. In that case, XSLT processor Xalan is used. Xalan does implement streaming and hence large messages can be processed without memory issues. The price to pay is that XSLT 1.0 is less feature-rich than version 2.0 and version 3.0, making it harder to code the intended XSLT transformation. Most XSLT processing senders and pipes use Saxon by default. The exception is the ``ForEachChildElementPipe``. That pipe uses Xalan by default. This exception has been made by WeAreFrank! because the ``ForEachChildElementPipe`` is meant to iterate over large messages.

Expand Down

0 comments on commit 02c660d

Please sign in to comment.