diff --git a/docs/source/advancedDevelopment/streaming/streaming.rst b/docs/source/advancedDevelopment/streaming/streaming.rst index db081e4e..6f527af3 100644 --- a/docs/source/advancedDevelopment/streaming/streaming.rst +++ b/docs/source/advancedDevelopment/streaming/streaming.rst @@ -1,15 +1,11 @@ *Work in progress* -Streaming -========= +About large messages +==================== -This section is about an implementation detail of the Frank!Framework. The Frank!Framework handles large messages differently from small messages to save memory usage - large messages are *streamed*. Usually Frank developers do not have to worry about streaming. However, some errors you may encounter have to do with streaming and hence you need some knowledge about it. +The Frank!Framework treats large messages in a special way to save memory. Frank developers sometimes need some knowledge about this to fix issues with their Frank applications. There is a property ``message.max.memory.size`` to configure the threshold: messages larger than this size are treated specially. -First, large messages are written to a temporary file before they are streamed. Under Linux, they are typically stored in the ``/tmp`` folder. If this folder is on a small disk partition, you may experience OutOfMemory exceptions. There is a property to configure the threshold for this behavior, ``xxx``. If a message is larger than ``xxx`` bytes, it is written to a temporary file and then streamed. - -TODO: Find the right property name for ``xxx`` and the right comparison criterion. - -Second, messages that are already in a file are not copied to a temp file before they are streamed. This typically applies to the contents of a file handled by a ``LocalFileSystemPipe``. This behavior is more general: the Frank!Framework has intelligence to determine whether a file has been stored already - it is only written to a temp file if it is not available otherwise. +First, large messages are written to a temporary file before they are processed. Under Linux, they are typically stored in the ``/tmp`` folder. If this folder is on a small disk partition, an OutOfMemory Java exception may result. As an exception, messages that are already in a file are not copied to a temp file before they are processed. This exception typically applies to the contents of a file handled by a ``LocalFileSystemPipe``. This exception is more general: the Frank!Framework has intelligence to determine whether a file has been stored already - it is only written to a temp file if it is not available otherwise. Ladybug helps you to debug issues with streaming. Checkpoints about streamed messages are clearly shown to be so.