diff --git a/docs/source/advancedDevelopment/advancedDevelopment.rst b/docs/source/advancedDevelopment/advancedDevelopment.rst index c32943eb..a37d9b6d 100644 --- a/docs/source/advancedDevelopment/advancedDevelopment.rst +++ b/docs/source/advancedDevelopment/advancedDevelopment.rst @@ -13,5 +13,4 @@ Chapter :ref:`gettingStarted` provides the basics of Frank development. This cha dockerDevel/dockerDevel narayana/narayana deployment/maven - streaming/streaming - + muchData/muchData diff --git a/docs/source/advancedDevelopment/muchData/muchData.rst b/docs/source/advancedDevelopment/muchData/muchData.rst new file mode 100644 index 00000000..179bad8b --- /dev/null +++ b/docs/source/advancedDevelopment/muchData/muchData.rst @@ -0,0 +1,8 @@ +*Work in progress* + +About large messages +==================== + +The Frank!Framework treats large messages in a special way to save memory. Frank developers sometimes need some knowledge about this to fix issues with their Frank applications. There is a property ``message.max.memory.size`` to configure the threshold: messages larger than this size are treated specially. Large messages are written to a temporary file before they are processed. Under Linux, they are typically stored in the ``/tmp`` folder. If this folder is on a small disk partition, an OutOfMemory Java exception may result. As an exception, messages that are already in a file are not copied to a temp file before they are processed. This exception typically applies to the contents of a file handled by a ``LocalFileSystemPipe``. This exception is more general: the Frank!Framework has intelligence to determine whether a file has been stored already - it is only written to a temp file if it is not available otherwise. + +TODO: What is different before 7.9 from the behavior from 7.9 and later? diff --git a/docs/source/advancedDevelopment/streaming/streaming.rst b/docs/source/advancedDevelopment/streaming/streaming.rst deleted file mode 100644 index 6f527af3..00000000 --- a/docs/source/advancedDevelopment/streaming/streaming.rst +++ /dev/null @@ -1,13 +0,0 @@ -*Work in progress* - -About large messages -==================== - -The Frank!Framework treats large messages in a special way to save memory. Frank developers sometimes need some knowledge about this to fix issues with their Frank applications. There is a property ``message.max.memory.size`` to configure the threshold: messages larger than this size are treated specially. - -First, large messages are written to a temporary file before they are processed. Under Linux, they are typically stored in the ``/tmp`` folder. If this folder is on a small disk partition, an OutOfMemory Java exception may result. As an exception, messages that are already in a file are not copied to a temp file before they are processed. This exception typically applies to the contents of a file handled by a ``LocalFileSystemPipe``. This exception is more general: the Frank!Framework has intelligence to determine whether a file has been stored already - it is only written to a temp file if it is not available otherwise. - -Ladybug helps you to debug issues with streaming. Checkpoints about streamed messages are clearly shown to be so. - -TODO: Add screenshot here. -TODO: What is different before 7.9 from the behavior from 7.9 and later?