stream rate

components/changelog/content/config_v1.1.5.mdx

@@ -0,0 +1,5 @@
+- Added [`streamRate`](/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings#streamrate) to the shared endpoint settings.
+    - This setting allows you to control the rate at which data is streamed from the endpoint.
+    - You can set a global value for all endpoints or customize it for each individual endpoint.
+    - The `all` setting can override all individual endpoint values, making them unnecessary if used.
+

pages/changelog/config_v1.1.5.mdx

@@ -0,0 +1,13 @@
+---
+date: 2024/07/13
+title: ⚙️ Config v1.1.5
+---
+
+import { ChangelogHeader } from '@/components/changelog/ChangelogHeader'
+import Content from '@/components/changelog/content/config_v1.1.5.mdx'
+
+<ChangelogHeader />
+
+---
+
+<Content />

pages/docs/configuration/librechat_yaml/object_structure/shared_endpoint_settings.mdx

@@ -0,0 +1,50 @@
+# Shared Endpoint Settings
+
+This page describes the shared configuration settings for all endpoints. The settings highlighted here are available to all configurations under the ["Endpoints"](/docs/configuration/librechat_yaml/object_structure/config#endpoints) field unless noted otherwise.
+
+## Example Configuration
+
+```yaml filename="Shared Endpoint Settings"
+endpoints:
+  openAI:
+    streamRate: 25
+  anthropic:
+    streamRate: 25
+  google:
+    streamRate: 1
+  azureOpenAI:
+    streamRate: 20
+  # the `all` setting would override all the above values, making them unnecessary to be set
+  all:
+    streamRate: 20
+```
+
+## streamRate
+
+**Key:**
+<OptionTable
+  options={[
+    ['streamRate', 'Number', 'The rate at which data is streamed from the endpoint. Useful for controlling the pace of streaming data.', 'streamRate: 25'],
+  ]}
+/>
+
+**Default:** 1
+
+> Allows for streaming data at the fastest rate possible while allowing the system to wait for the next tick
+
+**Notes:**
+- The `all` setting would override all individual endpoint values, making those specific settings unnecessary if used.
+- The value can be customized for each endpoint or set globally using the `all` key.
+- As of 7/13/24, this setting is not yet available for the assistants endpoints
+- Recommended values are between 25-40 for a smooth streaming experience
+- Using a higher rate is a must when serving the app to many users at scale.
+
+---
+
+# Endpoint Settings
+- [Custom Endpoints](/docs/configuration/librechat_yaml/object_structure/custom_endpoint)
+- [OpenAI](/docs/configuration/pre_configured_ai/openai)
+- [Anthropic](/docs/configuration/pre_configured_ai/anthropic)
+- [Google](/docs/configuration/pre_configured_ai/google)
+- [Azure OpenAI](/docs/configuration/librechat_yaml/object_structure/azure_openai)
+- [Assistants](/docs/configuration/librechat_yaml/object_structure/assistants)