diff --git a/real-time-stt/develop/encrypt-captions.mdx b/real-time-stt/develop/encrypt-captions.mdx index c7febfaac..32256f541 100644 --- a/real-time-stt/develop/encrypt-captions.mdx +++ b/real-time-stt/develop/encrypt-captions.mdx @@ -1,6 +1,6 @@ --- title: 'Encrypt captions' -sidebar_position: 1 +sidebar_position: 2 type: docs description: > Encrypt the captions transcribed with RTT diff --git a/real-time-stt/develop/parse-data.mdx b/real-time-stt/develop/parse-data.mdx index e9585cabb..fc5cb9e0d 100644 --- a/real-time-stt/develop/parse-data.mdx +++ b/real-time-stt/develop/parse-data.mdx @@ -1,6 +1,6 @@ --- title: 'Parse transcription data' -sidebar_position: 0.5 +sidebar_position: 1 type: docs description: > Encrypt the captions transcribed with RTT diff --git a/real-time-stt/develop/record-captions.mdx b/real-time-stt/develop/record-captions.mdx index ed98d670b..35eb49087 100644 --- a/real-time-stt/develop/record-captions.mdx +++ b/real-time-stt/develop/record-captions.mdx @@ -1,6 +1,6 @@ --- title: 'Record captions' -sidebar_position: 2 +sidebar_position: 3 type: docs description: > Record the captions transcribed with RTT in real time diff --git a/real-time-stt/develop/supported-languages.mdx b/real-time-stt/develop/supported-languages.mdx index 419a50b69..ba238a14a 100644 --- a/real-time-stt/develop/supported-languages.mdx +++ b/real-time-stt/develop/supported-languages.mdx @@ -1,6 +1,6 @@ --- title: 'Supported languages' -sidebar_position: 5 +sidebar_position: 6 type: docs description: > The list of languages supported for real-time speech-to-text diff --git a/real-time-stt/develop/transcribe-individual-host.mdx b/real-time-stt/develop/transcribe-individual-host.mdx index af93b7865..61efb3f42 100644 --- a/real-time-stt/develop/transcribe-individual-host.mdx +++ b/real-time-stt/develop/transcribe-individual-host.mdx @@ -1,6 +1,6 @@ --- title: 'Transcribe specified hosts' -sidebar_position: 3 +sidebar_position: 4 type: docs description: > Transcribe the speech of specific channel hosts only diff --git a/real-time-stt/develop/translation.mdx b/real-time-stt/develop/translation.mdx new file mode 100644 index 000000000..42be269fd --- /dev/null +++ b/real-time-stt/develop/translation.mdx @@ -0,0 +1,13 @@ +--- +title: 'Real-time translation' +sidebar_position: 1.5 +type: docs +description: > + Translate transcription text to multiple languages. +--- + +import EnableService from '@docs/shared/real-time-stt/develop/translation.mdx'; + +export const toc = [{}]; + + \ No newline at end of file diff --git a/real-time-stt/develop/update-service.mdx b/real-time-stt/develop/update-service.mdx index 563d690ac..cc3b3e6e6 100644 --- a/real-time-stt/develop/update-service.mdx +++ b/real-time-stt/develop/update-service.mdx @@ -1,6 +1,6 @@ --- title: 'Update service' -sidebar_position: 4 +sidebar_position: 5 type: docs description: > Update the Real-Time STT service diff --git a/real-time-stt/overview/pricing.mdx b/real-time-stt/overview/pricing.mdx index 8d1eae3ca..162c26b79 100644 --- a/real-time-stt/overview/pricing.mdx +++ b/real-time-stt/overview/pricing.mdx @@ -9,26 +9,24 @@ description: > export const toc = [{}]; -This page introduces the billing policy for the add-on provided by Agora. - -Your billing details may differ if you have signed a contract with Agora. +Agora calculates the billing for all projects under your Agora account on a monthly basis. Billing begins once you +enable . -## Overview +This page explains 's billing policy for the add-on. -Agora calculates the billing of all projects under your Agora account on a monthly basis. Billing begins once you -enable . + +Your billing details may differ if you have signed a contract with Agora. + ## Transcription fee -When is enabled for a channel, it transcribes the audio of its active hosts. When is enabled for specific hosts, it only transcribes the audio of the specified hosts and ignores the others. The service employs algorithms that remove the periods of silence and improve WER (Word Error Rate) of transcription. The processed audio is transcribed by the engine and referred to as transcription duration. Agora charges for the transcription duration of all or specified hosts in the channel. - -The unit price is as follows: +When is enabled for a channel, it transcribes the audio of the active hosts. When is enabled for specific hosts, it only transcribes the audio of the specified hosts and ignores others. The service employs algorithms that remove periods of silence and improve Word Error Rate (WER) of transcription. The processed audio is transcribed by the engine and its duration is referred to as the transcription duration. Agora charges for the transcription duration of all or specified hosts in the channel. The unit price is as follows: |Billing item |Usage, minutes per month |Pricing, US$/1,000 minutes| |-------------|--------------------|--------------------------| |Transcription duration | Above 0 | 16.99 | -**Example** +#### Example After you enable : - Host A speaks for 2 minutes and remains silent for 8 minutes. @@ -46,23 +44,49 @@ In this case, the total transcription minutes are calculated as 2 (Host A) + 3 ( ## Language identification fee - supports dynamic language detection when two or more languages are enabled for a channel or specific hosts. The LID (language identification) duration is the same as the transcription duration. + supports dynamic language detection when two or more languages are enabled for a channel or specific hosts. The Language Identification (LID) duration is the same as the transcription duration. |Billing item|Usage, minutes per month |Pricing, US$/1,000 minutes| |--------------------|--------------------|--------------------------| -|Language identification duration|Above 0 | 5.00 | +|Language identification duration | Above 0 | 5.00 | + +#### Example -Examples: -- Let's say there is a channel existing for 10 minutes. There are 3 active hosts - A, B, and C - all in the unmuted state. -- #3: If Spanish and Chinese LID is enabled for this channel at the start, the algorithm will remove 8 minutes of silent audio for host A, 7 minutes for host B and 7 minutes for host C. Therefore, the transcription duration is 2 + 3 + 3 = 8 minutes. the LID duration is 8 minutes, too, being the sum of 2 minutes for host A, 3 minutes for host B, and 3 minutes for host C. +- Suppose a channel exists for 10 minutes. There are three active, unmuted hosts, A, B, and C. +- If Spanish and Chinese LID is enabled for this channel at the start, the algorithm removes 8 minutes of silent audio for host A, 7 minutes for host B and 7 minutes for host C. Therefore, the transcription duration is 2 + 3 + 3 = 8 minutes. The LID duration is also 8 minutes, being the sum of 2 minutes for host A, 3 minutes for host B, and 3 minutes for host C. - If Spanish and Chinese LID is enabled for host A, then the transcription duration and LID duration are both 2 minutes. Notes: -- The transcription duration does not change if you enable more than 1 language. -- If only 1 language is set for a channel or a specified host, the language detection will not start. +- The transcription duration does not change if you enable more than one language. +- If only one language is set for a channel or a specified host, language detection does not start. + +## Translation fee + +When you enable real-time translation for a channel or a user, transcription is activated first. The transcription text is then translated to the target languages. The translation usage minutes are the same as the transcription usage minutes. The real-time transcription and translation usage and cost is shown in your monthly invoice. The unit price is as follows: + +|Billing item | Pricing, US$/1,000 minutes| +|-------------|------------------| +|Translation | 8.99 | + +#### Example + +After you enable : +- Host A speaks Russian for 2 minutes and remains silent for 8 minutes. +- Host B speaks French for 3 minutes and remains silent for 7 minutes. +- Host C speaks Russian for 3 minutes and remains silent for 7 minutes. +- All hosts are silent for the first 2 minutes of the call. +- Russian and French are translated to English. + +In this case, the total transcription minutes are calculated as 2 (Host A) + 3 (Host B) + 3 (Host C) = 8 minutes. The translation minutes are the same as transcription minutes. Agora charges 8 minutes transcription fee and 8 minutes translation fee. + +Total cost = 8/1000*$16.99 + 8/1000*$8.99 = $0.136 + $0.072 = $0.208. + +If you translate Russian and French to English and German, the translation cost is multiplied by 2: + +Total cost = 8/1000*$16.99 + 8/1000*$8.99*2 = $0.136 + $0.144 = $0.28. ## Free-of-charge duration - provides 300 minutes of free-of-charge duration for integration and testing purposes. + provides 300 free-of-charge minutes for integration and testing purposes. Contact sales@agora.io or your AE to get a discount. \ No newline at end of file diff --git a/real-time-stt/overview/product-overview.mdx b/real-time-stt/overview/product-overview.mdx index 4184a3404..0aa708c07 100644 --- a/real-time-stt/overview/product-overview.mdx +++ b/real-time-stt/overview/product-overview.mdx @@ -6,22 +6,26 @@ description: > Create a better user experience with the most accurate live transcription and subtitling. --- - }, { title: "Caption recording", - content: "Upload the transcriptions as .vtt files to cloud storage, then play back audio or video recordings with closed captions (CC). The timestamps in the .vtt file ensure that the text is perfectly synchronized with the audio or video, so it appears exactly where it was generated. ", - link: "" + content: "Upload the transcriptions as .vtt files to cloud storage, then play back audio or video recordings with closed captions (CC). The timestamps in the .vtt file ensure that the text is perfectly synchronized with the audio or video, so that it appears exactly where it was generated.", + link: "../develop/record-captions" }, { title: "Multi-language support", - content: "Real-time transcription supports all major languages and dialects, and each channel can support audio to text transcription for up to two languages simultaneously.", - link: "" - }, - { - title: "Enterprise-grade security and compliance", - content: "Agora is ISO and SOC 2 certified and meets compliance standards for regional privacy laws and industry regulations, including GDPR, CCPA, and HIPAA. Live captions and transcription can be encrypted the same way as the RTC audio or video. ", - link: "" + content: "Real-time transcription supports all major languages and dialects, and each channel can support audio to text transcription for up to two languages simultaneously. Real-time translation supports translation of up to two source languages into five target languages with support for 30+ languages.", + link: "../develop/supported-languages" }, ]} > -Agora Real-Time Speech-To-Text (STT) enables you to transcribe the voice stream of each host to provide live closed captions (CC) and transcription for improved accessibility. Using its advanced features, you can also remove silent audio segments to optimize transcription performance and reduce costs. The output text can be further processed as input for large language models, such as GPT. Real-Time STT serves as a gateway for real-time engagement to enter the AI arena. +Agora Real-Time Speech-To-Text (STT) enables you to transcribe the voice stream of each host to provide live closed captions (CC) and transcription for improved accessibility. Its advanced features, remove silent audio segments to optimize transcription performance and reduce costs. The output text can be translated to multiple languages and further processed as input for large language models, such as GPT. Real-Time STT serves as a gateway for real-time engagement to enter the AI arena. \ No newline at end of file diff --git a/shared/real-time-stt/develop/supported-languages.mdx b/shared/real-time-stt/develop/supported-languages.mdx index ad14bfd54..76d3cf457 100644 --- a/shared/real-time-stt/develop/supported-languages.mdx +++ b/shared/real-time-stt/develop/supported-languages.mdx @@ -1,36 +1,36 @@ supports the following languages: -| GUI Language | STT language parameters | -|---|---| -| Arabic (EG) | ar-EG | -| Arabic (JO) | ar-JO | -| Arabic (SA) | ar-SA | -| Arabic (UAE) | ar-AE | -| Bengali (IN) | bn-IN | -| Chinese | zh-CN | -| Chinese (HK) | zh-HK | -| Chinese (TW) | zh-TW | -| Dutch | nl-NL | -| English (IN) | en-IN | -| English (US) | en-US | -| Filipino | fil-PH | -| French | fr-FR | -| German | de-DE | -| Gujarati | gu-IN | -| Hebrew | he-IL | -| Hindi | hi-IN | -| Indonesian | id-ID | -| Italian | it-IT | -| Japanese | ja-JP | -| Kannada | kn-IN | -| Korean | ko-KR | -| Malay | ms-MY | -| Persian | fa-IR | -| Portuguese | pt-PT | -| Russian | ru-RU | -| Spanish | es-ES | -| Tamil | ta-IN | -| Telugu | te-IN | -| Thai | th-TH | -| Turkish | tr-TR | -| Vietnamese | vi-VN | +| Language | Parameter | +|:---|:---| +| Arabic (EG) | `ar-EG` | +| Arabic (JO) | `ar-JO` | +| Arabic (SA) | `ar-SA` | +| Arabic (UAE) | `ar-AE` | +| Bengali (IN) | `bn-IN` | +| Chinese | `zh-CN` | +| Chinese (HK) | `zh-HK` | +| Chinese (TW) | `zh-TW` | +| Dutch | `nl-NL` | +| English (IN) | `en-IN` | +| English (US) | `en-US` | +| Filipino | `fil-PH` | +| French | `fr-FR` | +| German | `de-DE` | +| Gujarati | `gu-IN` | +| Hebrew | `he-IL` | +| Hindi | `hi-IN` | +| Indonesian | `id-ID` | +| Italian | `it-IT` | +| Japanese | `ja-JP` | +| Kannada | `kn-IN` | +| Korean | `ko-KR` | +| Malay | `ms-MY` | +| Persian | `fa-IR` | +| Portuguese | `pt-PT` | +| Russian | `ru-RU` | +| Spanish | `es-ES` | +| Tamil | `ta-IN` | +| Telugu | `te-IN` | +| Thai | `th-TH` | +| Turkish | `tr-TR` | +| Vietnamese | `vi-VN` | diff --git a/shared/real-time-stt/develop/translation.mdx b/shared/real-time-stt/develop/translation.mdx new file mode 100644 index 000000000..4f6b9a2ab --- /dev/null +++ b/shared/real-time-stt/develop/translation.mdx @@ -0,0 +1,125 @@ + +In addition to transcribing the host's audio in real time, also supports translation during transcription. For example, in scenarios like international conferences, you can transcribe the host's speech, translate the transcribed content, and then push both the original text and the translation back to the channel as subtitles. + +## Understand the tech + +Following are the key features of real-time translation: + +- **Instant translation** + Live speech-to-text translation to keep conversations flowing seamlessly in real-time communication or live streaming. + +- **Multi-language support** + Manage multilingual interactions with speech translation from up to two source languages into five target languages, and support for 30+ languages. + +- **High accuracy** + Advanced Speech Recognition (ASR) captures spoken language and converts it accurately to text using sophisticated recognition technologies. + +- **Translated captions** + Live captions are continually updated during speech, providing readable, translated text. Video Text Track (VTT) files can be stored in the cloud for future reference, AI analysis, or compliance. + +- **Ultra-low Latency translation** + Seamless translation with an end-to-start latency of under 1 second and average end-to-end latency of under 3 seconds. + +- **LLM integration** + Process translated text using custom large language models (LLMs) or integrate additional AI services to enhance capabilities and streamline workflows. + + +This page shows you how to set up translation of the transcribed content when starting a transcription task. + +## Prerequisites + +To follow this guide, first implement basic speech-to-text transcription by following the [Rest quickstart](../get-started/quickstart). + +## Implementation + +When calling `start` to start a transcription task, set `translateConfig` to translate the transcribed text content. + +### Sample request + +The following example shows you how to set up translation when transcribing. The example includes the URL, header, and body. To record and encrypt at the same time in the transcription task, refer to [Record captions](../develop/record-captions) and [Encrypt captions](../develop/encrypt-captions). + +```bash +curl --location --request POST 'https://api.agora.io/v2/projects/{{appId}}/rtsc/speech-to-text/tasks?builderToken={{tokenName}}' \ +--header 'Content-Type: application/json' \ +--header 'Accept: application/json' \ +--header 'Authorization: ' \ +--data '{ + "languages": [ + "zh-CN" + ], + "maxIdleTime": 50, + "rtcConfig": { + "channelName": "", + "subBotUid": "", + "pubBotUid": "", + "subscribeAudioUids": [uint] + }, + "translateConfig": { + "languages": [ + { + "source": "zh-CN", + "target": [ + "en-US", + "ar-SA", + "id-ID", + "fr-FR", + "ja-JP" + ] + } + ] + } +}' +``` + +| Parameter | Type | Description | +|:---------------|:-------------|:-------------------------------------------| +| `source` | String Array | The source language for the translation. | +| `target` | Array | The target languages for translation. You can configure up to 5 target languages. See [Supported languages](#supported-languages). | + +### Sample response + +```json +{ + "taskId": "4e1cc28f21486930fcb77a805af20752", + "createTs": 1730974708, + "status": "STARTED" +} +``` + +| Parameter | Type | Description | +|:---------------|:-------------|:-----------------------------------------| +| `taskId` | string | The task ID. | +| `createTs` | number | The timestamp when the task was created. | +| `status` | enum | Task Status:
  • `IDLE`: The task is not initialized.
  • `PREPARING`: The task has received an initialization request.
  • `PREPARED`: Task initialization is complete.
  • `STARTING`: The task is starting.
  • `CREATED`: Part of the task startup is completed.
  • `STARTED`: All tasks are started and completed.
  • `IN_PROGRESS`: Transcription in progress.
  • `STOPPING`: The task is paused.
  • `STOPPED`: The ask has terminated.
  • `FAILURE_STOP`: Task termination has failed.
    • | + +To `query`, `update`, or `stop` the transcription task, refer to the [Rest quickstart](../get-started/quickstart). + +### Supported languages + +The translation feature of supports the following languages: + +| Language | Parameter | +|:--------------------------|:------------| +| Arabic (Egypt) | `ar-EG` | +| Arabic (Jordan) | `ar-JO` | +| Arabic (Saudi Arabia) | `ar-SA` | +| Arabic (UAE) | `ar-AE` | +| Cantonese (Traditional) | `zh-HK` | +| Chinese (Simplified) | `zh-CN` | +| Chinese (Traditional) | `zh-TW` | +| English | `en-US` | +| French | `fr-FR` | +| German | `de-DE` | +| Hindi | `hi-IN` | +| Indonesian | `id-ID` | +| Italian | `it-IT` | +| Japanese | `ja-JP` | +| Korean | `ko-KR` | +| Malay | `ms-MY` | +| Persian (Iran) | `fa-IR` | +| Portuguese | `pt-PT` | +| Russian | `ru-RU` | +| Spanish | `es-ES` | +| Thai | `th-TH` | +| Turkish | `tr-TR` | +| Vietnamese | `vi-VN` |