Merge pull request #3850 from AgoraIO/4.5.0-framework-release

4.5.0 framework release

en-US/dita/RTC-NG/config/keys-rtc-ng-links.ditamap

@@ -78,5 +78,12 @@
                 </keywords>
             </topicmeta>
         </keydef>
+        <keydef keys="page-size" href="https://developer.android.com/guide/practices/page-sizes" scope="external" format="html">
+                <topicmeta>
+                    <keywords>
+                        <keyword>Support 16 KB page sizes</keyword>
+                    </keywords>
+                </topicmeta>
+        </keydef>
 </topichead>
 </map>

en-US/dita/RTC-NG/release/release_notes.dita

@@ -86,7 +86,7 @@
     <li>In the <codeph><apiname keyref="DEGRADATION_PREFERENCE"/></codeph> enumeration class, a new <codeph><ph keyref="MAINTAIN_AUTO"/></codeph> (-1) enumeration is added, replacing the original <codeph><ph keyref="MAINTAIN_QUALITY"/></codeph> (1) as the default value. In this mode, the SDK will automatically choose between <codeph><ph keyref="MAINTAIN_FRAMERATE"/></codeph>, <codeph><ph keyref="MAINTAIN_BALANCED"/></codeph>, or <codeph><ph keyref="MAINTAIN_RESOLUTION"/></codeph> based on your video scene settings to achieve the optimal overall quality experience (QoE).</li>
     </ul></p>
 </li>
-<li props="android"><ph><b>16 KB memory page size (Android)</b></ph>
+<li props="android unity flutter rn unreal bp"><ph><b>16 KB memory page size (Android)</b></ph>
                         <p>Starting from Android 15, the system adds support for 16 KB memory page size, as detailed in <xref keyref="page-size"/>. To ensure the stability and performance of the app, starting from this version, the SDK supports 16 KB memory page size, ensuring seamless operation on devices with both 4 KB and 16 KB memory page sizes, enhancing compatibility and preventing crashes.</p>
                     </li>
                 </ol>
@@ -119,7 +119,7 @@
 <li><ph><b>Filter effects</b></ph>
     <p>This version introduces the <codeph><apiname keyref="setFilterEffectOptions"/></codeph> method. You can pass a cube map file (.cube) in the <codeph>config</codeph> parameter to achieve custom filter effects such as whitening, vivid, cool, black and white, etc. Additionally, the SDK provides a built-in <codeph>built_in_whiten_filter.cube</codeph> file for quickly achieving a whitening filter effect.</p>
 </li>
-<li><ph><b>Local audio mixing</b></ph>
+<li props="android apple cpp cs unity flutter electron rn unreal"><ph><b>Local audio mixing</b></ph>
     <p>This version introduces the local audio mixing feature. You can call the <codeph><apiname keyref="startLocalAudioMixer"/></codeph> method to mix the audio streams from the local microphone, media player, sound card, and remote audio streams into a single audio stream, which can then be published to the channel. When you no longer need audio mixing, you can call the <codeph><apiname keyref="stopLocalAudioMixer"/></codeph> method to stop local audio mixing. During the mixing process, you can call the <codeph><apiname keyref="updateLocalAudioMixerConfiguration"/></codeph> method to update the configuration of the audio streams being mixed.</p>
     <p>Example use cases for this feature include:
         <ul>
@@ -136,7 +136,7 @@
 <li props="android unity rn flutter unreal"><ph><b>EGL context (Android)</b></ph>
                         <p>This version introduces the <codeph><apiname keyref="setExternalRemoteEglContext"/></codeph> method, which is used to set the EGL context for rendering remote video streams. When using Texture format video data for remote video self-rendering, you can use this method to replace the SDK's default remote EGL context, achieving unified EGL context management.</p>
                     </li>
-<li><ph><b>Color space settings</b></ph>
+<li props="android apple cpp cs unity flutter electron rn unreal"><ph><b>Color space settings</b></ph>
     <p>This version adds the <parmname>colorSpace</parmname> parameter to <codeph><apiname keyref="VideoFrame"/></codeph> and <codeph><apiname keyref="ExternalVideoFrame"/></codeph>. You can use this parameter to set the color space properties of the video frame. By default, the color space uses Full Range and BT.709 standard configuration. You can flexibly adjust according to your own capture or rendering needs, further enhancing the customization capabilities of video processing.</p>
 </li>
 <li><ph><b>Others</b></ph>
@@ -161,15 +161,15 @@
 <li><ph><b>Snapshot at specified video observation points</b></ph>
     <p>This version introduces the <codeph><apiname keyref="takeSnapshot2"/></codeph> and <codeph><apiname keyref="takeSnapshotEx2"/></codeph> methods. You can use the <codeph>config</codeph> parameter when calling these methods to take snapshots at specified video observation points, such as before encoding, after encoding, or before rendering, to achieve more flexible snapshot effects.</p>
 </li>
-<li><ph><b>Custom audio capture improvements</b></ph>
+<li props="android apple cpp cs unity flutter electron rn unreal"><ph><b>Custom audio capture improvements</b></ph>
     <p>This version adds the <codeph>enableAudioProcessing</codeph> member parameter to <codeph><apiname keyref="AudioTrackConfig"/></codeph>, which is used to control whether to enable 3A audio processing for custom audio capture tracks of the <codeph>AUDIO_TRACK_DIRECT</codeph> type. The default value of this parameter is <codeph><ph keyref="false"/></codeph>, meaning that audio processing is not enabled. Users can enable it as needed, enhancing the flexibility of custom audio processing.</p>
 </li>
 <li><ph><b>Other Improvements</b></ph>
     <ul>
-        <li props="android ios unity flutter rn unreal">In scenarios where Alpha transparency effects are achieved by stitching video frames and Alpha data, the rendering performance on the receiving end has been improved, effectively reducing stuttering and latency. (Android, iOS)</li>
+        <li props="android ios unity flutter rn unreal bp">In scenarios where Alpha transparency effects are achieved by stitching video frames and Alpha data, the rendering performance on the receiving end has been improved, effectively reducing stuttering and latency. (Android, iOS)</li>
         <li>Optimizes the logic for calling <codeph><apiname keyref="queryDeviceScore"/></codeph> to obtain device score levels, improving the accuracy of the score results.</li>
         <li props="cpp framework">Supports using virtual cameras in YV12 format as video capture devices. (Windows)</li>
-        <li props="ios">After calling <codeph><apiname keyref="enableLocalAudio"/></codeph> to disable local audio capture within the channel, the mute side button on the phone can be used to mute the background sound effects played by the app. (iOS)</li>
+        <li props="ios unity flutter rn unreal bp">After calling <codeph><apiname keyref="enableLocalAudio"/></codeph> to disable local audio capture within the channel, the mute side button on the phone can be used to mute the background sound effects played by the app. (iOS)</li>
         <li>When calling <codeph><apiname keyref="switchSrc"/></codeph> to switch between live streams or on-demand streams of different resolutions, smooth and seamless switching can be achieved. An automatic retry mechanism has been added in case of switching failures. The SDK will automatically retry 3 times after a failure. If it still fails, the <codeph><apiname keyref="onPlayerEvent"/></codeph> callback will report the <codeph><apiname keyref="PLAYER_EVENT_SWITCH_ERROR"/></codeph> event, indicating an error occurred during media resource switching.</li>
         <li>When calling <codeph><apiname keyref="setPlaybackSpeed"/></codeph> to set the playback speed of an audio file, the minimum supported speed is 0.3x.</li>
     </ul>
@@ -178,7 +178,7 @@
             </body>
         </topic>
 <topic id="bugfix">
-    <title>Bug fixes</title>
+    <title>Issues fixed</title>
     <body>
         <p>This version fixes the following issues:</p>
         <ul>
@@ -195,8 +195,9 @@
             <li props="apple unity flutter rn unreal bp">In live streaming scenarios, the time taken to reconnect to the live room after the audience end disconnects due to network switching is longer than expected. (iOS)</li>
             <li props="apple unity flutter rn unreal bp">No sound when playing online media resources using the media player after the app starts. (iOS)</li>
             <li props="apple unity flutter rn unreal bp">Occasional instances of no sound in audio capture after resuming from being interrupted by other system apps during audio and video interaction. (iOS)</li>
-            <li>Calling <codeph><apiname keyref="startAudioMixing"/></codeph> and then immediately calling <codeph><apiname keyref="pauseAudioMixing"/></codeph> to pause the music file playback does not take effect.</li>
-            <li props="android unity flutter rn bp">Occasional crashes during audio and video interaction. (Android)</li>
+            <li props="android unity flutter rn unreal">Calling <codeph><apiname keyref="startAudioMixing"/></codeph> and then immediately calling <codeph><apiname keyref="pauseAudioMixing"/></codeph> to pause the music file playback does not take effect.</li>
+            <li props="bp">Calling <codeph><apiname keyref="startAudioMixing2"/></codeph> and then immediately calling <codeph><apiname keyref="pauseAudioMixing"/></codeph> to pause the music file playback does not take effect.</li>
+            <li props="android unity flutter rn bp unreal">Occasional crashes during audio and video interaction. (Android)</li>
         </ul>
     </body>
 </topic>

markdown/RTC 4.x/release-notes/en-US/framework/release_bluerprint_ng.md

@@ -9,6 +9,91 @@ The audio module of the Agora Unreal SDK is incompatible with the AudioMixer mod
 AudioMixerModuleName=
 ```
 
+## v4.5.0
+This version was released on November x, 2024.
+
+#### Compatibility changes
+
+This version includes optimizations to some features, including changes to SDK behavior, API renaming and deletion. To ensure normal operation of the project, update the code in the app after upgrading to this release.
+
+**Changes in strong video denoising implementation**
+This version adjusts the implementation of strong video denoising. The `EUABT_VIDEO_DENOISER_LEVEL` removes the `VIDEO_DENOISER_LEVEL_STRENGTH` enumeration. Instead, after enabling video denoising by calling `SetVideoDenoiserOptions`, you can call the `SetBeautyEffectOptions` method to enable the beauty skin smoothing feature. Using both together will achieve better video denoising effects. For strong denoising, it is recommended to set the skin smoothing parameters as detailed in `SetVideoDenoiserOptions`.
+
+Additionally, due to this adjustment, to achieve the best low-light enhancement effect with a focus on image quality, you need to enable video denoising first and use specific settings as detailed in `SetLowlightEnhanceOptions`.
+
+**Changes in camera plug and unplug status (macOS, Windows)**
+
+In previous versions, when the camera was unplugged and replugged, the OnVideoDeviceStateChanged callback would report the device status as `MEDIA_DEVICE_STATE_ACTIVE(1)` (device in use). Starting from this version, after the camera is replugged, the device status will change to `MEDIA_DEVICE_STATE_IDLE(0)` (device ready).
+
+**Changes in video encoding preferences**
+
+To enhance the user’s video interaction experience, this version optimizes the default preferences for video encoding:
+- In the `ECOMPRESSION_PREFERENCE` enumeration class, a new `PREFER_COMPRESSION_AUTO (-1)` enumeration is added, replacing the original `PREFER_QUALITY (1)` as the default value. In this mode, the SDK will automatically choose between `PREFER_LOW_LATENCY` or `PREFER_QUALITY` based on your video scene settings to achieve the best user experience.
+- In the `EUABT_DEGRADATION_PREFERENCE` enumeration class, a new `MAINTAIN_AUTO (-1)` enumeration is added, replacing the original `MAINTAIN_QUALITY (1)` as the default value. In this mode, the SDK will automatically choose between `MAINTAIN_FRAMERATE`, `MAINTAIN_BALANCED`, or `MAINTAIN_RESOLUTION` based on your video scene settings to achieve the optimal overall quality experience (QoE).
+
+
+**16 KB memory page size (Android)**
+
+Starting from Android 15, the system adds support for 16 KB memory page size, as detailed in [Support 16 KB page sizes](https://developer.android.com/guide/practices/page-sizes). To ensure the stability and performance of the app, starting from this version, the SDK supports 16 KB memory page size, ensuring seamless operation on devices with both 4 KB and 16 KB memory page sizes, enhancing compatibility and preventing crashes.
+
+
+#### New features
+
+**Live show scenario**
+
+This version adds the `APPLICATION_SCENARIO_LIVESHOW(3)` (Live Show) enumeration to the `EVIDEO_APPLICATION_SCENARIO_TYPE`. You can call `SetVideoScenario` to set the video business scenario to show room. In this scenario, fast video rendering and high image quality are crucial. The SDK implements several performance optimizations, such as enabling accelerated audio and video frame rendering to minimize first-frame latency (no need to call `EnableInstantMediaRendering`) to achieve better image quality and bandwidth efficiency.
+
+**Maximum frame rate for video rendering**
+
+This version adds the `SetLocalRenderTargetFps` and `SetRemoteRenderTargetFps` methods, which support setting the maximum frame rate for video rendering locally and remotely. The actual frame rate for video rendering by the SDK will be as close to this value as possible.
+
+In scenarios where the frame rate requirement for video rendering is not high (e.g., screen sharing, online education) or when the remote end uses mid-to-low-end devices, you can use this set of methods to limit the video rendering frame rate, thereby reducing CPU consumption and improving system performance.
+
+**Filter effects**
+
+This version introduces the `SetFilterEffectOptions` method. You can pass a cube map file (.cube) in the `config` parameter to achieve custom filter effects such as whitening, vivid, cool, black and white, etc. Additionally, the SDK provides a built-in `built_in_whiten_filter.cube` file for quickly achieving a whitening filter effect.
+
+**Others**
+
+- The `OnLocalVideoStateChanged` callback adds the `LOCAL_VIDEO_STREAM_REASON_DEVICE_DISCONNECTED` enumeration, indicating that the currently used video capture device has been disconnected (e.g., unplugged). (Windows)
+- The `EMEDIA_DEVICE_STATE_TYPE` adds the `MEDIA_DEVICE_STATE_PLUGGED_IN` enumeration, indicating that the device has been plugged in. (Windows)
+
+#### Improvements
+
+**Virtual background algorithm optimization**
+
+This version upgrades the virtual background algorithm, making the segmentation between the portrait and the background more accurate. There is no background exposure, the body contour of the portrait is complete, and the detail recognition of fingers is significantly improved. Additionally, the edges between the portrait and the background are more stable, reducing edge jumping and flickering in continuous video frames.
+
+**Snapshot at specified video observation points**
+
+This version introduces the `TakeSnapshotWithConfig` and `TakeSnapshotWithConfigEx` methods. You can use the `config` parameter when calling these methods to take snapshots at specified video observation points, such as before encoding, after encoding, or before rendering, to achieve more flexible snapshot effects.
+
+**Others**
+
+- In scenarios where Alpha transparency effects are achieved by stitching video frames and Alpha data, the rendering performance on the receiving end has been improved, effectively reducing stuttering and latency. (Android, iOS)
+- Optimizes the logic for calling `QueryDeviceScore` to obtain device score levels, improving the accuracy of the score results.
+- Supports using virtual cameras in YV12 format as video capture devices. (Windows)
+- When calling `SwitchSrc` to switch between live streams or on-demand streams of different resolutions, smooth and seamless switching can be achieved. An automatic retry mechanism has been added in case of switching failures. The SDK will automatically retry 3 times after a failure. If it still fails, the `OnPlayerEvent` callback will report the `PLAYER_EVENT_SWITCH_ERROR` event, indicating an error occurred during media resource switching.
+- When calling `SetPlaybackSpeed` to set the playback speed of an audio file, the minimum supported speed is 0.3x.
+
+#### Issues fixed
+
+This version fixes the following issues:
+
+- When calling `StartScreenCaptureByWindowId` to share the screen, the window capture area specified by `regionRect` was inaccurate, resulting in incorrect width and height of the screen sharing window seen by the receiving end. (Windows)
+- When the video source type of the sender is in JPEG format, the frame rate on the receiving end occasionally falls below expectations. (Android, iOS)
+- During audio and video interaction, after being interrupted by a system call, the user volume reported by the `OnAudioVolumeIndication` callback was incorrect. (Android)
+- When the receiving end subscribes to the video small stream by default and does not automatically subscribe to any video stream when joining the channel, calling `MuteRemoteVideoStream(uid, false)` after joining the channel to resume receiving the video stream results in receiving the video large stream, which is not as expected. (Android)
+- Occasional errors of not finding system files during audio and video interaction on Windows 7 systems. (Windows)
+- When calling `FollowSystemRecordingDevice` or `FollowSystemPlaybackDevice` to set the audio capture or playback device used by the SDK to not follow the system default audio playback device, the local audio state callback `OnLocalAudioStateChanged` is not triggered when the audio device is removed, which is not as expected. (Windows)
+- Occasional instances where the receiving end cannot hear the sender during audio and video interaction. (iOS)
+- During audio and video interaction, if the sender's device system version is iOS 17, the receiving end occasionally cannot hear the sender. (iOS)
+- In live streaming scenarios, the time taken to reconnect to the live room after the audience end disconnects due to network switching is longer than expected. (iOS)
+- No sound when playing online media resources using the media player after the app starts. (iOS)
+- Occasional instances of no sound in audio capture after resuming from being interrupted by other system apps during audio and video interaction. (iOS)
+- Calling `StartAudioMixing` and then immediately calling `PauseAudioMixing` to pause the music file playback does not take effect.
+- Occasional crashes during audio and video interaction. (Android)
+
 ## v4.4.0
 
 This version releases on August xx, 2024.
@@ -78,7 +163,7 @@ The image below demonstrates how to obtain the `EventHandler` through the [`GetE
 
    - The `CONTENT_INSPECT_IMAGE_MODERATION` enumeration is added in `ECONTENT_INSPECT_TYPE` which means using video moderation extensions from Agora Extensions Marketplace to take video screenshots and upload them.
    - An optional parameter `serverConfig` is added in `FContentInspectConfig`, which is for server-side configuration related to video screenshot and upload via extensions from Agora Extensions Marketplace. By configuring this parameter, you can integrate multiple third-party moderation extensions and achieve flexible control over extension switches and other features. For more details, please contact [technical support](mailto:[email protected]).
-   
+
    Additionally, this version introduces the `EnableContentInspectEx` method, which supports simultaneous screenshot and upload of multiple video streams.
 
 6. **Wildcard token**
@@ -129,9 +214,9 @@ if (Target.Platform == UnrealTargetPlatform.IOS)
 ```
 
 
-## v4.2.1 
+## v4.2.1
 
-This version was released on October xx, 2023. 
+This version was released on October xx, 2023.
 
 This is the first release of Video SDK for Unreal Blueprint.