Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unreal engine beta #1498

Closed
wants to merge 123 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
efda9bb
add new video streaming data doc, fix existing events doc
Kishan-Dhakan Nov 1, 2022
78bcb5c
fix old links
Kishan-Dhakan Nov 11, 2022
e506744
Small changes on review. Update the browsers so the Vercel build runs.
billy-the-fish Dec 2, 2022
bfd852b
Merge branch 'beta' of https://github.com/AgoraIO/Docs-Source into en…
billy-the-fish Dec 2, 2022
564df58
Reintegrate Broadcast streaming.
billy-the-fish Dec 2, 2022
97f4079
Merge pull request #818 from AgoraIO/enhance-broadcast-streaming-kish…
billy-the-fish Dec 2, 2022
dae84cf
Beta docs with new api for media push and latest broadcast streaming.
billy-the-fish Dec 2, 2022
a810d2d
Change header to English.
billy-the-fish Dec 5, 2022
804d4fa
Merge branch 'beta' of https://github.com/AgoraIO/Docs-Source into beta
shahriarpshuvo Dec 8, 2022
4df8ac2
unreal engine get started for video calling
hussain-khalid Dec 14, 2022
9187b9c
updates
hussain-khalid Dec 14, 2022
9c1df35
some updates
hussain-khalid Dec 14, 2022
183fa5e
commit
hussain-khalid Dec 16, 2022
120ef60
issue # 98
hussain-khalid Dec 18, 2022
d22809e
ILS and some updates for video calling
hussain-khalid Dec 20, 2022
daede85
commit
hussain-khalid Dec 20, 2022
180eaa0
updates
hussain-khalid Dec 20, 2022
b4bbc33
Added NCS docs
atovpeko Dec 21, 2022
9f26b1f
Updates on review.
billy-the-fish Dec 21, 2022
0cca870
Merge pull request #1047 from AgoraIO/939-add-docs
billy-the-fish Dec 21, 2022
bcfec80
Merge branch 'milestone39-beta-site' of https://github.com/AgoraIO/Do…
billy-the-fish Dec 21, 2022
5a1f4d2
Add AINS for beta docs.
billy-the-fish Dec 21, 2022
a87313f
call quality for unreal
hussain-khalid Dec 21, 2022
16497d3
some updates
hussain-khalid Dec 22, 2022
93b94b3
updates
hussain-khalid Dec 22, 2022
65e9835
updates
hussain-khalid Dec 22, 2022
de44f01
updates
hussain-khalid Dec 22, 2022
a3efe52
get started for voice calling
hussain-khalid Dec 22, 2022
07ccfa0
updates
hussain-khalid Dec 23, 2022
5403a82
some updates for ILS
hussain-khalid Dec 24, 2022
2c02e5b
updates
hussain-khalid Dec 24, 2022
e4a7a63
svgs
hussain-khalid Dec 24, 2022
4613bb0
commit
hussain-khalid Dec 26, 2022
c6814bb
Updates on review.
billy-the-fish Dec 26, 2022
c647f18
Merge pull request #1015 from AgoraIO/84-quickstart-for-unreal-engine
billy-the-fish Dec 26, 2022
4806c6f
Updates on review.
billy-the-fish Dec 26, 2022
d73e94b
Updates on review.
billy-the-fish Dec 26, 2022
f68c0f3
Merge pull request #1053 from AgoraIO/100-integrate-channel-quality-d…
billy-the-fish Dec 26, 2022
101eb61
product workflow doc for unreal
hussain-khalid Dec 27, 2022
d7c9888
Merge pull request #1055 from AgoraIO/973-add-the-unreal-implementati…
billy-the-fish Dec 27, 2022
9593d56
Merge branch 'milestone16-unrealengine' of https://github.com/AgoraIO…
billy-the-fish Dec 27, 2022
a973529
Updates on review.
billy-the-fish Dec 27, 2022
3df440e
Merge pull request #1060 from AgoraIO/101-implement-the-product-workf…
billy-the-fish Dec 27, 2022
5d9c815
Unreal Engine 4.x, Get Started and Call Quality.
billy-the-fish Dec 27, 2022
0d525ba
Unreal Engine 4.x, Get Started, Screen share and Call Quality.
billy-the-fish Dec 27, 2022
0a0fe39
Remove Broadcast Streaming.
billy-the-fish Dec 27, 2022
d1e76b5
Merge branch 'main' of https://github.com/AgoraIO/Docs-Source into beta
billy-the-fish Dec 27, 2022
7cdbbe5
audio and voice effect for unreal
hussain-khalid Dec 28, 2022
7ace043
updates
hussain-khalid Dec 28, 2022
10514b0
updates
hussain-khalid Dec 28, 2022
b35618f
updates
hussain-khalid Dec 28, 2022
9a75411
Merge branch 'milestone38-extensions-GA' of https://github.com/AgoraI…
billy-the-fish Dec 28, 2022
b2a4c16
updates after code testing
hussain-khalid Dec 28, 2022
bb477e9
geofencing doc for unreal
hussain-khalid Dec 28, 2022
89edfc0
encryption doc for unreal
hussain-khalid Dec 28, 2022
3d86234
Update to 3D Spatial Audio.
billy-the-fish Dec 28, 2022
ec7f151
Change experimental to beta for spatial audio.
billy-the-fish Dec 29, 2022
00675ce
Merge branch 'media-push-update-api' of https://github.com/AgoraIO/Do…
billy-the-fish Dec 29, 2022
cfb3a81
media player for unreal
hussain-khalid Dec 30, 2022
39464ee
updates
hussain-khalid Dec 30, 2022
773089b
Small changes on review.
billy-the-fish Dec 30, 2022
64d9839
Update to explain that there is no UIKit for unreal.
billy-the-fish Dec 30, 2022
1661def
Merge pull request #1068 from AgoraIO/107-geofencing-for-unrealengine
billy-the-fish Dec 30, 2022
c6da485
Updates on review.
billy-the-fish Dec 30, 2022
6703e14
updates
hussain-khalid Dec 30, 2022
aa068fc
Merge pull request #1069 from AgoraIO/106-secure-communication-with-c…
billy-the-fish Dec 30, 2022
11b596f
Merge pull request #1066 from AgoraIO/103-implement-audio-and-voice-e…
billy-the-fish Dec 30, 2022
bdd5480
Updates on review.
billy-the-fish Dec 30, 2022
0c7b0f6
Merge branch '102-play-media-files-to-a-channel-for-unrealengine' of …
billy-the-fish Dec 30, 2022
cf3e5cd
Updates on review.
billy-the-fish Dec 30, 2022
4dea14d
Merge pull request #1075 from AgoraIO/102-play-media-files-to-a-chann…
billy-the-fish Dec 30, 2022
e637faa
Integrate latest changes.
billy-the-fish Dec 30, 2022
db0b3c2
Integrate latest changes for Unreal Engine.
billy-the-fish Dec 30, 2022
0a7501d
Integrate latest changes for Unreal Engine. Try again to see all the …
billy-the-fish Dec 30, 2022
1cfe28a
Integrate latest changes for Unreal Engine. Try again to see all the …
billy-the-fish Jan 2, 2023
7dc3eb8
Integrate latest changes for Unreal Engine. Try again to see all the …
billy-the-fish Jan 2, 2023
302ca7e
cloud-proxy-doc-for-unreal
hussain-khalid Jan 2, 2023
23c811a
small fix
hussain-khalid Jan 2, 2023
30bbb06
:zap: implement metaindexable
shahriarpshuvo Jan 2, 2023
e42a982
:zap: add beta pages to search indexable
shahriarpshuvo Jan 5, 2023
650f510
:zap: add beta pages to search indexable
shahriarpshuvo Jan 5, 2023
02bab7a
multi-channel doc unreal
hussain-khalid Jan 7, 2023
642807c
Merge pull request #1083 from AgoraIO/1082-implement-cloud-proxy-doc-…
billy-the-fish Jan 10, 2023
3c4afab
Merge pull request #1094 from AgoraIO/1092-implement-multi-channel-li…
billy-the-fish Jan 10, 2023
3d6aba9
custom audio and video
hussain-khalid Jan 11, 2023
4405419
Merge branch 'beta' of https://github.com/AgoraIO/Docs-Source into mi…
billy-the-fish Jan 12, 2023
18e8ae6
Video SDK v4.1.0 for Unreal Engine, beta preview.
billy-the-fish Jan 12, 2023
b98fb61
Updates on review.
billy-the-fish Jan 12, 2023
2ad52d5
Merge pull request #1105 from AgoraIO/104-implement-custom-video-and-…
billy-the-fish Jan 12, 2023
2925e45
Video SDK v4.1.0 for Unreal Engine, beta preview 3.
billy-the-fish Jan 12, 2023
7c049e5
custom audio and video doc
hussain-khalid Jan 16, 2023
23e4c71
Update
saudsami Jan 17, 2023
dcfe835
Merge pull request #1125 from AgoraIO/1116-ncs-event-description-is-i…
billy-the-fish Jan 19, 2023
604274e
Merge branch '1126-best-practice-for-cloud-recording-media-push-and-m…
billy-the-fish Jan 19, 2023
c0f0f71
updates
hussain-khalid Jan 19, 2023
e2d37df
virtual background doc
hussain-khalid Jan 20, 2023
e9fa8a9
Add required audio profile setting and bluetooth microphone override …
maxxfrazer Jan 20, 2023
9e1ce43
Added a step to add SpatialAudio binary in Build Phases.
nirm2009 Jan 23, 2023
1568994
Small update on review.
billy-the-fish Jan 24, 2023
8c3ca4c
updates for voice calling
hussain-khalid Jan 24, 2023
84a54c9
Merge pull request #1118 from AgoraIO/105-implement-raw-video-and-aud…
billy-the-fish Jan 25, 2023
ea24dd2
Updates on review.
billy-the-fish Jan 25, 2023
200710e
Merge pull request #1139 from AgoraIO/spatial-ios
maxxfrazer Jan 25, 2023
d379141
Merge branch 'milestone16-unrealengine' of https://github.com/AgoraIO…
billy-the-fish Jan 25, 2023
bfd1458
Merge pull request #1145 from AgoraIO/110-link-check-and-final-check
billy-the-fish Jan 25, 2023
34772fd
1140 spatial audio updates for Android, Flutter and Web (#1167)
saudsami Feb 6, 2023
a3b13c7
Update supported version of Unreal Engine.
billy-the-fish Feb 9, 2023
45377a2
Unreal Engine Video SDK quick start validation code changes
Pankajg123 Feb 12, 2023
426e36a
Merge branch 'beta' of https://github.com/AgoraIO/Docs-Source into mi…
billy-the-fish Feb 13, 2023
dc2ec6f
Video SDK and Voice SDK v4.0.0 Beta 1 for Unreal Engine.
billy-the-fish Feb 14, 2023
ff09e9d
Language review
atovpeko Feb 16, 2023
2c1cb63
incorporated review suggestions.
Pankajg123 Feb 19, 2023
d616a8b
updates
hussain-khalid Feb 22, 2023
4771244
incorporated Hussain review coments.
Pankajg123 Feb 23, 2023
612c35b
Incorporated further suggestions.
Pankajg123 Feb 23, 2023
00e4a42
Merge branches '1152-unreal-engine-validate-the-code-in-these-docs' a…
hussain-khalid Feb 23, 2023
527e1a8
Update on review Get started for Unreal Engine.
billy-the-fish Feb 27, 2023
d0c6373
Merge pull request #1185 from AgoraIO/1152-unreal-engine-validate-the…
billy-the-fish Feb 27, 2023
1eab989
Merge branch 'staging' of https://github.com/AgoraIO/Docs-Source into…
billy-the-fish Jun 7, 2023
9c6b51c
Merge from staging.
billy-the-fish Jun 7, 2023
c06887f
Merge from staging.
billy-the-fish Jun 7, 2023
3790257
Merge from staging.
billy-the-fish Jun 7, 2023
adae677
Merge branch 'staging' of https://github.com/AgoraIO/Docs-Source into…
atovpeko Oct 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -6,6 +6,7 @@ import Electron from './electron.mdx';
import Flutter from './flutter.mdx';
import Unity from './unity.mdx';
import MacOS from './macos.mdx';
import Unreal from './unreal.mdx';

<MacOS />
<Android />
@@ -15,3 +16,4 @@ import MacOS from './macos.mdx';
<Electron/>
<Flutter />
<Unity />
<Unreal />
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
<PlatformWrapper platform="unreal">

### Implement the user interface

To enable the user to play sound and voice effects and modify the audio route, add the following elements to the user interface:

* A button to start and stop audio mixing.
* A button to play the sound effect.
* A button to apply various voice effects.
* Three text widgets for the buttons.

To Implement this UI, take the following steps:

1. Drag **Button** from the **Common** section of the **Palette Panel** to the canvas panel.

A button control appears on the canvas panel.

1. In **Details**, rename **Button_0** to `voiceEffectButton`, then change the following coordinates:

* **Position X** - 650
* **Position Y** - 960
* **Size X** - 338
* **Size Y** - 60

1. Drag **Text** from the **Common** section of the **Palette Panel** and drop it over `playEffectButton`.

You see a text widget is added to `voiceEffectButton`.

1. In **Details**, rename the text widget to `voiceEffectBtnText`, then change the **Text** field to `Play Audio Effect`.

1. Use the same procedure and create two buttons called `playEffectButton` and `audioMixingButton`.

Do not change the text widget names. You only access the play effect button from code.

1. Select `playEffectButton` and change the following coordinates in **Details**:

* **Position X** - 360
* **Position Y** - 960
* **Size X** - 272
* **Size Y** - 60

1. Select `audioMixingButton` and change the following coordinates in **Details**:

* **Position X** - 1296
* **Position Y** - 960
* **Size X** - 184
* **Size Y** - 60

1. Select the **Text** widget of `voiceEffectButton`, then in **Details** change the **Text** field to `Apply Voice Effect`.

1. Select the **Text** widget of `audioMixingButton`, then in **Details** change the **Text** field to `Mix Audio`.


### Handle the system logic

This section describes the steps required to set up access to the UI elements.

1. **Define variables to manage audio effects and access the UI elements**

In `AgoraImplementationDlg.h`, add the following declarations to the `CAgoraImplementation` class:

```cpp
int soundEffectId = 1; // Unique identifier for the sound effect file
std::string soundEffectFilePath = "https://www.soundjay.com/human/applause-01.mp3"; // URL or path to the sound effect
int soundEffectStatus = 0;
int voiceEffectIndex = 0;
bool audioPlaying = false; // Manage the audio mixing state
std::string audioFilePath = "https://www.kozco.com/tech/organfinale.mp3"; // URL or path to the audio mixing file
UPROPERTY(VisibleAnywhere, BlueprintReadWrite, meta = (BindWidget))
UButton* playEffectButton = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadWrite, meta = (BindWidget))
UButton* voiceEffectButton = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadWrite, meta = (BindWidget))
UButton* audioMixingButton = nullptr;
UPROPERTY(BlueprintReadWrite, meta = (BindWidget))
```

1. **Set up event listeners for the UI elements**

To add event listeners for the UI elements, do the following:

1. In `UMyUserWidget.h`, add the following to the `UMyUserWidget` after `void setupVideoSDKEngine();`

```cpp
UFUNCTION(BlueprintCallable)
void OnAudioMixingButtonClick();
UFUNCTION(BlueprintCallable)
void OnVoiceEffectButtonClick(float value);
UFUNCTION(BlueprintCallable)
void OnPlayEffectButtonClick(float value);
```

2. In `MyUserWidget.cpp`, add the following at the end of `setupVideoSDKEngine`:

```cpp
audioMixingButton->OnClicked.AddDynamic(this, &UMyUserWidget::OnAudioMixingButtonClick);
voiceEffectButton->OnClicked.AddDynamic(this, &UMyUserWidget::OnVoiceEffectButtonClick);
playEffectButton->OnClicked.AddDynamic(this, &UMyUserWidget::OnPlayEffectButtonClick);
```

### Implement sound and voice effects

To add sound and voice effect to your <Vpl k="CLIENT" />, take the following steps:

1. **Enable the user to start and stop audio mixing**

When the user presses **Mix Audio**, the <Vg k = "VSDK"/> fetches an audio file using `audioFilePath` and mixed it with the local audio stream. The remote and local users listen the audio file playing with the audio stream. To enable audio mixing in your <Vpl k = "CLIENT"/>, in `MyUserWidget.cpp`, add the following before `setupVideoSDKEngine`:

```cpp
void UMyUserWidget::OnAudioMixingButtonClick()
{
audioPlaying = !audioPlaying;
if (audioPlaying)
{
agoraEngine->startAudioMixing(audioFilePath.c_str(), false, 1, 0);
}
else
{
agoraEngine->stopAudioMixing();
}
}
```

1. **Pre-load sound effects**

To set up playing voice effects, call `preloadEffect` to pre-load the sound effects. In `MyUserWidget.cpp`, add the following at the end of `setupVideoSDKEngine`:

```cpp
// Pre-load sound effects to improve performance
agoraEngine->preloadEffect(soundEffectId, soundEffectFilePath.c_str());
```

1. **Play, pause, or resume playing the sound effect**

When a user presses **Audio Effect**, the sound effect is played. When they press the button again, the effect is paused and resumed alternately. To implement this workflow, in `MyUserWidget.cpp`, add the following before `setupVideoSDKEngine`:

```cpp
void UMyUserWidget::OnVoiceEffectButtonClick()
{
if (soundEffectStatus == 0)
{
// Stopped
agoraEngine->playEffect(
soundEffectId, // The ID of the sound effect file.
soundEffectFilePath.c_str(), // The path of the sound effect file.
0, // The number of sound effect loops. -1 means an infinite loop. 0 means once.
1, // The pitch of the audio effect. 1 represents the original pitch.
0.0, // The spatial position of the audio effect. 0.0 represents that the audio effect plays in the front.
100, // The volume of the audio effect. 100 represents the original volume.
true,// Whether to publish the audio effect to remote users.
0 // The playback starting position of the audio effect file in ms.
);
soundEffectStatus = 1;
}
else if (soundEffectStatus == 1)
{
// Playing
agoraEngine->pauseEffect(soundEffectId);
soundEffectStatus = 2;
}
else if (soundEffectStatus == 2)
{
// Paused
agoraEngine->resumeEffect(soundEffectId);
soundEffectStatus = 1;
}
}
```

1. **Update the UI after the effect finishes playing**

When <Vpd k="SDK" /> has finished playing the sound effect, the `onAudioEffectFinished` event is fired. Your <Vpl k = "CLIENT"/> listens to the callback and invoke the `OnEIDStopAudioEffect` method to update the UI and stop the audio effect. To Implement this logic, take the following steps:

1. In `MyUserWidget.h`, add the following method declaration to `UMyUserWidget` after `void setupVideoSDKEngine();`:

```cpp
void onAudioEffectFinished(int soundId) override;
```

1. In `MyUserWidget.cpp`, add the following before `setupVideoSDKEngine`:

```cpp
void UMyUserWidget::onAudioEffectFinished(int soundId)
{
agoraEngine->stopEffect(soundId);
soundEffectStatus = 0; // Stopped
UE_LOG(LogTemp, Warning, TEXT("UMyUserWidget::onAudioEffectFinished:: Sound effect with id: %u has been finished"), soundId);
}
```

1. **Set an audio profile**

For applications where special audio performance is required, you set a suitable audio profile and audio scenario. In `MyUserWidget.cpp`, add the following at the end of `setupVideoSDKEngine`:

```cpp
// Specify the audio scenario and audio profile
agoraEngine->setAudioProfile(AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO);
agoraEngine->setAudioScenario(AUDIO_SCENARIO_GAME_STREAMING);
```

1. **Apply various voice and audio effects**

When a user presses **Voice Effect**, apply a new voice effect and change the text on the button to describe the effect. To implement this workflow, in `MyUserWidget.cpp`, add the following before `setupVideoSDKEngine`:

```cpp
void UMyUserWidget::OnPlayEffectButtonClick()
{
voiceEffectIndex++;
// Turn off all previous effects
agoraEngine->setVoiceBeautifierPreset(VOICE_BEAUTIFIER_OFF);
agoraEngine->setAudioEffectPreset(AUDIO_EFFECT_OFF);
agoraEngine->setVoiceConversionPreset(VOICE_CONVERSION_OFF);
if (voiceEffectIndex == 1)
{
agoraEngine->setVoiceBeautifierPreset(CHAT_BEAUTIFIER_MAGNETIC);
voiceEffectBtnText->SetText(FText::FromString("Effect: Chat Beautifier"));
}
else if (voiceEffectIndex == 2)
{
agoraEngine->setVoiceBeautifierPreset(SINGING_BEAUTIFIER);
voiceEffectBtnText->SetText(FText::FromString("Effect: Singing Beautifier"));
}
else if (voiceEffectIndex == 3)
{
agoraEngine->setAudioEffectPreset(VOICE_CHANGER_EFFECT_HULK);
voiceEffectBtnText->SetText(FText::FromString("Effect: Hulk"));
}
else if (voiceEffectIndex == 4)
{
agoraEngine->setVoiceConversionPreset(VOICE_CHANGER_BASS);
voiceEffectBtnText->SetText(FText::FromString("Effect: Voice Changer"));
}
else if (voiceEffectIndex == 5)
{
// Sets the local voice equalization.
// The first parameter sets the band frequency. The value ranges between 0 and 9.
// Each value represents the center frequency of the band:
// 31, 62, 125, 250, 500, 1k, 2k, 4k, 8k, and 16k Hz.
// The second parameter sets the gain of each band between -15 and 15 dB.
// The default value is 0.
agoraEngine->setLocalVoiceEqualization(AUDIO_EQUALIZATION_BAND_FREQUENCY::AUDIO_EQUALIZATION_BAND_4K, 3);
agoraEngine->setLocalVoicePitch(0.5);
voiceEffectBtnText->SetText(FText::FromString("Effect: Voice Equalization"));
}
else if (voiceEffectIndex > 5)
{
// Remove all effects
voiceEffectIndex = 0;
agoraEngine->setLocalVoicePitch(1.0);
agoraEngine->setLocalVoiceEqualization(AUDIO_EQUALIZATION_BAND_FREQUENCY::AUDIO_EQUALIZATION_BAND_4K, 0);
voiceEffectBtnText->SetText(FText::FromString("Apply voice effect"));
}
}
```
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -6,6 +6,7 @@ import Unity from './unity.mdx';
import Electron from './electron.mdx';
import Flutter from './flutter.mdx';
import MacOS from './macos.mdx';
import Unreal from './unreal.mdx';

<MacOS />
<Android />
@@ -15,4 +16,4 @@ import MacOS from './macos.mdx';
<Unity />
<Electron/>
<Flutter />

<Unreal />
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
<PlatformWrapper platform="unreal">

3. In `MyUserWidget.cpp`, update `appId`, `channelName` and `token` with the values for your temporary token.

4. In **Unreal Editor**, click **Play**. A moment later you see the project running on your development device.

If this is the first time you run the project, grant microphone and camera access to your app.

<ProductWrapper notAllowed="video-calling">
5. To join as a host, select **Host** and click **Join**.
</ProductWrapper>

<ProductWrapper product="video-calling">
5. To connect to a channel, click **Join**.
</ProductWrapper>

6. Press **Mix Audio**.

You hear the audio file played in the channel mixed with the microphone audio. Press the button again to stop audio mixing.

7. Press **Play Audio Effect**.

You hear the audio file being played. Press the button again to pause and resume the audio. When the audio has finished playing, you see message in **Output Log**.

8. Press **Apply voice effect**.

Put on headphones connected to your web browser app and speak into the microphone of your development device. You hear your voice through the headphones with the voice effect name displayed on the button. Press the button again to test all the implemented voice effects one by one.
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -5,11 +5,14 @@ import Electron from './electron.mdx';
import Unity from './unity.mdx';
import Flutter from './flutter.mdx';
import ReactNative from './react-native.mdx';
import Unreal from './unreal.mdx';


<Android />
<Ios />
<Web />
<Electron/>
<Unity />
<Flutter />
<ReactNative />
<ReactNative />
<Unreal />
Original file line number Diff line number Diff line change
@@ -0,0 +1,255 @@
<PlatformWrapper platform="unreal">

### Implement the user interface

To enable the user to play sound and voice effects and modify the audio route, add the following elements to the user interface:

* A button to start and stop audio mixing.
* A button to play the sound effect.
* A button to apply various voice effects.
* Three text widgets for the buttons.

To Implement this UI, take the following steps:

1. Drag **Button** from the **Common** section of the **Palette Panel** to the canvas panel.

A button control appears on the canvas panel.

1. In **Details**, rename **Button_0** to `voiceEffectButton`, then change the following coordinates:

* **Position X** - 650
* **Position Y** - 960
* **Size X** - 338
* **Size Y** - 60

1. Drag **Text** from the **Common** section of the **Palette Panel** and drop it over `playEffectButton`.

You see a text widget is added to `voiceEffectButton`.

1. In **Details**, rename the text widget to `voiceEffectBtnText`, then change the **Text** field to `Play Audio Effect`.

1. Use the same procedure and create two buttons called `playEffectButton` and `audioMixingButton`.

Do not change the text widget names. You only access the play effect button from code.

1. Select `playEffectButton` and change the following coordinates in **Details**:

* **Position X** - 360
* **Position Y** - 960
* **Size X** - 272
* **Size Y** - 60

1. Select `audioMixingButton` and change the following coordinates in **Details**:

* **Position X** - 1296
* **Position Y** - 960
* **Size X** - 184
* **Size Y** - 60

1. Select the **Text** widget of `voiceEffectButton`, then in **Details** change the **Text** field to `Apply Voice Effect`.

1. Select the **Text** widget of `audioMixingButton`, then in **Details** change the **Text** field to `Mix Audio`.


### Handle the system logic

This section describes the steps required to set up access to the UI elements.

1. **Define variables to manage audio effects and access the UI elements**

In `AgoraImplementationDlg.h`, add the following declarations to the `CAgoraImplementation` class:

```cpp
int soundEffectId = 1; // Unique identifier for the sound effect file
std::string soundEffectFilePath = "https://www.soundjay.com/human/applause-01.mp3"; // URL or path to the sound effect
int soundEffectStatus = 0;
int voiceEffectIndex = 0;
bool audioPlaying = false; // Manage the audio mixing state
std::string audioFilePath = "https://www.kozco.com/tech/organfinale.mp3"; // URL or path to the audio mixing file
UPROPERTY(VisibleAnywhere, BlueprintReadWrite, meta = (BindWidget))
UButton* playEffectButton = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadWrite, meta = (BindWidget))
UButton* voiceEffectButton = nullptr;
UPROPERTY(VisibleAnywhere, BlueprintReadWrite, meta = (BindWidget))
UButton* audioMixingButton = nullptr;
UPROPERTY(BlueprintReadWrite, meta = (BindWidget))
```

1. **Set up event listeners for the UI elements**

To add event listeners for the UI elements, do the following:

1. In `UMyUserWidget.h`, add the following to the `UMyUserWidget` after `void setupVoiceSDKEngine();`

```cpp
UFUNCTION(BlueprintCallable)
void OnAudioMixingButtonClick();
UFUNCTION(BlueprintCallable)
void OnVoiceEffectButtonClick(float value);
UFUNCTION(BlueprintCallable)
void OnPlayEffectButtonClick(float value);
```

2. In `MyUserWidget.cpp`, add the following at the end of `setupVoiceSDKEngine`:

```cpp
audioMixingButton->OnClicked.AddDynamic(this, &UMyUserWidget::OnAudioMixingButtonClick);
voiceEffectButton->OnClicked.AddDynamic(this, &UMyUserWidget::OnVoiceEffectButtonClick);
playEffectButton->OnClicked.AddDynamic(this, &UMyUserWidget::OnPlayEffectButtonClick);
```

### Implement sound and voice effects

To add sound and voice effect to your <Vpl k="CLIENT" />, take the following steps:

1. **Enable the user to start and stop audio mixing**

When the user presses **Mix Audio**, the <Vpd k = "SDK"/> fetches an audio file using `audioFilePath` and mixed it with the local audio stream. The remote and local users listen the audio file playing with the audio stream. To enable audio mixing in your <Vpl k = "CLIENT"/>, in `MyUserWidget.cpp`, add the following before `setupVoiceSDKEngine`:

```cpp
void UMyUserWidget::OnAudioMixingButtonClick()
{
audioPlaying = !audioPlaying;
if (audioPlaying)
{
agoraEngine->startAudioMixing(audioFilePath.c_str(), false, 1, 0);
}
else
{
agoraEngine->stopAudioMixing();
}
}
```

1. **Pre-load sound effects**

To set up playing voice effects, call `preloadEffect` to pre-load the sound effects. In `MyUserWidget.cpp`, add the following at the end of `setupVoiceSDKEngine`:

```cpp
// Pre-load sound effects to improve performance
agoraEngine->preloadEffect(soundEffectId, soundEffectFilePath.c_str());
```

1. **Play, pause, or resume playing the sound effect**

When a user presses **Audio Effect**, the sound effect is played. When they press the button again, the effect is paused and resumed alternately. To implement this workflow, in `MyUserWidget.cpp`, add the following before `setupVoiceSDKEngine`:

```cpp
void UMyUserWidget::OnVoiceEffectButtonClick()
{
if (soundEffectStatus == 0)
{
// Stopped
agoraEngine->playEffect(
soundEffectId, // The ID of the sound effect file.
soundEffectFilePath.c_str(), // The path of the sound effect file.
0, // The number of sound effect loops. -1 means an infinite loop. 0 means once.
1, // The pitch of the audio effect. 1 represents the original pitch.
0.0, // The spatial position of the audio effect. 0.0 represents that the audio effect plays in the front.
100, // The volume of the audio effect. 100 represents the original volume.
true,// Whether to publish the audio effect to remote users.
0 // The playback starting position of the audio effect file in ms.
);
soundEffectStatus = 1;
}
else if (soundEffectStatus == 1)
{
// Playing
agoraEngine->pauseEffect(soundEffectId);
soundEffectStatus = 2;
}
else if (soundEffectStatus == 2)
{
// Paused
agoraEngine->resumeEffect(soundEffectId);
soundEffectStatus = 1;
}
}
```

1. **Update the UI after the effect finishes playing**

When <Vpd k="SDK" /> has finished playing the sound effect, the `onAudioEffectFinished` event is fired. Your <Vpl k = "CLIENT"/> listens to the callback and invoke the `OnEIDStopAudioEffect` method to update the UI and stop the audio effect. To Implement this logic, take the following steps:

1. In `MyUserWidget.h`, add the following method declaration to `UMyUserWidget` after `void setupVoiceSDKEngine();`:

```cpp
void onAudioEffectFinished(int soundId) override;
```

1. In `MyUserWidget.cpp`, add the following before `setupVoiceSDKEngine`:

```cpp
void UMyUserWidget::onAudioEffectFinished(int soundId)
{
agoraEngine->stopEffect(soundId);
soundEffectStatus = 0; // Stopped
UE_LOG(LogTemp, Warning, TEXT("UMyUserWidget::onAudioEffectFinished:: Sound effect with id: %u has been finished"), soundId);
}
```

1. **Set an audio profile**

For applications where special audio performance is required, you set a suitable audio profile and audio scenario. In `MyUserWidget.cpp`, add the following at the end of `setupVoiceSDKEngine`:

```cpp
// Specify the audio scenario and audio profile
agoraEngine->setAudioProfile(AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO);
agoraEngine->setAudioScenario(AUDIO_SCENARIO_GAME_STREAMING);
```

1. **Apply various voice and audio effects**

When a user presses **Voice Effect**, apply a new voice effect and change the text on the button to describe the effect. To implement this workflow, in `MyUserWidget.cpp`, add the following before `setupVoiceSDKEngine`:

```cpp
void UMyUserWidget::OnPlayEffectButtonClick()
{
voiceEffectIndex++;
// Turn off all previous effects
agoraEngine->setVoiceBeautifierPreset(VOICE_BEAUTIFIER_OFF);
agoraEngine->setAudioEffectPreset(AUDIO_EFFECT_OFF);
agoraEngine->setVoiceConversionPreset(VOICE_CONVERSION_OFF);
if (voiceEffectIndex == 1)
{
agoraEngine->setVoiceBeautifierPreset(CHAT_BEAUTIFIER_MAGNETIC);
voiceEffectBtnText->SetText(FText::FromString("Effect: Chat Beautifier"));
}
else if (voiceEffectIndex == 2)
{
agoraEngine->setVoiceBeautifierPreset(SINGING_BEAUTIFIER);
voiceEffectBtnText->SetText(FText::FromString("Effect: Singing Beautifier"));
}
else if (voiceEffectIndex == 3)
{
agoraEngine->setAudioEffectPreset(VOICE_CHANGER_EFFECT_HULK);
voiceEffectBtnText->SetText(FText::FromString("Effect: Hulk"));
}
else if (voiceEffectIndex == 4)
{
agoraEngine->setVoiceConversionPreset(VOICE_CHANGER_BASS);
voiceEffectBtnText->SetText(FText::FromString("Effect: Voice Changer"));
}
else if (voiceEffectIndex == 5)
{
// Sets the local voice equalization.
// The first parameter sets the band frequency. The value ranges between 0 and 9.
// Each value represents the center frequency of the band:
// 31, 62, 125, 250, 500, 1k, 2k, 4k, 8k, and 16k Hz.
// The second parameter sets the gain of each band between -15 and 15 dB.
// The default value is 0.
agoraEngine->setLocalVoiceEqualization(AUDIO_EQUALIZATION_BAND_FREQUENCY::AUDIO_EQUALIZATION_BAND_4K, 3);
agoraEngine->setLocalVoicePitch(0.5);
voiceEffectBtnText->SetText(FText::FromString("Effect: Voice Equalization"));
}
else if (voiceEffectIndex > 5)
{
// Remove all effects
voiceEffectIndex = 0;
agoraEngine->setLocalVoicePitch(1.0);
agoraEngine->setLocalVoiceEqualization(AUDIO_EQUALIZATION_BAND_FREQUENCY::AUDIO_EQUALIZATION_BAND_4K, 0);
voiceEffectBtnText->SetText(FText::FromString("Apply voice effect"));
}
}
```
</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<PlatformWrapper platform="unreal">

3. In `MyUserWidget.cpp`, update `appId`, `channelName` and `token` with the values for your temporary token.

4. In **Unreal Editor**, click **Play**. A moment later you see the project running on your development device.

If this is the first time you run the project, grant microphone and camera access to your app.

5. To connect to a channel, click **Join**.

6. Press **Mix Audio**.

You hear the audio file played in the channel mixed with the microphone audio. Press the button again to stop audio mixing.

7. Press **Play Audio Effect**.

You hear the audio file being played. Press the button again to pause and resume the audio. When the audio has finished playing, you see message in **Output Log**.

8. Press **Apply voice effect**.

Put on headphones connected to your web browser app and speak into the microphone of your development device. You hear your voice through the headphones with the voice effect name displayed on the button. Press the button again to test all the implemented voice effects one by one.

</PlatformWrapper>