Skip to content

Commit

Permalink
CN Voice effect updates (#1152)
Browse files Browse the repository at this point in the history
* updates

* review updates

* review update

---------

Co-authored-by: saudsami <[email protected]>
  • Loading branch information
hussain-khalid and saudsami authored Aug 12, 2024
1 parent 67a4d7e commit 5b83aad
Show file tree
Hide file tree
Showing 17 changed files with 658 additions and 71 deletions.
2 changes: 1 addition & 1 deletion broadcast-streaming/advanced-features/voice-effects.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: 'Voice effects'
sidebar_position: 8
type: docs
excluded_platforms: ["web", "electron", "react-native", "unity", "flutter", "react-js", "unreal", "blueprint"]
excluded_platforms: ["web", "react-js", "flutter", "unreal", "blueprint"]
description: >
---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: 'Voice effects'
sidebar_position: 8
type: docs
excluded_platforms: ["web", "electron", "flutter", "react-native", "react-js", "unity", "unreal", "blueprint"]
excluded_platforms: ["web", "react-js", "flutter", "unreal", "blueprint"]
description: >
---

Expand Down
23 changes: 17 additions & 6 deletions shared/video-sdk/develop/voice-effects/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import NotAvailable from '@docs/shared/common/not-available.mdx'

<Vpd k="SDK" /> makes it simple for you to publish audio captured through the microphone to subscribers in a channel. In some cases, users want to modify the captured audio to add voice effects, or change the voice quality before the audio is published. <Vpd k="SDK" /> provides several options that enable you to apply voice effects. This page shows you how to implement these features in your channel.

<PlatformWrapper platform="android, ios, macos, windows">
<PlatformWrapper platform="android, ios, macos, windows, react-native, electron, unity">

## Understand the tech

Expand Down Expand Up @@ -40,10 +40,21 @@ This section shows how to use different sound beautifiers to enhance your audio

Use the following methods to set the desired vocal effects:

* Call the `setVoiceBeautifierPreset` method to apply effects such as chatting bel canto, singing bel canto, and timbre change.
* Utilize the `setAudioEffectPreset` method to configure sound effects, genre sound effects, space shaping, and other effects.
* Apply the `setVoiceConversionPreset` method to entirely transform the original voice.
If the preset effects don't meet your requirements, customize vocal effects using methods such as `setLocalVoicePitch`, `setLocalVoiceEqualization`, and `setLocalVoiceReverb`.
<PlatformWrapper notAllowed="unity">

* Call the `setVoiceBeautifierPreset` method to apply effects such as chatting bel canto, singing bel canto, and timbre change.
* Utilize the `setAudioEffectPreset` method to configure sound effects, genre sound effects, space shaping, and other effects.
* Apply the `setVoiceConversionPreset` method to entirely transform the original voice.
If the preset effects don't meet your requirements, customize vocal effects using methods such as `setLocalVoicePitch`, `setLocalVoiceEqualization`, and `setLocalVoiceReverb`.
</PlatformWrapper>

<PlatformWrapper platform="unity">

* Call the `SetVoiceBeautifierPreset` method to apply effects such as chatting bel canto, singing bel canto, and timbre change.
* Utilize the `SetAudioEffectPreset` method to configure sound effects, genre sound effects, space shaping, and other effects.
* Apply the `SetVoiceConversionPreset` method to entirely transform the original voice.
If the preset effects don't meet your requirements, customize vocal effects using methods such as `SetLocalVoicePitch`, `SetLocalVoiceEqualization`, and `SetLocalVoiceReverb`.
</PlatformWrapper>

Choose the one that best fits your requirements. To implement various voice effects in your project, take the following steps:

Expand All @@ -56,6 +67,6 @@ This section contains information that completes the information in this page, o
<Reference />

</PlatformWrapper>
<PlatformWrapper notAllowed="android, ios, macos, windows">
<PlatformWrapper notAllowed="android, ios, macos, windows, react-native, electron, unity">
<NotAvailable />
</PlatformWrapper>
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ mRtcEngine.setAudioProfile(Constants.AudioProfile.getValue(Constants.AudioProfil

### Voice beautifiers

Call `setAudioEffectPreset` to set music style, space shaping, electronic music, and other effects.
Call `setVoiceBeautifierPreset` to set music style, space shaping, electronic music, and other effects.

- Chat voice beautifier

Expand All @@ -39,7 +39,7 @@ Call `setAudioEffectPreset` to set music style, space shaping, electronic music,
// Example 1: Set male voice effect
// Set the singing voice preset to beautify the male voice and add a small room reverberation effect
mRtcEngine.setVoiceBeautifierPreset(Constants.SINGING_BEAUTIFIER);
// Turn off the effect
// Turn off the effect
mRtcEngine.setVoiceBeautifierPreset(Constants.VOICE_BEAUTIFIER_OFF);

// Example 2: Set female voice effect
Expand Down Expand Up @@ -122,38 +122,38 @@ To implement basic voice-changing effects, call the `setAudioEffectPreset` metho

Call `setLocalVoicePitch`, `setLocalVoiceEqualization`, and `setLocalVoiceReverb` to fine-tune vocal output parameters, including pitch, equalization, and reverberation effects. The following example shows you how to transform a human voice into the voice of the Hulk by manually setting parameter values:

```java
double pitch = 0.5;
mRtcEngine.setLocalVoicePitch(pitch);

// Set the center frequency of the local voice equalization band
// The first parameter is the spectrum sub-band index, the value range is [0,9], representing 10 frequency bands, and the corresponding center frequencies are [31,62,125,250,500,1000,2000,4000,8000,16000] Hz
// The second parameter is the gain value of each frequency interval, the value range is [-15,15], the unit is dB, the default value is 0
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(0), -15);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(1), 3);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(2), -9);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(3), -8);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(4), -6);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(5), -4);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(6), -3);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(7), -2);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(8), -1);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(9), 1);

// Original voice intensity or dry signal, value range [-20,10], unit is dB
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(0), 10);

// Early reflection signal intensity or wet signal, value range [-20,10], unit is dB
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(1), 7);

// The room size for the required reverberation effect. Generally, the larger the room, the stronger the reverberation effect. Value range: [0,100]
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(2), 6);

// Initial delay length of wet signal, value range: [0,200], unit: ms
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(3), 124);

// The continuous strength of reverberation effect, value range: [0,100], the larger the value, the stronger the reverberation effect
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(4), 78);
```
```java
double pitch = 0.5;
mRtcEngine.setLocalVoicePitch(pitch);

// Set the center frequency of the local voice equalization band
// The first parameter is the spectrum sub-band index, the value range is [0,9], representing 10 frequency bands, and the corresponding center frequencies are [31,62,125,250,500,1000,2000,4000,8000,16000] Hz
// The second parameter is the gain value of each frequency interval, the value range is [-15,15], the unit is dB, the default value is 0
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(0), -15);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(1), 3);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(2), -9);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(3), -8);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(4), -6);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(5), -4);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(6), -3);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(7), -2);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(8), -1);
mRtcEngine.setLocalVoiceEqualization(Constants.AUDIO_EQUALIZATION_BAND_FREQUENCY.fromInt(9), 1);

// Original voice intensity or dry signal, value range [-20,10], unit is dB
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(0), 10);

// Early reflection signal intensity or wet signal, value range [-20,10], unit is dB
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(1), 7);

// The room size for the required reverberation effect. Generally, the larger the room, the stronger the reverberation effect. Value range: [0,100]
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(2), 6);

// Initial delay length of wet signal, value range: [0,200], unit: ms
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(3), 124);

// The continuous strength of reverberation effect, value range: [0,100], the larger the value, the stronger the reverberation effect
mRtcEngine.setLocalVoiceReverb(Constants.AUDIO_REVERB_TYPE.fromInt(4), 78);
```

</PlatformWrapper>
Original file line number Diff line number Diff line change
@@ -1,4 +1,157 @@

<PlatformWrapper platform="electron">
Voice effects for Electron is coming soon.

### Set audio scenario and audio profile

For optimal audio quality, take the following steps:

- Call <Link to="{{Global.API_REF_ELECTRON_ROOT}}/class_irtcengine.html#api_irtcengine_setaudioscenario">setAudioScenario</Link> to set the audio scenario to high-quality `AudioScenarioGameStreaming`.
- Call <Link to="{{Global.API_REF_ELECTRON_ROOT}}/class_irtcengine.html#api_irtcengine_setaudioprofile">setAudioProfile</Link> to set the audio encoding properties to high-quality encoding:

- For mono transmission, set the profile to `AudioProfileMusicHighQuality`.
- For stereo transmission, set the profile to `AudioProfileMusicHighQualityStereo`

```javascript
// Create a RtcEngine object
rtcEngine = createAgoraRtcEngine();
// Set the audio scene audio scene settings
rtcEngine.setAudioScenario(AudioScenarioType.AudioScenarioGameStreaming);
// Set audio encoding properties
rtcEngine.setAudioProfile(AudioProfileType.AudioProfileMusicHighQualityStereo);
```

### Voice beautifiers

Call `setVoiceBeautifierPreset` to set music style, space shaping, electronic music, and other effects.

- Chat voice beautifier

```javascript
// Set the vocal effect to magnetic (for male voices)
rtcEngine.setVoiceBeautifierPreset(VoiceBeautifierPreset.ChatBeautifierMagnetic);

// Turn off the effect
rtcEngine.setVoiceBeautifierPreset(VoiceBeautifierPreset.VoiceBeautifierOff);
```

- Singing voice beautifier

```javascript
// Example 1: Set male voice effect
// Set the singing voice preset to beautify the male voice and add a small room reverberation effect
rtcEngine.setVoiceBeautifierPreset(VoiceBeautifierPreset.SingingBeautifier);
// Turn off the effect
rtcEngine.setVoiceBeautifierPreset(VoiceBeautifierPreset.VoiceBeautifierOff);

// Example 2: Set female voice effect
// Call the setVoiceBeautifierParameters method to beautify the female voice and add hall reverberation effect
rtcEngine.setVoiceBeautifierParameters(VoiceBeautifierPreset.SingingBeautifier, 2, 3);
// Turn off the effect
rtcEngine.setVoiceBeautifierPreset(VoiceBeautifierPreset.VoiceBeautifierOff);
```

* Tone change

```javascript
// Set the timbre to be thick
rtcEngine.setVoiceBeautifierPreset(VoiceBeautifierPreset.TimbreTransformationVigorous);
// Turn off the effect
rtcEngine.setVoiceBeautifierPreset(VoiceBeautifierPreset.VoiceBeautifierOff);
```

### Sound Effects

Call `setAudioEffectPreset` to set music style, space shaping, electronic music and other effects.

* Music Style

```javascript
// Set the style to pop
rtcEngine.setAudioEffectPreset(AudioEffectPreset.StyleTransformationPopular);
// Turn off the effect
rtcEngine.setAudioEffectPreset(AudioEffectPreset.AudioEffectOff);
```

* Space shaping

```javascript
// Set the space shaping effect to KTV
rtcEngine.setAudioEffectPreset(AudioEffectPreset.RoomAcousticsKtv);
// Turn off the effect
rtcEngine.setAudioEffectPreset(AudioEffectPreset.AudioEffectOff);
```

* Electronic sound effects

```javascript
// Example 1: Use the preset pitch adjustment method to achieve the electronic music sound effect
// The preset is based on the natural major key with the tonic pitch of C, and corrects the actual pitch of the audio
rtcEngine.setAudioEffectPreset(AudioEffectPreset.PitchCorrection);
// Turn off the effect
rtcEngine.setAudioEffectPreset(AudioEffectPreset.AudioEffectOff);

// Example 2: Adjust the basic mode and tonic pitch to achieve the electronic music sound effect
// Call setAudioEffectParameters to adjust the basic mode of the tone to the natural minor key and the tonic pitch to D
rtcEngine.setAudioEffectParameters(AudioEffectPreset.PitchCorrection, 2, 6);
// Turn off the effect
rtcEngine.setAudioEffectPreset(AudioEffectPreset.AudioEffectOff);
```

### Voice change effects

To implement basic voice-changing effects, call the `setAudioEffectPreset` method. For advanced voice-changing effects, use the `setVoiceConversionPreset` method.

* Basic voice changer

```javascript
// Set the voice change effect to a more neutral voice
rtcEngine.setVoiceConversionPreset(VoiceConversionPreset.VoiceChangerNeutral);
// Turn off the effect
rtcEngine.setVoiceConversionPreset(VoiceConversionPreset.VoiceConversionOff);
```

* Advanced voice changer

```javascript
// Set the vocal effect to 'Hulk'
rtcEngine.setAudioEffectPreset(AudioEffectPreset.VoiceChangerEffectHulk);
// Turn off the effect
rtcEngine.setAudioEffectPreset(AudioEffectPreset.AudioEffectOff);
```

### Custom audio effects

Call `setLocalVoicePitch`, `setLocalVoiceEqualization`, and `setLocalVoiceReverb` to fine-tune vocal output parameters, including pitch, equalization, and reverberation effects. The following example shows you how to transform a human voice into the voice of the Hulk by manually setting parameter values:

```javascript
rtcEngine.setLocalVoicePitch(0.5);

// Set the center frequency of the local voice equalization band
// The first parameter is the spectrum sub-band index, the value range is [0,9], representing 10 frequency bands, and the corresponding center frequencies are [31,62,125,250,500,1000,2000,4000,8000,16000] Hz
// The second parameter is the gain value of each frequency interval, the value range is [-15,15], the unit is dB, the default value is 0
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand31, -15);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand62, 3);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand125, -9);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand250, -8);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand500, -6);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand1k, -4);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand2k, -3);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand4k, -2);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand8k, -1);
rtcEngine.setLocalVoiceEqualization(AudioEqualizationBandFrequency.AudioEqualizationBand16k, 1);

// Original voice intensity or dry signal, value range [-20,10], unit is dB
rtcEngine.setLocalVoiceReverb(AudioReverbType.AudioReverbDryLevel, 10);

// Early reflection signal intensity or wet signal, value range [-20,10], unit is dB
rtcEngine.setLocalVoiceReverb(AudioReverbType.AudioReverbWetLevel, 7);

// The room size for the required reverberation effect. Generally, the larger the room, the stronger the reverberation effect. Value range: [0,100]
rtcEngine.setLocalVoiceReverb(AudioReverbType.AudioReverbRoomSize, 6);

// Initial delay length of wet signal, value range: [0,200], unit: ms
rtcEngine.setLocalVoiceReverb(AudioReverbType.AudioReverbWetDelay, 124);

// The continuous strength of reverberation effect, value range: [0,100], the larger the value, the stronger the reverberation effect
rtcEngine.setLocalVoiceReverb(AudioReverbType.AudioReverbStrength, 78);
```
</PlatformWrapper>
Loading

0 comments on commit 5b83aad

Please sign in to comment.