Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to use this library as a newcomer to Unity #1

Open
WilliamCheen opened this issue Nov 15, 2023 · 3 comments
Open

How to use this library as a newcomer to Unity #1

WilliamCheen opened this issue Nov 15, 2023 · 3 comments

Comments

@WilliamCheen
Copy link

Hi, I'm new to Unity, how can I use this library, can I request a detailed manual?

@WilliamCheen WilliamCheen changed the title How can i use this library as a Unity newer? How to use this library as a newcomer to Unity Nov 15, 2023
@mochi-neko
Copy link
Owner

  1. Add following dependencies to /Packages/manifest.json in your Unity project:
{
    "dependencies": {
        "com.mochineko.voice-activity-detection": "https://github.com/mochi-neko/voice-activity-detection-unity.git?path=/Assets/Mochineko/VoiceActivityDetection#0.4.2",
        "com.cysharp.unitask": "https://github.com/Cysharp/UniTask.git?path=src/UniTask/Assets/Plugins/UniTask",
        "com.neuecc.unirx": "https://github.com/neuecc/UniRx.git?path=Assets/Plugins/UniRx/Scripts",
        "com.naudio.core": "https://github.com/mochi-neko/simple-audio-codec-unity.git?path=/Assets/NAudio/NAudio.Core#0.2.0",
    }
}
  1. Opne your Unity project then these packages will be installed.

  2. Create an AssemblyDefinition file (.asmdef) and add references: Mochineko.VoiceActivityDetection, UniTask and UniRx to "Assembly Definition References".

  3. Implement some script that you want to do with reference to a basic sample code.

  4. You can select voice audio source (microphone or AudioSource component) and output of active voice data (AudioClip, WAV file or nothing (detection only)) with reference to other samples.

  5. You can use voice activity changing event like

 vad
    .VoiceIsActive
    .Subscribe(isActive => Log.Debug("[VAD.Sample] IsActive: {0}", isActive))
    .AddTo(this);

and AudioClip of buffered voice data for active segment like

buffer
    .OnVoiceInactive
    .Subscribe(clip =>
    {
        Log.Info("[VAD.Sample] OnInactive and receive AudioClip and play.");
        audioSource.clip = clip;
        audioSource.Play();
    })
   .AddTo(this);

.

Is there anything unclear to you?

@GitHubUsername1
Copy link

GitHubUsername1 commented Dec 4, 2023

Hey, this VAD software is really useful, thankyou. As a relative newcomer to Unity with some coding experience, I too struggled to incorporate this software into my Unity project. Your reply to this person who has asked for help has helped me a great deal and I am now using your software thanks to it. It might be worth adding some more instruction to your readme as it sounds like you just need to edit your manifest.json file and it'll work, but of course you need to create new .asmdef files.

To any noob reading this, this is what I did to get this working - I included the extra lines as instructed on the readme into my manifest.json of the project I want to use this VAD software in. Then, with the voice-activity-detection project opened in Unity and the other project open, I dragged the Mochineko folders over into my new project as they don't otherwise appear (in Package Manager the package shows as installed but the Mochineko Package/Assets don't appear in the Project window).

Then follow the instruction on the link given above to create the asmdef file. I'll briefly describe what I did for this too - there are 2 subfolders in the Mochineko Asset which should now be in your project window after you dragged the folders across. They both have a asmdef each. You need to create 2 new asmdef files, 1 for each folder. To create a new asmdef file right click in the folder in the Project window, select create new Assembly Definition File, name them (I copied the orginal names). Do it for each folder.

You need to apply some settings in the Inspector for each new asmdef file. You can get the settings by looking at the original asmdef files that were already in the folders, just copy each setting from the original to your new asmdef's. Then delete the original asmdef's. It should all now work.

Edit : I nearly forgot, if you want to use the sample Whisper API project, you'll also need to install this https://github.com/mochi-neko/Whisper-API-unity

@mochi-neko
Copy link
Owner

Thank you for using this library and for providing additional information!
Such feedback is a great motivation for OSS development.

If there is a need, I might be able to make the following adjustments to make it more accessible for beginners:

  • Change settings to make it easier to use even for those who do not use AssemblyDefinition.
  • Provide components that do not require additional scripting.
  • Enable installation of sample code in the Unity Editor through UPM samples.
  • Add more detailed installation instructions to the README.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants