It is possible to play note directly without read midi file?

[Werasilz]

If I want to create piano in Unity how can I play a note without create and read midi file or just create once and play it?
and I worry about performance, is it expensive to create and read midi file multiple time?

[melanchall]

You can always create Playback manually from desired objects:

var tempoMap = TempoMap.Default;
var playback = new Playback(
    new[]
    {
        new Note(NoteName.ASharp, 2).SetLength(MusicalTimeSpan.Quarter, tempoMap)
    },
    tempoMap);

Here we create playback for single quarter note – A#2. Of course you can pass output device to the Playback's constructor. Please read documentation on the playback, you'll find there all required info – https://melanchall.github.io/drywetmidi/api/Melanchall.DryWetMidi.Multimedia.Playback.html.

Also if you create piano, maybe it's better to just send a Note On on key press and a Note Off on key release?

[Werasilz]

Thank you!

[Napst-a]

I've attempted to execute the code you've used as an example but by Unity crashes each time I try to execute. I'm unsure what I'm doing wrong. Here is my code:

using System;
using System.Linq;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Melanchall.DryWetMidi.Core;
using Melanchall.DryWetMidi.Interaction;
using Melanchall.DryWetMidi.Multimedia;
using Melanchall.DryWetMidi.MusicTheory;

public class PlayNote : MonoBehaviour
{
    private const string OutputDeviceName = "Microsoft GS Wavetable Synth";
    private OutputDevice _outputDevice;

    void Start()
    {
        InitializeOutputDevice();
    }
    private void InitializeOutputDevice()
    {
        Debug.Log($"Initializing output device [{OutputDeviceName}]...");

        var allOutputDevices = OutputDevice.GetAll();
        if (!allOutputDevices.Any(d => d.Name == OutputDeviceName))
        {
            var allDevicesList = string.Join(Environment.NewLine, allOutputDevices.Select(d => $"  {d.Name}"));
            Debug.Log($"There is no [{OutputDeviceName}] device presented in the system. Here the list of all device:{Environment.NewLine}{allDevicesList}");
            return;
        }

        _outputDevice = OutputDevice.GetByName(OutputDeviceName);
        Debug.Log($"Output device [{OutputDeviceName}] initialized.");
    }

    void Update()
    {
        if (Input.GetKeyDown(KeyCode.Space))
        {
            PlayOneNote();
        }
    }

    private void PlayOneNote()
    {
        Debug.Log("Attempting to play...");
        var tempoMap = TempoMap.Default;
        var playback = new Playback(
            new[]
            {
        new Melanchall.DryWetMidi.Interaction.Note(NoteName.ASharp, 2).SetLength(MusicalTimeSpan.Quarter, tempoMap)
            },
            tempoMap, _outputDevice);

        playback.Start();
    }
}

Additionally, I plan to use keyboard as an input (as if simulating a piano) but I am really struggling with the NoteOn & NoteOff documentation, specifically how to turn an event into MIDI output. Could you please show a small example of how this could be integrated?

Thank you so much for reading! This is for a university project so help is really appreciated :)

[melanchall]

@Napst-a,

Unity crashes

What do you mean? What exception is thrown? Can you see the error in the debugger?

Also I see the problem with your code:

var playback = new Playback(...);
...
playback.Start();

You create Playback, then call Start and then method PlayOneNote exits. It leads to playback is collected by GC and thus no note will be played: https://melanchall.github.io/drywetmidi/articles/playback/Common-problems.html#playback-doesnt-produce-sound-or-events-logs

[Napst-a]

Thank you so much! It crashed to desktop each time before so wasn't able to diagnose.

Although it works now, I've noticed a large delay between the pressing of space and the hearing of the audio. Is this intended? I need extremely low latency for the audio output and am wondering if this is possible.

Again, thanks so much for your time. I owe you a coffee!

[melanchall]

Is this intended?

No. But because of

it works now

I suppose you've changed the code. Show it please.

how to turn an event into MIDI output

_outputDevice.SendEvent(...);

[Napst-a]

using System;
using System.Linq;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Melanchall.DryWetMidi.Core;
using Melanchall.DryWetMidi.Interaction;
using Melanchall.DryWetMidi.Multimedia;
using Melanchall.DryWetMidi.MusicTheory;

public class PlayNote : MonoBehaviour
{
    private const string OutputDeviceName = "Microsoft GS Wavetable Synth";
    private OutputDevice _outputDevice;
    private Playback _playback;

    void Start()
    {
        InitializeOutputDevice();
    }
    private void InitializeOutputDevice()
    {
        Debug.Log($"Initializing output device [{OutputDeviceName}]...");

        var allOutputDevices = OutputDevice.GetAll();
        if (!allOutputDevices.Any(d => d.Name == OutputDeviceName))
        {
            var allDevicesList = string.Join(Environment.NewLine, allOutputDevices.Select(d => $"  {d.Name}"));
            Debug.Log($"There is no [{OutputDeviceName}] device presented in the system. Here the list of all device:{Environment.NewLine}{allDevicesList}");
            return;
        }

        _outputDevice = OutputDevice.GetByName(OutputDeviceName);
        Debug.Log($"Output device [{OutputDeviceName}] initialized.");
    }

    void Update()
    {
        if (Input.GetKeyDown(KeyCode.Space))
        {
            PlayOneNote();
        }
    }

    private void PlayOneNote()
    {
        Debug.Log("Attempting to play...");
        var tempoMap = TempoMap.Default;
        _playback = new Playback(
            new[]
            {
                new Melanchall.DryWetMidi.Interaction.Note(NoteName.ASharp, 2).SetLength(MusicalTimeSpan.Quarter, tempoMap)
            },
            tempoMap, _outputDevice);

        _playback.Start();
    }
}

As you can see, the only change I made was to make _playback be stored privately, out of scope from the method so it is not collected by the GC. In Unity, there is a delay of about 0.5s. After hitting space and hearing the A#.

[melanchall]

Well, I think we need to use Stopwatch to correctly measure intervals:

    private Stopwatch _stopwatch = new Stopwatch();
    private List<TimeSpan> _timeSpans = new List<TimeSpan>();

    void Update()
    {
        if (Input.GetKeyDown(KeyCode.Space))
        {
            _stopwatch.Start();
            PlayOneNote();
        }
    }

    private void PlayOneNote()
    {
        _timeSpans.Add(_stopwatch.Elapsed);
        Debug.Log("Attempting to play...");
        var tempoMap = TempoMap.Default;
        _playback = new Playback(
            new[]
            {
                new Melanchall.DryWetMidi.Interaction.Note(NoteName.ASharp, 2).SetLength(MusicalTimeSpan.Quarter, tempoMap)
            },
            tempoMap, _outputDevice);

        _playback.Finished += OnFinished;
        _playback.EventPlayed += OnEventPlayed;

        _timeSpans.Add(_stopwatch.Elapsed);
        _playback.Start();
    }

    private void OnEventPlayed(object sender, MidiEventPlayedEventArgs e)
    {
        _timeSpans.Add(_stopwatch.Elapsed);
    }

    private void OnFinished(object sender, EventArgs e)
    {
        foreach (var ts in _timeSpans)
        {
            Debug.Log(ts);
        }
    }

We want to see four times when playback finished. Show them.

As for crash I think it's because you're not disposing Playback objects. You can write this line before creating a new instance of the Playback:

_playback?.Dispose();

[Napst-a]

Press 1

Press 2

Again, I tested it on my desktop and my laptop. I pressed space at the exact same time on both devices but my laptop played the audio marginally faster. I realised this could be my wireless headset, but I checked and found it has a 16-ms fixed latency.

---

For the crashes, I added the disposal exactly as you suggested. Now I do not receive the console error and can press space as fast as I want which is great. However, the crash to the desktop occurs every time and appears to not be related to the previous issue. Either in the second instance of starting or stopping the scene, the crash will occur. The crash logs all followed the same exception information, except one which read:

[melanchall]

Well, as for times: looks like the desktop very slow, because this code:

var tempoMap = TempoMap.Default;
_playback = new Playback(
    new[]
    {
        new Melanchall.DryWetMidi.Interaction.Note(NoteName.ASharp, 2).SetLength(MusicalTimeSpan.Quarter, tempoMap)
    },
    tempoMap, _outputDevice);

_playback.Finished += OnFinished;
_playback.EventPlayed += OnEventPlayed;

executed for ~20 ms according to your first dump. There is nothing there that should take such a long time. And then first event was played ~200 ms later after start of the playback. Looks really strange.

The second dump is what I expect from your code since it's super simple and lightweight.

As for crash, I have no ideas. Please try to examine dmp file. Maybe it's a good idea to report an issue to the Unity tech support.

[Napst-a]

Hi, so I don't think I correctly communicated the issue.

You're correct the the code execution time is excellent and there is no problem after the first dump. However, there is still a delay in hearing the audio. This delay is significant enough that my project is unfeasible with this library. I need my keyboard to feel like a piano when input but currently this is not close to the case because of the delay. Perhaps if you could try for yourself in Unity you would understand my problem better. This is also why I brought up testing on my laptop - the code execution times are the same but the responsiveness of audio is slightly faster.

Do you know any reason why this delay could be occurring or how to reduce it?

Many thanks.

[melanchall]

Hi,

Unfortunately producing the audio is not related with my library. Looking at the second dump I see that all MIDI events are sent to output device at the correct times. So I can't say why Windows (or sound card or something else) adds delay. I can help with DryWetMIDI only, but I don't see any issues.

There is a chance this

void Update()
{
    if (Input.GetKeyDown(KeyCode.Space))

is the place where delay occurs. I don't know how fast Update is called. You can place logging there:

void Update()
{
    Debug.Log("AAA");
    if (Input.GetKeyDown(KeyCode.Space))

Then press space and look at the console how fast AAA appears.

[Napst-a]

Update occurs every frame so that's not the issue. Thank you very much for all your help regardless. Time for me to take a different approach! Please add this information to the Unity package site to let others know realtime sounds are not feasible if you need low latency.

…

On Tue, 21 Mar 2023, 13:26 Maxim Dobroselsky, ***@***.***> wrote: Hi, Unfortunately producing the audio is not related with my library. Looking at the second dump I see that all MIDI events are sent to output device at the correct times. So I can't say why Windows (or sound card or something else) adds delay. I can help with DryWetMIDI only, but I don't see any issues. There is a chance this void Update() { if (Input.GetKeyDown(KeyCode.Space)) is the place where delay occurs. I don't know how fast Update is called. You can place logging there: void Update() { Debug.Log("AAA"); if (Input.GetKeyDown(KeyCode.Space)) Then press space and look at the console how fast AAA appears. — Reply to this email directly, view it on GitHub <#231 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANLFFOMQJVMJKMLXUO2I2PDW5GT7TANCNFSM6AAAAAASOCCCUU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[melanchall]

I think it's not related with my library. As I said events are sent at the correct times. I'm not responsible for how fast devices produce sound. I just send events to devices and that's all.