iOS: Capture device cannot be initialized or is initialized with NULL backend

[nortio]

UPDATE: the title was updated in light of the second comment which simplifies the problem, though the two scenarios still seem to be related. Read both for context.

I wasn't sure if this was the right place to ask, but it seems to be an issue within miniaudio.

The problem occurs when using ma_device_init on a device with type ma_device_type_capture on the iOS simulator (iOS 17.2). Here's the code:

    // CONTEXT: I have a single miniaudio context with two devices: one is a playback device,
    // the other (the one below) is a capture one. This is the first scenario, while the second
    // is described in the next comment on this issue
    ma_device_config device_config_capture;

    device_config_capture = ma_device_config_init(ma_device_type_capture);
    //device_config_capture.capture.format = ma_format_s16;
    //device_config_capture.capture.channels = 1;
    //device_config_capture.sampleRate = sample_rate;
    device_config_capture.dataCallback = data_callback_capture;
    //device_config_capture.periodSizeInMilliseconds = 10; // 10ms
    //device_config_capture.performanceProfile = ma_performance_profile_low_latency;

    ma_result res = ma_device_init(&context, &device_config_capture, &device_capture);
    if (res != MA_SUCCESS) {
        LERROR("Failed to init capture device. Error: %d - %s", res, ma_result_description(res))
        return -4;
    }

All the configuration options are commented out but they don't make any difference. Initially the call falls with MA_ERROR: with some debugging, I've found the issue to be originating from here:

    /* Initialize the audio unit. */
    status = ((ma_AudioUnitInitialize_proc)pContext->coreaudio.AudioUnitInitialize)(pData->audioUnit);
    if (status != noErr) {
        ma_free(pData->pAudioBufferList, &pContext->allocationCallbacks);
        pData->pAudioBufferList = NULL;
        ((ma_AudioComponentInstanceDispose_proc)pContext->coreaudio.AudioComponentInstanceDispose)(pData->audioUnit);
        return ma_result_from_OSStatus(status);
    }

The problem here is in pContext->coreaudio.AudioUnitInitialize which fails with kAudioFormatUnsupportedDataFormatError (or similar, the error is 1819304813 which on ossstatus.com is reported as that - I don't know much about iOS development so this may be wrong). Adding some debug print statements I got these values for bestFormat:

Error returned from AudioUnitInitialize: 1718449215
id: 1819304813, sample_rate: 8000.000000, mChannelsPerFrame: 1, mFormatFlags: 12, mBitsPerChannel: 16, mBytesPerFrame: 4, mBytesPerPacket: 4, mFramesPerPacket: 1

Changing the parameters such as number of channels or sample rate does not change any of the above variables. If I understand correctly, this should be the correct behavior on iOS (as far as I understood from reading the code).

Looking through the issues I found #804 which seems to be related, even though I've basically stripped out everything from my code leaving essentially just the example provided above (context is still initialized properly and everything). Initializing playback devices works perfectly and it's in fact what I was doing before (same as #804, I have two separate devices initialized within the same context). I tried different formats and sample rates with no success. What could be the issue? Am I doing something wrong?

Microphone permissions are also granted.

[nortio]

So, I've made more tests reducing all possible variables and simplifying the test case. First of all, I created a simple app in Xcode from scratch using ObjC (not using Switft) and then added miniaudio in the source directory. I've then modified the ViewController.m like this (please ignore any potential ObjC ugliness, I'm not very well versed in either iOS development or ObjC):

#import "ViewController.h"

#define MA_DEBUG_OUTPUT
#define MA_NO_RUNTIME_LINKING
#define MINIAUDIO_IMPLEMENTATION
#include "miniaudio.h"

bool init = false;
ma_device device;
ma_device_config config;
ma_waveform_config sineWaveConfig;
ma_waveform sineWave;
ma_context context;

ma_device capture_device;
ma_device_config capture_config;

void data_callback(ma_device* pDevice, void* pOutput, const void* pInput, ma_uint32 frameCount)
{
    ma_waveform* pSineWave;

    MA_ASSERT(pDevice->playback.channels == 2);

    pSineWave = (ma_waveform*)pDevice->pUserData;
    MA_ASSERT(pSineWave != NULL);

    ma_waveform_read_pcm_frames(pSineWave, pOutput, frameCount, NULL);

    (void)pInput;   /* Unused. */
}

void capture_callback(ma_device* pDevice, void* pOutput, const void* pInput, ma_uint32 frameCount)
{
    float* input = (float*)pInput;
    
    float acc = 0;
    
    for(uint32_t i = 0; i < frameCount; i++) {
        acc += input[i];
    }
    
    float avg = acc / frameCount;

}

@interface ViewController ()

@end

@implementation ViewController

- (void)buttonPressed: (UIButton*)button {
    NSLog(@"Button pressed");
    if(init) {
        //ma_device_uninit(&device);
        ma_device_uninit(&capture_device);
        NSLog(@"Uninit device");
        init = false;
        return;
    }
    
    /*if(ma_context_init(NULL, 0, NULL, &context) != MA_SUCCESS) {
        NSLog(@"Error initializing context");
        return;
    }*/

    /*config = ma_device_config_init(ma_device_type_playback);
    config.playback.format = ma_format_f32;
    config.playback.channels = 2;
    config.dataCallback = data_callback;
    config.sampleRate = 48000;
    config.pUserData = &sineWave;
    
    if (ma_device_init(&context, &config, &device) != MA_SUCCESS) {
            printf("Failed to open playback device.\n");
            return;
    }

    printf("Playback Device Name: %s\n", device.playback.name);
     
    sineWaveConfig = ma_waveform_config_init(device.playback.format, device.playback.channels, device.sampleRate, ma_waveform_type_sine, 0.2, 220);
    ma_waveform_init(&sineWaveConfig, &sineWave);

    if (ma_device_start(&device) != MA_SUCCESS) {
        printf("Failed to start playback device.\n");
        ma_device_uninit(&device);
        return;
    }
    */
    
    capture_config = ma_device_config_init(ma_device_type_capture);
    capture_config.capture.format = ma_format_f32;
    capture_config.dataCallback = capture_callback;
    
    ma_result res = ma_device_init(NULL /* &context */, &capture_config, &capture_device);
    if(res != MA_SUCCESS) {
        NSLog(@"Capture init error: %s", ma_result_description(res));
        return;
    }
    
    printf("Capture Device Name: %s\n", capture_device.capture.name);

    
    if(ma_device_start(&capture_device)!= MA_SUCCESS) {
        NSLog(@"Error starting capture device");
        return;
    }
    
    init = true;
}

- (void)permButtonPressed: (UIButton*)button {
    [AVAudioApplication requestRecordPermissionWithCompletionHandler:^(bool res){
        NSLog(@"Granted: %d", res);
    }];
}

- (void)viewDidLoad {
    [super viewDidLoad];
    // Do any additional setup after loading the view.
    
    UIButton *button = [UIButton buttonWithType:UIButtonTypeRoundedRect];
    [button setTitle:@"Init" forState:UIControlStateNormal];
    [button sizeToFit];
    button.center = CGPointMake(50, 150);
    
    [button addTarget:self action:@selector(buttonPressed:)
     forControlEvents:UIControlEventTouchUpInside];
    
    [self.view addSubview:button];
    
    UIButton *perm = [UIButton buttonWithType:UIButtonTypeRoundedRect];
    [perm setTitle:@"Permission" forState:UIControlStateNormal];
    [perm sizeToFit];
    perm.center = CGPointMake(50, 250);
    
    [perm addTarget:self action:@selector(permButtonPressed:)
     forControlEvents:UIControlEventTouchUpInside];
    
    [self.view addSubview:perm];
}


@end

(once again, I set permissions with NSMicrophoneUsageDescription string in the Info.plist file, then I added this button to request the permission interactively - you can see the result in the next log)

As you can see from the commented out code I've tried both the original approach (one context, two devices) and a simpler approach (single capture device without context). The former yields the same results as previously described in the older comment (Unknown error and everything), while the latter has a different outcome (here's the log):

Granted: 1
Button pressed
DEBUG: WASAPI backend is disabled.
DEBUG: DirectSound backend is disabled.
DEBUG: WinMM backend is disabled.
DEBUG: Attempting to initialize Core Audio backend...
AddInstanceForFactory: No factory registered for id <CFUUID 0x60000026e120> F8BB1C28-BAE8-11D6-9C31-00039315CD46
DEBUG: System Architecture:
DEBUG:   Endian: LE
DEBUG:   SSE2:   YES
DEBUG:   AVX2:   NO
DEBUG:   NEON:   NO
PCMConverter.cpp:738   failed to determine suitable PCM converter
PCMConverter.cpp:738   failed to determine suitable PCM converter
HALSystem.cpp:2216   AudioObjectPropertiesChanged: no such object
AQMEIO_HAL.cpp:2552  timeout
AudioHardware-mac-imp.cpp:2706   AudioDeviceStop: no device with given ID
DEBUG: sndio backend is disabled.
DEBUG: audio(4) backend is disabled.
DEBUG: OSS backend is disabled.
DEBUG: PulseAudio backend is disabled.
DEBUG: ALSA backend is disabled.
DEBUG: JACK backend is disabled.
DEBUG: AAudio backend is disabled.
DEBUG: OpenSL|ES backend is disabled.
DEBUG: Web Audio backend is disabled.
DEBUG: Failed to initialize Custom backend.
DEBUG: Attempting to initialize Null backend...
DEBUG: System Architecture:
DEBUG:   Endian: LE
DEBUG:   SSE2:   YES
DEBUG:   AVX2:   NO
DEBUG:   NEON:   NO
INFO: [Null]
INFO:   NULL Capture Device (Capture)
INFO:     Format:      32-bit IEEE Floating Point -> 32-bit IEEE Floating Point
INFO:     Channels:    2 -> 2
INFO:     Sample Rate: 48000 -> 48000
INFO:     Buffer Size: 480*3 (1440)
INFO:     Conversion:
INFO:       Pre Format Conversion:  NO
INFO:       Post Format Conversion: NO
INFO:       Channel Routing:        NO
INFO:       Resampling:             NO
INFO:       Passthrough:            YES
INFO:       Channel Map In:         {CHANNEL_FRONT_LEFT CHANNEL_FRONT_RIGHT}
INFO:       Channel Map Out:        {CHANNEL_FRONT_LEFT CHANNEL_FRONT_RIGHT}
Capture Device Name: NULL Capture Device

The device can be initialized correctly but with the wrong backend I believe, so it results in empty incoming audio. What is interesting is the error PCMConverter.cpp:738 failed to determine suitable PCM converter which I forgot to mention also happened in the previous comment's case (also happens in the "original" test case with the new project). This time I've also experimented with the runtime linking options: neither the MA_NO_RUNTIME_LINKING with linkage of required frameworks nor the normal behaviour with entitlements worked.

I also would have liked to test with a real device but unfortunately I don't really have access to an iPhone at the moment, so when I'll get it I will do some more tests.

Edit: I also forgot to add that the playback works fine, the issue is only present with capture.

Update: I've confirmed this issue only happens with iOS devices, while on MacOS everything works fine.

[mackron]

Sorry for not getting back to you on this. Unfortunately I'm not going to have a satisfying answer for you. I don't know what's going on here. Certainly your simplified code in your original comment should work. The falling back to the NULL backend is expected - that's just standard fallback functionality.

The fact that it's happening on a simulator is encouraging because at least then I have a chance of reproducing it myself. But I rarely dig out my MacBook, and I'm certainly no iOS expert (I lean heavily on the community for advice) so no timeframe on when I'll get to this one.