Document ogg and opus codecs (and publish their source code) (AUD-2784)

[douardda]

Hi,
I am trying to implement a snapcast client using an esp32 (currently an esp32s2 WROVER-I + PCM5142 DAC), and I cannot make the opus decoder work.

The (early) code is available on:

https://github.com/douardda/snapclient-esp-adf

It's basically just a pipeline consisting in a source stream implementing the snapcast protocol, a decoder (can be ogg, flac, or opus) then an i2s writer (so no latency control for now).

I have made the ogg decoder work (but with a lot of glitches), but I had to "feed" the ogg decoder with the stream header sent by the snapserver. However this does not work at all for the opus decoder, which is the preferred codec for such a usage.

So I guess the decoder also expects a header, but this is completely undocumented. So, if this header is needed:

• what format is expected? only the opus codec ones (aka the Identification Header and the Comment Header, as described in the rfc7845), or
• does it expect the opus stream to be in a (ogg?) container?
• if so, what's the point of requiring these headers if the decoder is configured via the usual ADF API (via audio_element_setinfo() )?

I've tried to hand write a minimal opus header (with both the identification header and the comment header, no ogg container) but it does not work either.

And last but not least: why these codecs' source code not available? There is not event a license properly advertized in esp-adf-libs. The NOTICE file (in esp-adp-libs/esp_codec/lib/NOTICE) shows a 2008 copyright file from AOSP with an Apache license, so I expect these codecs should also be under the Apache v2. Could it be possible to get access to the code, then?

Thanks

[douardda]

Also, if nobody watches the esp-adf-libs on gh and reply to issues there, make it clear in the readme page of the github project, please.

[houhaiyan]

@douardda Currently , the ADF supports OGGopus. And the adf will support opus without ogg header in future. If you have some questions about ogg contain decoder、ogg decoder、opus decoder、Ogg-opus decoder， you can push your questiones in github. we can work this out together.

[douardda]

Hi @houhaiyan thanks for the answer.

When you say "you can push your questions in github", on which repo should I do thaht? I guess here and not on the swh-adf-libs one, since issues there are never handled by anyone (which is fine, but it should be documented).

So now, the opus stream I get from the snapcast server only comes with a very thin header (which I believe is specific to snapcast and not related with the RFC6716 or RFC7845) and is not encapsulated in an ogg container (as far as I understand it).

So my question is rather: what should I do to make the opus decoder actually decode this stream? And how can I debug it or understand why it does not work at all? I tried to run it at DEBUG log level with no luck. So due to the lack of source code, how am I supposed to deal with this?

And once again, why isn't the source code available? I get it for other proprietary codecs, but ogg and opus are open, and the only license-related doc I found in swh-adf-libs suggests the underlying source code is actually under the Apache 2 license.

[houhaiyan]

@douardda Do you want to debug the OGG/OGGopus stream which is from the snapcast server? Can you give me the steam? I think I can help you to decoder the stream.

[douardda]

The stream is an 48k/16bits/2ch opus one not encapsulated in an ogg container. See https://github.com/badaix/snapcast/blob/develop/doc/binary_protocol.md for more details.

What I really want is to get access to the source code of this opus decoder, but I never get an answer to the reason for this code not to be available nor under which license it is provided.

[CarlosDerSeher]

I am kind of struggling with this library (esp-adf-libs) too. I am trying to use flac decoder but since we don't have the source code, I wanted to use libFlac directly. But since esp-adf-libs uses libFlac internally I am getting compilation errors regarding redefinition of symbols. Is there any way to exclude this library, I tried to exclude the components which seem to use it but without any success. Looks like some component always pulls it in. At the moment it seems to me the only way to get this to work is to drop ADF completely... But then I will have to modify the DAC drivers and HAL supplied by ADF and add them to my project. Is there any way around this?

[winzkigermany]

ogg and flac support is less-than-great. Lots of hiccups and programming against esp-adf-libs is like black-box-development.
For example: How would I start playing back an ogg file not from the beginning, and instead from -say- 5 seconds into the file?
Also I, too, wonder if the code base is licensed correctly - since that might create downstream licensing issues, too.

[douardda]

once again, why on earth is this code not published (under clear license), and why do we never get answers (from espressif people) to these questions (source code availability and licenses)? see:

• OpenSource esp-adf-libs#12
• Build details of libesp-opus.a esp-adf-libs#10
• Add a License esp-adf-libs#4
• the source code of libesp_codec.a esp-adf-libs#2
• No source? esp-adf-libs#1

[HengYongChao]

Hi @douardda

The ADF copyrights and licenses are here: https://docs.espressif.com/projects/esp-adf/en/latest/COPYRIGHT.html
The esp-adf-libs's attributions is here: https://github.com/espressif/esp-adf-libs/blob/master/esp_codec/lib/Attributions.txt

[maowen]

In case it is helpful, there is an open opus code component for the ESP32 available here: https://github.com/XasWorks/esp-libopus