Return from elevenlabs.textToSpeech.convert is not returning readable stream #90

code-a-cola · 2024-10-07T14:41:22Z

I am using the following code:

          const response: any = await elevenlabs.textToSpeech.convert(voice, {
            enable_logging: true,
            optimize_streaming_latency: ElevenLabs.OptimizeStreamingLatency.Zero,
            model_id: 'eleven_turbo_v2_5',
            output_format: 'ulaw_8000',
            text,
            voice_settings: {
                stability: 0.1,
                similarity_boost: 0.3,
                style: 0.2
            },
          });

Per the documenation found here:

https://elevenlabs.io/docs/api-reference/integrating-with-twilio

However the readable stream is always returning empty:

{"response":{"encoding":null,"events":{"close":[],"data":[],"end":[],"error":[],"pause":[],"readable":[],"resume":[]},"paused":false,"readableStream":{},"reader":{},"resumeCallback":null}}

Even though on the website I can view the converted text, so the tts is working it's just not responding with the stream I can send back to Twilio.

The text was updated successfully, but these errors were encountered:

code-a-cola · 2024-10-07T15:13:49Z

However... elevenlabs.textToSpeech.convertWithTimestamps is returning results

code-a-cola · 2024-10-07T15:14:44Z

So I believe something is happening... or not happening properly on the convert method.

code-a-cola · 2024-10-08T10:29:13Z

I obtained help from your discord community. I was able to play back audio on Twilio using your raw javascript API. So there is definitely an issue with the .convert method in the SDK.

louisjoecodes · 2024-10-11T15:15:26Z

Hi Shane,

Thanks for bringing this up. The textToSpeech.convert method in the latest elevenlabs SDK version 0.16.1 (at the time of writing) now returns a different type of stream compared to version 0.2.2 used in the tutorial repository. This change is likely causing the issue you're experiencing.

You can fix this by converting the response into a Node.js readable stream using Readable.from(). Here's how you can modify your code:

const response = await elevenlabs.textToSpeech.convert(voice, options);

// Convert the response into a Node.js readable stream
const readableStream = Readable.from(response);

// Proceed with readableStream as before

This should resolve the issue you're experiencing. I'll update the tutorial to reflect these changes to ensure it's aligned with the latest SDK version.

Details:

In version 0.16.1, the textToSpeech.convert method returns a Node18UniversalStreamWrapper that wraps a Web Streams ReadableStream.
This differs from version 0.2.2, where it returned a Node.js Gunzip stream.
Because of this change, your existing code that processes the stream may not work as expected.
By using Readable.from(response), you convert the Web Streams ReadableStream into a Node.js readable stream, making it compatible with your existing code and allowing the audio to play correctly when sent to Twilio.

louisjoecodes · 2024-10-11T16:28:04Z

I have just updated the Twilio guide & example repository to reflect the SDK structural changes mentioned above.

Sorry this happened, let me know if I can help with anything else!

louisjoecodes · 2024-10-11T16:44:27Z

Looks like we will also need to align the .convert return type: #89

This was referenced Oct 11, 2024

chore: bump elevenlabs-js version in twilio example elevenlabs/elevenlabs-examples#55

Merged

fix: twilio guide compatibility with updated elevenlabs-js SDK elevenlabs/elevenlabs-docs#413

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Return from elevenlabs.textToSpeech.convert is not returning readable stream #90

Return from elevenlabs.textToSpeech.convert is not returning readable stream #90

code-a-cola commented Oct 7, 2024 •

edited

Loading

code-a-cola commented Oct 7, 2024

code-a-cola commented Oct 7, 2024

code-a-cola commented Oct 8, 2024

louisjoecodes commented Oct 11, 2024

louisjoecodes commented Oct 11, 2024

louisjoecodes commented Oct 11, 2024 •

edited

Loading

Return from elevenlabs.textToSpeech.convert is not returning readable stream #90

Return from elevenlabs.textToSpeech.convert is not returning readable stream #90

Comments

code-a-cola commented Oct 7, 2024 • edited Loading

code-a-cola commented Oct 7, 2024

code-a-cola commented Oct 7, 2024

code-a-cola commented Oct 8, 2024

louisjoecodes commented Oct 11, 2024

louisjoecodes commented Oct 11, 2024

louisjoecodes commented Oct 11, 2024 • edited Loading

code-a-cola commented Oct 7, 2024 •

edited

Loading

louisjoecodes commented Oct 11, 2024 •

edited

Loading