Merging the chatgpt and whisper apis to be able to talk to gpt-powered NPCs.

[achraf-oujjir]

Hello. I hope you are well.

I loved your video and your package. Easy to work with and minimal bugs.

I am very novice to unity and c# (1 month) and I have a group school project around "Education and VR". I have been tasked to add the feature of a talking npc (a professor) for which we'll make customized prompts. Different professors, different prompts for different subjects. A kind of virtual school. The idea came from Mesut's video on youtube: https://www.youtube.com/watch?v=QMogW-A9oD8&t=6s&ab_channel=MesutTomruk and I want to reproduce a similar interaction with the NPCs.
I want to know how I can connect the two APIs in regard to the following issues:

1. Is it possible to solve the problem of the duration of the recording? Having a conversation-like flow of the messages will be a very good thing to have. I am thinking maybe a constant recording that stops when the person stays silent for more than 3 seconds. It might cost in RAM and CPU which will make the replies even slower. Is there another solution ?

2. Getting very long prompts. While using chatgpt, it seems like it doesn't finish its answers sometimes. Is there a way to bypass that ??

Another issue is, if I want to interrupt the npc mid-sentence.

Thank you in advance for your help.

[srcnalt]

Hi @achraf-oujjir,

• Yes, you can make the audio sent longer, in the sample it was 5 seconds however OpenAI does not show a max amount in the API Docs. You can turn the sample code into a push-to-talk system which should work for you.

• You can add a part to your prompt such as "Reply in maximum 100 words" then this should help you shorten the result.

[GamedevBranislav]

Yes I use mine keys I used openai and unity just print error : "You exceeded your current quota, please check your plan and billing details"
Yours working just fine?

[achraf-oujjir]

I haven't tested it in a week or so. Did you use the api extensively or did you just get your api keys ? Are they yours ? Have you shared them with other people ??

[GamedevBranislav]

I used only my keys I generate new, I didn't share with anyone, maybe I did something wrong but I wanna confirm this, it is possible make game build and which works so people can download and play with?

[achraf-oujjir]

I just tested right now and I get the same message as you. Here is the link for the api rates: https://platform.openai.com/docs/guides/rate-limits/overview and here is a stackoverflow post about it: https://stackoverflow.com/questions/75898276/openai-chatgpt-gpt-3-5-api-error-429-you-exceeded-your-current-quota-please

[achraf-oujjir]

Hey @GamedevBranislav, did you solve your issue ? if yes, how ??

[zicas2000]

@achraf-oujjir
If I may allowed to contribute in this discussion, this is a good solution to text to speech ,it's free and easy to implement,
https://www.youtube.com/watch?v=XRXbVtr1fog

[achraf-oujjir]

Interesting! I should check if it gives faster speech.