Whats an Voice prompting

[AnonymousCoderArtist]

##Whats Voice prompting

I am willing to contribute and help in voice prompting and UI and can improve your prompts if u want.

-CoFounder of HelpingAI and Webscout

[AnonymousCoderArtist]

Why did you completed this without letting me know? This is not good. I just wanted to know about voice prompting.

[dustinwloring1988]

@AnonymousCoderArtist sorry the issue was stale here are 2 pull requests to follow:
#281
#510

[thecodacus]

Will close this issue once the the feature is implemented.
@AnonymousCoderArtist you are welcome to contribute if you like. but also check the related PRs that are in queue if these are something you are thinking of adding

[AnonymousCoderArtist]

Thanks a lot for the time and support @thecodacus

[AnonymousCoderArtist]

I think we can add voice prompting as the normal javascript voice speech recognition is quite fast and usable. I would surely work on it. @thecodacus @dustinwloring1988 Whats your's opinion?

[thecodacus]

thanks, I am not sure how fast any external speech recognition library will be, as most of the stuff this project does is local to browser and very few things happens from server-side.
most modern browsers already has a speech recognition api, if we can leverage that it would be great.

[dustinwloring1988]

@AnonymousCoderArtist I have used whisper with transformers.js and yes it is near real time

[AnonymousCoderArtist]

@dustinwloring1988 @thecodacus I am talking about the inbuilt speech recognition of javascript. Its quite fast and accurate.

[thecodacus]

@dustinwloring1988 , ohh nice how much heavy it is for browser. can we run it on mobile devices ?

@AnonymousCoderArtist I don't think there is any built in speech recognition as far as I remember. I guess you are talking about browser API

[dustinwloring1988]

@AnonymousCoderArtist are you referring to the Android one?
@thecodacus it is lite weight and runs on mobile.
There GitHub repo has some examples projects you can even get a small llm running in js on a edge device.
It should be 'easy' to bring in. (I used the transformers library all the time in python so I was excited to see the transformers.js version)

[AnonymousCoderArtist]

Ywah lik see @thecodacus @dustinwloring1988 Here is what i am talking about.

const click_to_record = document.getElementById('click_to_record');
          const convert_text = document.getElementById('convert_text');
          const is_recording = document.getElementById('is_recording');
          const confidence_id = document.getElementById('confidence_id');
          const language_select = document.getElementById('language_select');
  
          click_to_record.addEventListener('click',function(){
              var speech = true;
              window.SpeechRecognition = window.webkitSpeechRecognition;
  
              const recognition = new SpeechRecognition();
              recognition.interimResults = true;
              recognition.lang = language_select.value; 
  
              recognition.addEventListener('start', () => {
                  is_recording.innerHTML = "Recording: True";
              });
  
              recognition.addEventListener('end', () => {
                  is_recording.innerHTML = "Recording: False";
              });
  
              recognition.addEventListener('result', e => {
                  const transcript = Array.from(e.results)
                      .map(result => result[0])
                      .map(result => result.transcript)
                      .join('');
  
                  convert_text.innerHTML = transcript;
                  console.log(transcript);
  
                  const confidence = Array.from(e.results)
                  .map(result => result[0])
                  .map(result => result.confidence)
                  .join('');
                  confidence_id.innerHTML = `Confidence: ${confidence}`;
                  console.log(confidence);
              });
  
              if (speech == true) {
                  recognition.start();
              }
          })

I am talkingabout this the inbuilt webkitspeechrecognition.

[thecodacus]

yes thats browser api. its only available in browser thats using webkit like chrome

[AnonymousCoderArtist]

yes its fast and free i think