-
Notifications
You must be signed in to change notification settings - Fork 121
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Demonstrates Moonshine models running in the browser using onnxruntime-web
- Loading branch information
Showing
13 changed files
with
1,453 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| node_modules | ||
| public/moonshine/base/* | ||
| public/moonshine/tiny/* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # moonshine-web | ||
|
|
||
| This directory is a self-contained demo of the Moonshine models running directly on a user's device in a web browser using `onnxruntime-web`. You can try this demo out in our [HuggingFace space](https://huggingface.co/spaces/UsefulSensors/moonshine-web) or, alternatively, install and run it on your own device by following the instructions below. If you want to run Moonshine in the browser in your own projects, `src/moonshine.js` provides a bare-bones implementation of inferences using the ONNX models. | ||
|
|
||
| ## Installation | ||
|
|
||
| You must have Node.js (or another JavaScript toolkit like [Bun](https://bun.sh/)) installed to get started. Install [Node.js](https://nodejs.org/en) if you don't have it already. | ||
|
|
||
| Once you have your JavaScript toolkit installed, clone the `moonshine` repo and navigate to this directory: | ||
|
|
||
| ```shell | ||
| git clone [email protected]:usefulsensors/moonshine.git | ||
| cd moonshine/moonshine/demo/moonshine-web | ||
| ``` | ||
|
|
||
| Then install the project's dependencies: | ||
|
|
||
| ```shell | ||
| npm install | ||
| ``` | ||
|
|
||
| The demo expects the Moonshine Tiny and Base ONNX models to be available in `public/moonshine/tiny` and `public/moonshine/base`, respectively. To preserve space, they are not included here. However, we've included a helper script that you can run to conveniently download them from HuggingFace: | ||
|
|
||
| ```shell | ||
| npm run get-models | ||
| ``` | ||
|
|
||
| This project uses Vite for bundling and development. Run the following to start a development server and open the demo in your web browser: | ||
|
|
||
| ```shell | ||
| npm run dev | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| // Helper script for downloading Moonshine ONNX models from HuggingFace for local development. | ||
| import * as fs from 'fs'; | ||
| import * as hub from "@huggingface/hub"; | ||
|
|
||
| const repo = { type: "model", name: "UsefulSensors/moonshine" }; | ||
|
|
||
| var models = [ | ||
| "tiny", | ||
| "base" | ||
| ] | ||
|
|
||
| var layers = [ | ||
| "preprocess.onnx", | ||
| "encode.onnx", | ||
| "uncached_decode.onnx", | ||
| "cached_decode.onnx" | ||
| ] | ||
|
|
||
| console.log("Downloading Moonshine ONNX models from HuggingFace...") | ||
|
|
||
| models.forEach(model => { | ||
| var dir = "public/moonshine/" + model | ||
| if (!fs.existsSync(dir)){ | ||
| fs.mkdirSync(dir, { recursive: true }); | ||
| } | ||
| layers.forEach(layer => { | ||
| hub.downloadFile({ repo, path: "onnx/" + model + "/" + layer }).then((file) => { | ||
| file.arrayBuffer().then((buffer) => { | ||
| fs.writeFile(dir + "/" + layer, Buffer.from(buffer), () => { | ||
| console.log("\tDownloaded " + model + "/" + layer + " successfully.") | ||
| }); | ||
| }) | ||
| }) | ||
|
|
||
| }) | ||
| }); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| <!DOCTYPE html> | ||
| <html lang="en"> | ||
| <head> | ||
| <meta charset="UTF-8" /> | ||
| <link rel="icon" href="/favicon.png" /> | ||
| <link rel="stylesheet" href="/index.css"> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> | ||
| <title>Moonshine – lightweight ASR by Useful Sensors</title> | ||
| </head> | ||
| <body> | ||
| <div id="root"> | ||
| <div class="container"> | ||
| <div class="logo-container"> | ||
| <svg version="1.1" id="logo" xmlns="http://www.w3.org/2000/svg" x="0px" y="0px" viewBox="0 0 800 800" > | ||
| <path class="st0" d="M409,760C205.6,760,40,594.4,40,390.9C40,228.5,144.4,88.5,294.8,40C236.6,100.9,203,182.9,203,268.3 | ||
| c0,181.7,147.3,329.5,328.3,329.5c85.8,0,168.1-34.2,228.8-93.4C711.9,655.3,571.8,760,409,760L409,760z"/> | ||
| <line class="st1" id="l1" x1="310.1" y1="293.8" x2="310.1" y2="325.8"/> | ||
| <line class="st1" id="l2" x1="729.8" y1="293.8" x2="729.8" y2="325.8"/> | ||
| <line class="st1" id="l3" x1="370" y1="220" x2="370" y2="399.6"/> | ||
| <line class="st1" id="l4" x1="430" y1="245.9" x2="430" y2="373.7"/> | ||
| <line class="st1" id="l5" x1="489.9" y1="293.8" x2="489.9" y2="325.8"/> | ||
| <line class="st1" id="l6" x1="548.1" y1="278.2" x2="548.1" y2="342.1"/> | ||
| <line class="st1" id="l7" x1="609.9" y1="220.4" x2="609.9" y2="400"/> | ||
| <line class="st1" id="l8" x1="669.8" y1="245.9" x2="669.8" y2="373.7"/> | ||
| </svg> | ||
| <h1>Moonshine</h1> | ||
| fast, accurate, and lightweight speech-to-text models running in your browser | ||
| </div> | ||
| <div class="justify-center"> | ||
| <select name="models" id="models"> | ||
| <option value="tiny" selected>moonshine/tiny</option> | ||
| <option value="base">moonshine/base</option> | ||
| </select> | ||
| <input type="file" id="upload" style="display: none;" /> | ||
| <button id="browse" onclick="document.getElementById('upload').click();">Browse</button> | ||
| <button id="startRecord">Record</button> | ||
| <button id="stopRecord" style="display: none;">Stop</button> | ||
| </div> | ||
| <div id="audioPanel"> | ||
| <div class="justify-center"> | ||
| <audio id="sound" type="audio/wav" controls></audio> | ||
| </div> | ||
| <div class="justify-center"> | ||
| <button id="transcribe">Transcribe</button> | ||
| </div> | ||
| </div> | ||
| <div class="justify-center"> | ||
| <span id="transcription"></span> | ||
| </div> | ||
| </div> | ||
| </div> | ||
| <script type="module" src="/src/index.js"></script> | ||
| </body> | ||
| </html> |
Oops, something went wrong.