Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration with Open AI #18

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
OPENAI_API_KEY=
OPENAI_ENDPOINT=
29 changes: 21 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,24 +11,37 @@ View the demo on [Vimeo](https://vimeo.com/215612852/)
This is how this web app works:

1. Using the Web Speech API’s `SpeechRecognition` interface to listen your voice from a microphone
2. Send your message to [API.ai](https://api.ai) (the natural language processing platform) as a text string
3. Once the AI from the API.ai returns the reply text back, use the `SpeechSynthesis` interface to give it a synthetic voice.


2. Send your message to [azure openai](https://learn.microsoft.com/en-us/azure/ai-services/openai/) (the natural language processing platform) as a text string
3. Once the AI from the azure openai returns the reply text back, use the `SpeechSynthesis` interface to give it a synthetic voice.


### Try It on Your Own Server

Rename the `.env.local` to `.env` and fill the env vars:

```
APIAI_TOKEN=
APIAI_SESSION_ID=some_unique_session_id
OPENAI_API_KEY=
OPENAI_ENDPOINT=https://<your openai endpoint>/openai/deployments/<your deployment name>/chat/completions?api-version=<api version>
```

- You can follow this MicroSoft page to generate key and endpoint
- https://learn.microsoft.com/en-us/azure/ai-services/openai/chatgpt-quickstart?tabs=command-line%2Cpython-new&pivots=rest-api

You just need to fill out the env vars with the API key and a OPENAI_ENDPOINT. No need to create an `.env` file.


## run the program

```text
npm install
npm start
```

The first one is an API.ai API key (Please get one by sign up with [API.ai](https://api.ai)), and the second one is a session ID, which is an arbitrary string (we could make this unique within the app, but that's beyond the scope of this demo).
once started, you can access url at
```text
http://127.0.0.1:5000/
```

Or use this Heroku button to deploy to Heroku server. You just need to fill out the env vars with the API key and a session ID. No need to create an `.env` file.

[![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy?template=https://github.com/girliemac/web-speech-ai)

Expand Down
50 changes: 32 additions & 18 deletions index.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
'use strict';

require('dotenv').config()
const APIAI_TOKEN = process.env.APIAI_TOKEN;
const APIAI_SESSION_ID = process.env.APIAI_SESSION_ID;
const OPENAI_API_KEY = process.env.OPENAI_API_KEY;
const OPENAI_ENDPOINT = process.env.OPENAI_ENDPOINT; // URL of your Azure OpenAI endpoint

const express = require('express');
const app = express();
Expand All @@ -19,34 +19,48 @@ io.on('connection', function(socket){
console.log('a user connected');
});

const apiai = require('apiai')(APIAI_TOKEN);
const axios = require('axios'); // for making HTTP requests

// Web UI
app.get('/', (req, res) => {
res.sendFile('index.html');
});

io.on('connection', function(socket) {
socket.on('chat message', (text) => {
socket.on('chat message', async(text) => {
console.log('Message: ' + text);

// Get a reply from API.ai
const headers = {
"Content-Type": "application/json",
"api-key": OPENAI_API_KEY

let apiaiReq = apiai.textRequest(text, {
sessionId: APIAI_SESSION_ID
});
};

apiaiReq.on('response', (response) => {
let aiText = response.result.fulfillment.speech;
console.log('Bot reply: ' + aiText);
socket.emit('bot reply', aiText);
});
const data = {
"messages": [
{"role": "user", "content": text},
{"role": "user", "content": "STOP_HERE"}
],
"max_tokens": 80,
"temperature": 0.7,
"frequency_penalty": 0,
"presence_penalty": 0,
"top_p": 0.95,
"stop": "STOP_HERE"
};

apiaiReq.on('error', (error) => {
console.log(error);
await axios.post(OPENAI_ENDPOINT, data, { headers: headers })
.then(response => {
if (response.status === 200) { // Check for success
const generatedText = response.data.choices[0].message.content.trim();
console.log(generatedText);
socket.emit('bot reply', generatedText);
} else {
console.error(`Error: API call failed with status ${response.status}`);
}
})
.catch(error => {
console.error("An error occurred:", error);
});

apiaiReq.end();

});
});