Skip to content

Commit

Permalink
Adds Initial Commit
Browse files Browse the repository at this point in the history
Adds dialogflow-sample-voice-application to nexmo-community
  • Loading branch information
tchancse authored and amdcavallaro committed Jul 16, 2021
0 parents commit e64eb12
Show file tree
Hide file tree
Showing 8 changed files with 2,536 additions and 0 deletions.
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
API_KEY=xxxxxxxx
API_SECRET=xxxxxxxxxxxx
APP_ID=xxxxxxxxxxxxxxxxxxxxxxxxxx
PRIVATE_KEY_FILE=./private.key
SERVICE_NUMBER=xxxxxxxxxxx
DF_CONNECTING_SERVER=xxxxxxxxxxxxxx
229 changes: 229 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,229 @@
Apache License

Version 2.0, January 2004

http://www.apache.org/licenses/ TERMS
AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

1. Definitions.




"License" shall mean the terms and conditions for use, reproduction, and
distribution as defined by Sections 1 through 9 of this document.




"Licensor" shall mean the copyright owner or entity authorized by the copyright
owner that is granting the License.



"Legal Entity" shall mean the
union of the acting entity and all other entities that control, are controlled
by, or are under common control with that entity. For the purposes of this
definition, "control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or otherwise, or (ii)
ownership of fifty percent (50%) or more of the outstanding shares, or (iii)
beneficial ownership of such entity.



"You" (or "Your") shall mean
an individual or Legal Entity exercising permissions granted by this License.




"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation source, and
configuration files.



"Object" form shall mean any form resulting
from mechanical transformation or translation of a Source form, including but not
limited to compiled object code, generated documentation, and conversions to
other media types.



"Work" shall mean the work of authorship,
whether in Source or Object form, made available under the License, as indicated
by a copyright notice that is included in or attached to the work (an example is
provided in the Appendix below).



"Derivative Works" shall mean any
work, whether in Source or Object form, that is based on (or derived from) the
Work and for which the editorial revisions, annotations, elaborations, or other
modifications represent, as a whole, an original work of authorship. For the
purposes of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of, the Work
and Derivative Works thereof.



"Contribution" shall mean any work
of authorship, including the original version of the Work and any modifications
or additions to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner or by an
individual or Legal Entity authorized to submit on behalf of the copyright owner.
For the purposes of this definition, "submitted" means any form of electronic,
verbal, or written communication sent to the Licensor or its representatives,
including but not limited to communication on electronic mailing lists, source
code control systems, and issue tracking systems that are managed by, or on
behalf of, the Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise designated in
writing by the copyright owner as "Not a Contribution."




"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of
whom a Contribution has been received by Licensor and subsequently incorporated
within the Work.

2. Grant of Copyright License. Subject to the terms and
conditions of this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license
to reproduce, prepare Derivative Works of, publicly display, publicly perform,
sublicense, and distribute the Work and such Derivative Works in Source or Object
form.

3. Grant of Patent License. Subject to the terms and conditions of this
License, each Contributor hereby grants to You a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this
section) patent license to make, have made, use, offer to sell, sell, import, and
otherwise transfer the Work, where such license applies only to those patent
claims licensable by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s) with the Work to
which such Contribution(s) was submitted. If You institute patent litigation
against any entity (including a cross-claim or counterclaim in a lawsuit)
alleging that the Work or a Contribution incorporated within the Work constitutes
direct or contributory patent infringement, then any patent licenses granted to
You under this License for that Work shall terminate as of the date such
litigation is filed.

4. Redistribution. You may reproduce and distribute
copies of the Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You meet the following
conditions:

(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and

(b) You must cause any
modified files to carry prominent notices stating that You changed the files;
and

(c) You must retain, in the Source form of any Derivative Works that
You distribute, all copyright, patent, trademark, and attribution notices from
the Source form of the Work, excluding those notices that do not pertain to any
part of the Derivative Works; and

(d) If the Work includes a "NOTICE" text
file as part of its distribution, then any Derivative Works that You distribute
must include a readable copy of the attribution notices contained within such
NOTICE file, excluding those notices that do not pertain to any part of the
Derivative Works, in at least one of the following places: within a NOTICE text
file distributed as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or, within a display
generated by the Derivative Works, if and wherever such third-party notices
normally appear. The contents of the NOTICE file are for informational purposes
only and do not modify the License. You may add Your own attribution notices
within Derivative Works that You distribute, alongside or as an addendum to the
NOTICE text from the Work, provided that such additional attribution notices
cannot be construed as modifying the License.

You may add Your own
copyright statement to Your modifications and may provide additional or different
license terms and conditions for use, reproduction, or distribution of Your
modifications, or for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with the conditions
stated in this License.

5. Submission of Contributions. Unless You explicitly
state otherwise, any Contribution intentionally submitted for inclusion in the
Work by You to the Licensor shall be under the terms and conditions of this
License, without any additional terms or conditions. Notwithstanding the above,
nothing herein shall supersede or modify the terms of any separate license
agreement you may have executed with Licensor regarding such Contributions.


6. Trademarks. This License does not grant permission to use the trade names,
trademarks, service marks, or product names of the Licensor, except as required
for reasonable and customary use in describing the origin of the Work and
reproducing the content of the NOTICE file.

7. Disclaimer of Warranty. Unless
required by applicable law or agreed to in writing, Licensor provides the Work
(and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including,
without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT,
MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible
for determining the appropriateness of using or redistributing the Work and
assume any risks associated with Your exercise of permissions under this
License.

8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise, unless required
by applicable law (such as deliberate and grossly negligent acts) or agreed to in
writing, shall any Contributor be liable to You for damages, including any
direct, indirect, special, incidental, or consequential damages of any character
arising as a result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill, work stoppage,
computer failure or malfunction, or any and all other commercial damages or
losses), even if such Contributor has been advised of the possibility of such
damages.

9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer, and charge a fee
for, acceptance of support, warranty, indemnity, or other liability obligations
and/or rights consistent with this License. However, in accepting such
obligations, You may act only on Your own behalf and on Your sole responsibility,
not on behalf of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability incurred by, or
claims asserted against, such Contributor by reason of your accepting any such
warranty or additional liability. END OF TERMS AND CONDITIONS

APPENDIX: How to
apply the Apache License to your work.

To apply the Apache License to your work,
attach the following boilerplate notice, with the fields enclosed by brackets
"[]" replaced with your own identifying information. (Don't include the
brackets!) The text should be enclosed in the appropriate comment syntax for the
file format. We also recommend that a file or class name and description of
purpose be included on the same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright [yyyy] Tony Chan

Licensed
under the Apache License, Version 2.0 (the "License");

you may not use this file
except in compliance with the License.

You may obtain a copy of the License
at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law
or agreed to in writing, software

distributed under the License is distributed
on an "AS IS" BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
express or implied.

See the License for the specific language governing
permissions and

limitations under the License.
1 change: 1 addition & 0 deletions Procfile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
web: node ./df-application.js
151 changes: 151 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Vonage API - Google Dialogflow integration sample application

This sample application allows you to call a phone number to interact with a Google Dialogflow agent using Vonage Voice API, including getting real time transcripts and sentiment analysis.

This application uses a Dialogflow reference connection code (more details below) for the actual 2-way audio interaction with the Dialogflow agent.

## About this sample application

This sample application makes use of Vonage Voice API to answer incoming voice calls and set up a WebSocket connection to stream audio to and from the Dialogflow reference connection for each call.

The Dialogflow reference connection code will:
- Send audio to the Dialogflow agent from caller's speech,
- Stream audio responses from the Dialogflow agent to the caller via the WebSocket,
- Post back in real time transcripts and caller's speech sentiment scores via webhooks call back to this Voice API sample application.

Once this application will be running, you call in to the **`phone number linked`** to your application (as explained below) to interact via voice with your Dialogflow agent.</br>

## Set up the Dialogflow reference connection code - Host server public hostname and port

First set up a Dialogflow reference connection code from the [dialogflow-reference-connection](https://github.com/nexmo-community/dialogflow-reference-connection).

Default local (not public!) reference connection code `port` is: 5000.

If you plan to test using `Local deployment` with ngrok (Internet tunneling service) for both the Dialogflow reference connection code and this sample application, you may set up [multiple ngrok tunnels](https://ngrok.com/docs#multiple-tunnels).

For the next steps, you will need:
- The Dialogflow reference connection code server's public hostname and if necessary public port,</br>
e.g. `xxxxxxxx.ngrok.io`, `xxxxxxxx.herokuapp.com`, `myserver.mycompany.com:32000` (as **`DF_CONNECTING_SERVER`**, no `port` is necessary with ngrok or heroku as public hostname)

## Set up your Vonage Voice API application credentials and phone number

[Log in to your](https://dashboard.nexmo.com/sign-in) or [sign up for a](https://dashboard.nexmo.com/sign-up) Vonage APIs account.

Go to [Your applications](https://dashboard.nexmo.com/applications), access an existing application or [+ Create a new application](https://dashboard.nexmo.com/applications/new).

Under Capabilities section (click on [Edit] if you do not see this section):

Enable Voice
- Under Answer URL, leave HTTP GET, and enter https://\<host\>:\<port\>/answer (replace \<host\> and \<port\> with the public host name and if necessary public port of the server where this sample application is running)</br>
- Under Event URL, **select** HTTP POST, and enter https://\<host\>:\<port\>/event (replace \<host\> and \<port\> with the public host name and if necessary public port of the server where this sample application is running)</br>
Note: If you are using ngrok for this sample application, the answer URL and event URL look like:</br>
https://yyyyyyyy.ngrok.io/answer</br>
https://yyyyyyyy.ngrok.io/event</br>
- Click on [Generate public and private key] if you did not yet create or want new ones, save the private.key file in this application folder.</br>
**IMPORTANT**: Do not forget to click on [Save changes] at the bottom of the screen if you have created a new key set.</br>
- Link a phone number to this application if none has been linked to the application.

Please take note of your **application ID** and the **linked phone number** (as they are needed in the very next section.)

For the next steps, you will need:</br>
- Your [Vonage API key](https://dashboard.nexmo.com/settings) (as **`API_KEY`**)</br>
- Your [Vonage API secret](https://dashboard.nexmo.com/settings), not signature secret, (as **`API_SECRET`**)</br>
- Your `application ID` (as **`APP_ID`**),</br>
- The **`phone number linked`** to your application (as **`SERVICE_NUMBER`**), your phone will **call that number**,</br>
- The Dialogflow reference connection code server public hostname and port (as **`DF_CONNECTING_SERVER`**)</br>

## Overview on how this sample Voice API application works

- On an incoming call to the **`phone number linked`** to your application, GET `/answer` route plays a TTS greeting to the caller ("action": "talk"), then start a WebSocket connection to the Lex reference connection ("action": "connect"),
- Once the WebSocket is established (GET `/ws_event` with status "answered"), it plays a TTS greeting to Lex bot, as Lex expects the user to speak first, we need to start the conversation as one would do in a phone call, with the answerer greeting the caller. The result is that the caller will immediately hear the Dialogflow agent initial greeting (e.g. "How may I help you?") without having to say anything yet.
You can customize that inital TTS played to Dialogflow to correspond to your Dialogflow agent programming and use case.
- Transcripts and sentiment scores will be received by this application in real time,</br>
- When the caller hangs up, both phone call leg and WebSocket leg will be automatically terminated.

## Running Dialogflow sample Voice API application

You may select one of the following 2 types of deployments.

### Local deployment

To run your own instance of this sample application locally, you'll need an up-to-date version of Node.js (we tested with version 14.3.0).

Download this sample application code to a local folder, then go to that folder.

Copy the `.env.example` file over to a new file called `.env`:
```bash
cp .env.example .env
```

Edit `.env` file, and set the five parameter values:</br>
API_KEY=</br>
API_SECRET=</br>
APP_ID=</br>
SERVICE_NUMBER=</br>
DF_CONNECTING_SERVER=</br>


Install dependencies once:
```bash
npm install
```

Launch the applicatiom:
```bash
node df-application
```

### Command Line Heroku deployment

You must first have deployed your application locally, as explained in previous section, and verified it is working.

Install [git](https://git-scm.com/downloads).

Install [Heroku command line](https://devcenter.heroku.com/categories/command-line) and login to your Heroku account.

If you do not yet have a local git repository, create one:</br>
```bash
git init
git add .
git commit -am "initial"
```

Start by creating this application on Heroku from the command line using the Heroku CLI:

```bash
heroku create myappname
```

Note: In above command, replace "myappname" with a unique name on the whole Heroku platform.

On your Heroku dashboard where your application page is shown, click on `Settings` button,
add the following `Config Vars` and set them with their respective values:</br>
API_KEY</br>
API_SECRET</br>
APP_ID</br>
SERVICE_NUMBER</br>
DF_CONNECTING_SERVER</br>

add also the parameter PRIVATE_KEY_FILE with the value ./private.key</br>

Now, deploy the application:


```bash
git push heroku master
```

On your Heroku dashboard where your application page is shown, click on `Open App` button, that hostname is the one to be used under your corresponding [Vonage Voice API application Capabilities](https://dashboard.nexmo.com/applications) (click on your application, then [Edit]).</br>

For example, the respective links would be like:</br>
https://myappname.herokuapp.com/answer</br>
https://myappname.herokuapp.com/event</br>

See more details in above section **Set up your Vonage Voice API application credentials and phone number**.






From any phone, dial the Vonage number (the one in the .env file). This will connect to the DialogFlow agent (as specified in the .env file), and you will be able to have voice interaction with it.
Loading

0 comments on commit e64eb12

Please sign in to comment.