Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve spec compliance, Android interim results, iOS fix for simultaneous speech synthesis. #114

Open
wants to merge 33 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
7f9cfcb
Removed EXTRA_CALLING_PACKAGE since it is not recommended for use by
wifisher May 15, 2018
6bf1c20
Updated the Android code to pass up the proper error code and message.
wifisher May 15, 2018
297ce6c
Changed the error code from the native side to be a number to match iOS.
wifisher May 15, 2018
19b1a0d
Rework results to more closely match spec and Chrome implementation.
wifisher May 16, 2018
5d0fb3e
Changed SpeechRecognitionResult to use isFinal.
wifisher May 16, 2018
25d658a
Added support for interim results on Android.
wifisher May 17, 2018
9d573b6
Corrected name of iSpeechSDK.h include.
wifisher May 17, 2018
d088a55
Added event types for result and error events.
wifisher May 25, 2018
fda708b
Commented out a log message.
wifisher May 25, 2018
bdfcdad
Added more speech recognition events for iOS.
wifisher May 29, 2018
c5166cb
Allow control over audio session category using config.xml preference…
wifisher May 30, 2018
bfeed27
Allow disable of iSpeech service using the API key.
wifisher May 30, 2018
adbb260
Updates to the readme and changelog.
wifisher May 30, 2018
05e2af0
Added information on SPEECH_RECOGNITION_USAGE_DESCRIPTION variable to
wifisher May 30, 2018
27c3abf
Added speechRecognitionAllowAudioOutput to the readme.
wifisher May 30, 2018
a42060a
Changed to use cordova-plugin-compat v1.2.0.
wifisher May 30, 2018
e7ec247
Improved error reporting for Android.
wifisher Jun 4, 2018
cd9abd1
More Android error handling. nomatch event uses results type.
wifisher Jun 4, 2018
a14d0e8
Limit to Cordova Android 6.3.0 or newer.
wifisher Nov 21, 2018
d49daeb
Moved the "end" event to occur after the results.
wifisher Nov 23, 2018
bef376d
Added support for "local" serviceURI for Android.
wifisher Nov 23, 2018
b320f8d
Improved error handling under Android.
wifisher Nov 30, 2018
4979c61
Removed a comma.
wifisher Dec 18, 2018
5b29f90
iOS: Now tracks route changes to allow the audioEngine to be re-initi…
wifisher Jan 21, 2019
93276ed
Removed support for the iSpeechSDK.
wifisher Jan 21, 2019
eb8ade8
Now handles addition/removal of a headset during recognition on iOS.
wifisher Jan 21, 2019
4bcfe68
Updated the readme and changelog.
wifisher Jan 21, 2019
d66eefb
Changed the usage descriptions to be variables in the plugin.xml file.
wifisher Feb 27, 2019
c56d18e
Changed back to preferences per the online docs.
wifisher Feb 27, 2019
7e33d0d
Monitor the audio category and update if it changes to something inco…
wifisher Mar 13, 2019
9e8c6bc
Fixed crash in iOS due to NaN from divide by zero due to no transcripts.
wifisher Mar 13, 2019
ffb985e
Updated gitignore and added .sln file.
wifisher Mar 20, 2019
a97a4a3
Fixed issue with double results on some Android devices.
wifisher Mar 23, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ local.properties
.project

.DS_Store
.vs
/node_modules/
72 changes: 72 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1 +1,73 @@
# Change Log

Update 2013/08/05
=================

Hi, you are all probably wondering where the code is after seeing my PhoneGap Day US presentation or reading the slides.
Well, I've been dealing with an illness in the family and have not had as much spare time as I would have hoped to update this project.
However, things are working out better than I could have hoped for and I should have time to concentrate on this very soon.

Update 2013/09/05
=================

Back to work on this but it's not ready yet so don't try to use.

Update 2015/04/04
=================

Basic example is working on iOS and android
```
<script type="text/javascript">
var recognition;
document.addEventListener('deviceready', onDeviceReady, false);

function onDeviceReady() {
recognition = new SpeechRecognition();
recognition.onresult = function(event) {
if (event.results.length > 0) {
q.value = event.results[0][0].transcript;
q.form.submit();
}
}
}
</script>
<form action="http://www.example.com/search">
<input type="search" id="q" name="q" size=60>
<input type="button" value="Click to Speak" onclick="recognition.start()">
</form>
```

Example from section 6.1 Speech Recognition Examples of the W3C page
(https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html#examples)

Update 2018/05/30
=================

Improved compatibility with the W3C specification (https://w3c.github.io/speech-api/webspeechapi.html) and other improvements.
* Result events are returned as `SpeechRecognitionResult` objects with the content matching the spec.
* Error events are returned as `SpeechRecognitionError` objects.
* Improved error reporting for Android.
* Added support for interim results for Android.
* Added support for more of the speech recognition events for iOS.
* Added support for the `config.xml` preference, `speechRecognitionAllowAudioOutput` to allow iOS apps to also output audio when using this plugin.
* Added ability to disable use of the iSpeech SDK for iOS using `speechRecognitionApiKey` preference.
* The `speechRecognitionApiKey` preference now replaces the original `apiKey` preference.


Update 2019/01/21
=================

* Improvements to error handling for Android
* Added support for "local" serviceURI for Android indicate a preference for offline (local) recognition.
* Fix to iOS support to allow for use of a Bluetooth Headset.
* Removed support for iSpeechSDK for iOS. Now requires iOS 10 or newer.

Update 2019/03/13
=================

* Fixed exception in iOS from divide by zero due to no transcripts being available.

Update 2020/03.23
=================

* Fixed issue with double results on some Android devices (LG G7 ThinQ for example; API 28)
138 changes: 90 additions & 48 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,12 @@
SpeechRecognitionPlugin
=======================

W3C Web Speech API - Speech Recognition plugin for PhoneGap
W3C Web Speech API - Speech Recognition plugin for Cordova/PhoneGap

Update 2013/09/05
=================
Basic Example
=============

Back to work on this but it's not ready yet so don't try to use.

Update 2013/08/05
=================

Hi, you are all probably wondering where the code is after seeing my PhoneGap Day US presentation or reading the slides. Well, I've been dealing with an illness in the family and have not has as much spare time as I would have hoped to update this project. However, things are working out better than I could have hoped for and I should have time to concentrate on this very soon.

Update 2015/04/04
=================

Basic example is working on iOS and android
Basic example is working on iOS, Android, and some Browsers.
```
<script type="text/javascript">
var recognition;
Expand All @@ -41,50 +31,102 @@ function onDeviceReady() {
Example from section 6.1 Speech Recognition Examples of the W3C page
(https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html#examples)

To install the plugin use
Installation
============

To install the plugin from Github, use

```
cordova plugin add https://github.com/macdonst/SpeechRecognitionPlugin
cordova plugin add https://github.com/wifisher/SpeechRecognitionPlugin --variable MICROPHONE_USAGE_DESCRIPTION="your usage message" --variable SPEECH_RECOGNITION_USAGE_DESCRIPTION="your usage message"
```

Since iOS 10 it's mandatory to add a `NSMicrophoneUsageDescription` in the info.plist to access the microphone.


To add this entry you can pass the `MICROPHONE_USAGE_DESCRIPTION` variable on plugin install.

iOS Quirks
==========

Example:
By default, this plugin sets up the app to only allow for audio input.
As such, after the first recognition request, the app will no longer allow for audio output.

`cordova plugin add https://github.com/macdonst/SpeechRecognitionPlugin --variable MICROPHONE_USAGE_DESCRIPTION="your usage message"`
If you need to also support audio output, for example with the `phonegap-plugin-speech-synthesis` plugin,
enable audio output with the `speechRecognitionAllowAudioOutput` preference in the `config.xml` file:
```
<preference name="speechRecognitionAllowAudioOutput" value="true" />
```

If the variable is not provided it will use an empty message, but a usage description string is mandatory to submit your app to the Apple Store.

iOS 10 and Newer
----------------

On iOS 10 and greater it uses the native SFSpeechRecognizer (same as Siri).

Supported locales for SFSpeechRecognizer are:
ro-RO, en-IN, he-IL, tr-TR, en-NZ, sv-SE, fr-BE, it-CH, de-CH, pl-PL, pt-PT, uk-UA, fi-FI, vi-VN, ar-SA, zh-TW, es-ES, en-GB, yue-CN, th-TH, en-ID, ja-JP, en-SA, en-AE, da-DK, fr-FR, sk-SK, de-AT, ms-MY, hu-HU, ca-ES, ko-KR, fr-CH, nb-NO, en-AU, el-GR, ru-RU, zh-CN, en-US, en-IE, nl-BE, es-CO, pt-BR, es-US, hr-HR, fr-CA, zh-HK, es-MX, id-ID, it-IT, nl-NL, cs-CZ, en-ZA, es-CL, en-PH, en-CA, en-SG, de-DE
Since iOS 10 it's mandatory to add `NSMicrophoneUsageDescription` and `NSSpeechRecognitionUsageDescription` in the info.plist to access the microphone and speech recognition.
To add this entry you can pass the `MICROPHONE_USAGE_DESCRIPTION` and `SPEECH_RECOGNITION_USAGE_DESCRIPTION` variables on plugin install.

Two-character codes can be used too.
If the variable is not provided it will use an empty message, but a usage description string is mandatory to submit your app to the Apple Store.

On iOS 9 and older it uses iSpeech SDK, an API key is required, get one on https://www.ispeech.org/, it's free.
To provide the key, add this preference inside the config.xml
```
<preference name="apiKey" value="yourApiKeyHere" />
```
If none is provided it will use the demo key "developerdemokeydeveloperdemokey"

iSpeech supported languages are:

English (Canada) (en-CA)
English (United States) (en-US)
Spanish (Spain) (es-ES)
French (France) (fr-FR)
Italian (Italy) (it-IT)
Polish (Poland) (pl-PL)
Portuguese (Portugal) (pt-PT)


Two-character codes can be used too, but for English, "en" will use "en-US"


Supported locales for SFSpeechRecognizer are:
* ar-SA
* ca-ES
* cs-CZ
* da-DK
* de-AT
* de-CH
* de-DE
* el-GR
* en-AE
* en-AU
* en-CA
* en-GB
* en-ID
* en-IE
* en-IN
* en-NZ
* en-PH
* en-SA
* en-SG
* en-US
* en-ZA
* es-CL
* es-CO
* es-ES
* es-MX
* es-US
* fi-FI
* fr-BE
* fr-CA
* fr-CH
* fr-FR
* he-IL
* hr-HR
* hu-HU
* id-ID
* it-CH
* it-IT
* ja-JP
* ko-KR
* ms-MY
* nb-NO
* nl-BE
* nl-NL
* pl-PL
* pt-BR
* pt-PT
* ro-RO
* ru-RU
* sk-SK
* sv-SE
* th-TH
* tr-TR
* uk-UA
* vi-VN
* yue-CN
* zh-CN
* zh-HK
* zh-TW
* possibly others

Two-character codes can be used as well.

iOS 9 and Older
---------------

On iOS 9 and older are not supported.
35 changes: 35 additions & 0 deletions SpeechRecognition.sln
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@

Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Express 2012 for Web
Project("{E24C65DC-7377-472B-9ABA-BC803B73C61A}") = "SpeechRecognition", ".", "{66E24856-A60A-4CEC-A98C-7FAE2D74279F}"
ProjectSection(WebsiteProperties) = preProject
TargetFrameworkMoniker = ".NETFramework,Version%3Dv4.0"
Debug.AspNetCompiler.VirtualPath = "/SpeechRecognition"
Debug.AspNetCompiler.PhysicalPath = "..\SpeechRecognition\"
Debug.AspNetCompiler.TargetPath = "PrecompiledWeb\SpeechRecognition\"
Debug.AspNetCompiler.Updateable = "true"
Debug.AspNetCompiler.ForceOverwrite = "true"
Debug.AspNetCompiler.FixedNames = "false"
Debug.AspNetCompiler.Debug = "True"
Release.AspNetCompiler.VirtualPath = "/SpeechRecognition"
Release.AspNetCompiler.PhysicalPath = "..\SpeechRecognition\"
Release.AspNetCompiler.TargetPath = "PrecompiledWeb\SpeechRecognition\"
Release.AspNetCompiler.Updateable = "true"
Release.AspNetCompiler.ForceOverwrite = "true"
Release.AspNetCompiler.FixedNames = "false"
Release.AspNetCompiler.Debug = "False"
VWDPort = "50678"
EndProjectSection
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{66E24856-A60A-4CEC-A98C-7FAE2D74279F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{66E24856-A60A-4CEC-A98C-7FAE2D74279F}.Debug|Any CPU.Build.0 = Debug|Any CPU
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
16 changes: 11 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,19 @@
{
"name": "phonegap-plugin-speech-recognition",
"version": "0.3.0",
"version": "0.3.4",
"description": "Cordova Speech Recognition Plugin",
"cordova": {
"id": "phonegap-plugin-speech-recognition",
"platforms": ["android", "ios"]
"platforms": [ "android", "browser", "ios" ]
},
"engines": {
"cordovaDependencies": {
"0.3.0": { "cordova-android": ">6.3.0" }
}
},
"repository": {
"type": "git",
"url": "git+https://github.com/macdonst/SpeechRecognitionPlugin.git"
"url": "git+https://github.com/wifisher/SpeechRecognitionPlugin.git"
},
"keywords": [
"cordova",
Expand All @@ -19,9 +24,10 @@
"cordova-ios"
],
"author": "Simon MacDonald",
"contributors": ["Wayne Fisher <[email protected]"],
"license": "MIT",
"bugs": {
"url": "https://github.com/macdonst/SpeechRecognitionPlugin/issues"
"url": "https://github.com/wifisher/SpeechRecognitionPlugin/issues"
},
"homepage": "https://github.com/macdonst/SpeechRecognitionPlugin#readme"
"homepage": "https://github.com/wifisher/SpeechRecognitionPlugin#readme"
}
26 changes: 9 additions & 17 deletions plugin.xml
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
<?xml version="1.0" encoding="UTF-8"?>
<plugin
xmlns="http://www.phonegap.com/ns/plugins/1.0"
xmlns:android="http://schemas.android.com/apk/res/android" id="phonegap-plugin-speech-recognition" version="0.3.0">
xmlns:android="http://schemas.android.com/apk/res/android"
id="phonegap-plugin-speech-recognition"
version="0.3.4">
<name>SpeechRecognition</name>
<description>Cordova Speech Recognition Plugin</description>
<license>MIT</license>
<keywords>cordova,speech,recognition</keywords>
<dependency id="cordova-plugin-compat" version="^1.0.0" />
<!-- android -->
<engines>
<engine name="cordova-android" version=">=6.3.0" />
</engines>
<!-- android -->
<platform name="android">
<js-module src="www/SpeechRecognition.js" name="SpeechRecognition">
<clobbers target="SpeechRecognition" />
Expand Down Expand Up @@ -75,25 +79,13 @@
</feature>
</config-file>
<source-file src="src/ios/SpeechRecognition.m" />
<source-file src="src/ios/libiSpeechSDK.a" framework="true" />
<header-file src="src/ios/SpeechRecognition.h" />
<header-file src="src/ios/Headers/iSpeechSDK.h" />
<header-file src="src/ios/Headers/ISSpeechRecognition.h" />
<header-file src="src/ios/Headers/ISSpeechRecognitionLocales.h" />
<header-file src="src/ios/Headers/ISSpeechRecognitionResult.h" />
<header-file src="src/ios/Headers/ISSpeechSynthesis.h" />
<header-file src="src/ios/Headers/ISSpeechSynthesisVoices.h" />
<framework src="AudioToolbox.framework" />
<framework src="SystemConfiguration.framework" />
<framework src="Security.framework" />
<framework src="CFNetwork.framework" />
<framework src="Speech.framework" weak="true" />
<resource-file src="src/ios/iSpeechSDK.bundle" />
<preference name="MICROPHONE_USAGE_DESCRIPTION" default=" " />
<preference name="MICROPHONE_USAGE_DESCRIPTION" default="Allow for voice recognition." />
<config-file target="*-Info.plist" parent="NSMicrophoneUsageDescription">
<string>$MICROPHONE_USAGE_DESCRIPTION</string>
</config-file>
<preference name="SPEECH_RECOGNITION_USAGE_DESCRIPTION" default=" " />
<preference name="SPEECH_RECOGNITION_USAGE_DESCRIPTION" default="Allow you to provide voice commands to the app." />
<config-file target="*-Info.plist" parent="NSSpeechRecognitionUsageDescription">
<string>$SPEECH_RECOGNITION_USAGE_DESCRIPTION</string>
</config-file>
Expand Down
Loading