Adding an accurate, multilingual Voice Search capability into your app
By now you must have configured and published your Assistant via the Slang Console and also maybe customized it as required. Congratulations! :) If you have not already done that, you can do so by following the instructions here.
Let's start coding!
For testing, we recommend using a physical device instead of an emulator because most emulators don't work well with microphones.
1. Configure the build system
The first step is to update the app's build system to include Slang's Retail Assistant SDK.
Add the Slang dependency to your Gradle files
Add the path to the Slang maven repository to your top-level Gradle file
# Add this to your top-level Gradle file
allprojects {
repositories {
…
maven { url "https://gitlab.com/api/v4/projects/25706948/packages/maven" }
}
}
Add the Slang Retails Assistant dependency to your app's Gradle file
# Add this to your app's Gradle file
dependencies {
…
implementation 'in.slanglabs.assistants:slang-retail-assistant:8.+'
}
Setting up via Cocoapods
Add the path to the Slang Cocoapod repository to your Podfile
# Add this to your podfile
source 'https://github.com/SlangLabs/cocoapod-specs'
source 'https://github.com/CocoaPods/Specs.git'
Add the dependency to SlangRetailAssistant in your Podfile
pod 'SlangRetailAssistant'
Add support for granting microphone permission
In iOS, the user must explicitly grant permission for an app to access the user’s data and resources. An app with the SlangRetailAssistant requires access to User’s device microphone for voice interactions.
To comply with this requirement, you must add NSMicrophoneUsageDescription key to the Info.plist file of your app and provide a message about why your app requires access to the microphone. The message will be displayed only when the SlangRetailAssistant needs to activate the microphone.
To add the key:
In the Xcode project, go to the Info tab.
In the Custom iOS Target Properties section, hover over any key in the list and click the plus icon to the right.
From the list, select Privacy - Microphone Usage Description.
In the Value field to the right, provide a description for the added key. This description will be displayed to the user when the app is launched.
For example: "We require microphone permission to enable the voice assistant platform"
Install the Slang Retail Assistant package
The next step is to install the required packages inside your code repo
yarn setup
If you use yarn for install packages, run the below command
This should ideally be done in the componentDidMount of your main app component
import SlangConvaTrigger, {SlangRetailAssistant} from'@slanglabs/slang-conva-react-native-retail-assistant';SlangRetailAssistant.initialize({ assistantId:'<assistant id>',// The Assistant ID from the console apiKey:'<API Key>',// The API key from the console})
This should ideally be done inside the main method.
import SlangRetailAssistant from'@slanglabs/slang-retail-assistant';awaitSlangRetailAssistant.init({ assistantID:'<assistant id>',// The Assistant ID from the console apiKey:'<API Key>',// The API key from the console})
If you're using Promise callbacks, then -
import SlangRetailAssistant from'@slanglabs/slang-retail-assistant';SlangRetailAssistant.init({ assistantID:'<assistant id>',// The Assistant ID from the console apiKey:'<API Key>',// The API key from the console}).then(() => {// other integration code...})
If you are using a framework like React, we recommend putting this code inside a useEffect block.
If you are building a simple HTML project, then put this code inside an document.addEventListener('DOMContentLoaded', () => {}) event listener.
2.2 Show Slang CONVA Trigger
Trigger refers to the UI element appearing on the screen, which the end user will click to bring up the Assistant.
The most common way to add Voice Search into apps, i.e. adding the trigger (which by default is a microphone icon) is either inside or next to the search bar.
Add the below XML element to your UI definition in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).
///1. Just set the custom class as `SlangConvaTrigger` for an UIImageView on Identify Inspector panel///2. Drag the element into the ViewController and you should see the line below.@IBOutletweakvar trigger: SlangConvaTrigger!
Programmatically
weakvar trigger: SlangConvaTrigger!overridefuncviewDidLoad() { super.viewDidLoad()/// Init the SlangConvaTrigger self.trigger =SlangConvaTrigger()/// Add the button to the view self.view.addSubview(self.trigger) self.trigger.translatesAutoresizingMaskIntoConstraints =false/// Align the trigger on the viewlet views = ["trigger": self.trigger!]let verticalButton = NSLayoutConstraint.constraints(withVisualFormat:"V:|-(>=0@299)-[trigger(64)]-40-|", options: NSLayoutConstraint.FormatOptions(), metrics:nil, views: views)let horizontalButton = NSLayoutConstraint.constraints(withVisualFormat:"H:|-(>=0@299)-[trigger(64)]-20-|", options: NSLayoutConstraint.FormatOptions(), metrics:nil, views: views) self.view.addConstraints(verticalButton + horizontalButton)}
Add the below element to your UI definition in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).
Incorporate the following component into your UI definition at the desired location to display the trigger (typically represented by a microphone icon), often positioned alongside the search bar.
Note:- There are other options to customize your trigger:-
1- icon => If you want to specify an icon instead of an image asset
2- iconColor => For specifying icon colour
2- backgroundColor => For specifying the background colour of the circle
Add the below HTML element to your HTML markup in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).
<slang-conva-trigger></slang-conva-trigger>
To specify a custom image for the microphone icon, please specify the src attribute in the trigger element.
src: A string representing the remote URL of the image.
Once the app has integrated the trigger, the next step is to register a callback to handle the voice search commands from the end user. CONVA will process the utterance spoken by the end user and if it detects that the user is trying to do a valid search operation, it will invoke the registered callback with the search string. Note that the search string will always be in English irrespective of which language the end user spoke it in. CONVA will automatically translate other language inputs into English.
SlangRetailAssistant.setOnSearchListener(newOnSearchListener() { @OverridepublicvoidonSearch(SearchInfo searchInfo,finalSearchUserJourney searchUserJourney) {String searchString =searchInfo.getSearchString();// Fire the actual search using the searchString } });
Or, utilize the onSearch callback provided by the SlangConvaTriggerView component.
SlangConvaTrigger convaTrigger =findViewById(R.id.your_id);convaTrigger.setOnSearchListener(newOnSearchListener() { @OverridepublicvoidonSearch(SearchInfo searchInfo,finalSearchUserJourney searchUserJourney) {String searchString =searchInfo.getSearchString();// Fire the actual search using the searchString } });
SlangRetailAssistant.onSearch = { (searchInfo, searchUserJourney) inlet searchString = searchInfo.searchString// Fire the actual search using the searchString...}
SlangRetailAssistant.setAction({onSearch: (searchInfo, searchUserJourney) => { String searchString = searchInfo["description"]// Fire the actual search using the search string }});
SlangRetailAssistant.setOnSearchListener((searchInfo, searchUserJourney) {String searchString = searchInfo.toString();// Fire the actual search using the searchString});