CONVA Search

Adding an accurate, multilingual Voice Search capability into your app

By now you must have configured and published your Assistant via the Slang Console and also maybe customized it as required. Congratulations! :) If you have not already done that, you can do so by following the instructions here.

Let's start coding!

For testing, we recommend using a physical device instead of an emulator because most emulators don't work well with microphones.

1. Configure the build system

The first step is to update the app's build system to include Slang's Retail Assistant SDK.

Add the Slang dependency to your Gradle files

Add the path to the Slang maven repository to your top-level Gradle file

# Add this to your top-level Gradle file

allprojects {  
    repositories {    
        …    
        maven { url "https://gitlab.com/api/v4/projects/25706948/packages/maven" }  
    }
}

Add the Slang Retails Assistant dependency to your app's Gradle file

# Add this to your app's Gradle file

dependencies {  
    …   
    implementation 'in.slanglabs.assistants:slang-retail-assistant:8.+'
}

Setting up via Cocoapods

Add the path to the Slang Cocoapod repository to your Podfile

# Add this to your podfile

source 'https://github.com/SlangLabs/cocoapod-specs'
source 'https://github.com/CocoaPods/Specs.git'

Add the dependency to SlangRetailAssistant in your Podfile

pod 'SlangRetailAssistant'

Add support for granting microphone permission In iOS, the user must explicitly grant permission for an app to access the user’s data and resources. An app with the SlangRetailAssistant requires access to User’s device microphone for voice interactions.

To comply with this requirement, you must add NSMicrophoneUsageDescription key to the Info.plist file of your app and provide a message about why your app requires access to the microphone. The message will be displayed only when the SlangRetailAssistant needs to activate the microphone.

To add the key:

In the Xcode project, go to the Info tab.
In the Custom iOS Target Properties section, hover over any key in the list and click the plus icon to the right.
From the list, select Privacy - Microphone Usage Description.
In the Value field to the right, provide a description for the added key. This description will be displayed to the user when the app is launched. For example: "We require microphone permission to enable the voice assistant platform"

Install the Slang Retail Assistant package

The next step is to install the required packages inside your code repo

yarn setup

If you use yarn for install packages, run the below command

yarn add @slanglabs/slang-conva-react-native-retail-assistant

npm setup

If you use npm for managing your packages, run the below command

npm install @slanglabs/slang-conva-react-native-retail-assistant --save

Because Slang uses native libraries, you need to link the package to your codebase to run the automatic linking steps

react-native link @slanglabs/slang-conva-react-native-retail-assistant

Try out the playground app for developers to understand the assistant.

Install the Slang Retail Assistant package

Run the below command to install the required packages inside your code repo.

 $ flutter pub add slang_retail_assistant

Once done, run the command 'dart pub get' and ensure Slang assistant is added to the dependencies.


dependencies:
  slang_retail_assistant: ^8.0.0

Next is to import Slang Retail Assistant in your dart code.

import 'package:slang_retail_assistant/slang_retail_assistant.dart';

Try out the playground app for developers to understand the assistant.

Using Slang Retail Assistant Package

You can add our NPM package to your project using any of your preferred package manager -

yarn

$ yarn add @slanglabs/slang-retail-assistant@8.6.0

npm

$ npm install @slanglabs/slang-retail-assistant@8.6.0 --save

If you are building a simple HTML project, you can still use our SDK using the script below -

<script type="text/javascript" src="https://unpkg.com/@slanglabs/slang-retail-assistant@8.6.0/index-iife.js" defer></script>

2. Code integration

2.1 Initialization

The next step is to initialize the SDK with the keys you obtained after creating the Assistant in the Slang console.

The recommendation is to perform the initialization in the onCreate method of the main Activity where the search bar will be visible.

// Your main activity class 

protected void onCreate(Bundle savedInstance) {  
    ...
    AssistantConfiguration configuration = new AssistantConfiguration.Builder()    
        .setAPIKey(<API Key>)    
        .setAssistantId(<AssistantId>)
        .build();  
    SlangRetailAssistant.initialize(this, configuration);
}

The recommendation is to perform the initialization in the viewDidLoad method of the main ViewController where the search bar will be visible.

import SlangRetailAssistant

override func viewDidLoad() {
    super.viewDidLoad()
    ...
    let config = AssistantConfiguration.Builder()
        .setAPIKey("<APIKey>")
        .setAssistantId("<AssistantId")
        .build()
    SlangRetailAssistant.initialize(with: config)
}

This should ideally be done in the componentDidMount of your main app component

import SlangConvaTrigger, {SlangRetailAssistant} from '@slanglabs/slang-conva-react-native-retail-assistant';

SlangRetailAssistant.initialize({
    assistantId: '<assistant id>',        // The Assistant ID from the console
    apiKey: '<API Key>',                  // The API key from the console
})

This should ideally be done inside the main method.

import 'package:slang_retail_assistant/slang_retail_assistant.dart';

var assistantConfig = new AssistantConfiguration()
 ..assistantId = "<AssistantId>"
 ..apiKey = "<APIKey>"
 ..enableCustomTrigger = true;

SlangRetailAssistant.initialize(assistantConfig);

If you're using async/await, then -

import SlangRetailAssistant from '@slanglabs/slang-retail-assistant';

await SlangRetailAssistant.init({
    assistantID: '<assistant id>',        // The Assistant ID from the console
    apiKey: '<API Key>',                  // The API key from the console
})

If you're using Promise callbacks, then -

import SlangRetailAssistant from '@slanglabs/slang-retail-assistant';

SlangRetailAssistant.init({
    assistantID: '<assistant id>',        // The Assistant ID from the console
    apiKey: '<API Key>',                  // The API key from the console
}).then(() => {
    // other integration code...
})

If you are using a framework like React, we recommend putting this code inside a useEffect block. If you are building a simple HTML project, then put this code inside an document.addEventListener('DOMContentLoaded', () => {}) event listener.

2.2 Show Slang CONVA Trigger

Trigger refers to the UI element appearing on the screen, which the end user will click to bring up the Assistant.

The most common way to add Voice Search into apps, i.e. adding the trigger (which by default is a microphone icon) is either inside or next to the search bar.

Add the below XML element to your UI definition in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).

<in.slanglabs.assistants.retail.SlangConvaTrigger
  android:id="@+id/your_id"
  android:layout_width="45dp"
  android:layout_height="45dp"
/>

Via Storyboard/Xib

///1. Just set the custom class as `SlangConvaTrigger` for an UIImageView on Identify Inspector panel
///2. Drag the element into the ViewController and you should see the line below.

@IBOutlet weak var trigger: SlangConvaTrigger!

Programmatically

weak var trigger: SlangConvaTrigger!

override func viewDidLoad() {
    super.viewDidLoad()
 
    ///  Init the SlangConvaTrigger
    self.trigger = SlangConvaTrigger()

    /// Add the button to the view
    self.view.addSubview(self.trigger)
    self.trigger.translatesAutoresizingMaskIntoConstraints = false

    /// Align the trigger on the view
    let views = ["trigger" : self.trigger!]
    let verticalButton = NSLayoutConstraint.constraints(withVisualFormat: "V:|-(>=0@299)-[trigger(64)]-40-|", options: NSLayoutConstraint.FormatOptions(), metrics: nil, views: views)
    let horizontalButton = NSLayoutConstraint.constraints(withVisualFormat: "H:|-(>=0@299)-[trigger(64)]-20-|", options: NSLayoutConstraint.FormatOptions(), metrics: nil, views: views)
    self.view.addConstraints(verticalButton + horizontalButton)   
}

Add the below element to your UI definition in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).

const styles = StyleSheet.create({
  triggerStyle: {
    width: 45,
    height: 45,
  },
});

<SlangConvaTrigger   
  style={styles.triggerStyle}
/>

To specify a custom image for the microphone icon, please specify the src field in the trigger element.

src: A string representing the remote URL of the image.

Example :

<SlangConvaTrigger
  src={{uri: Image.resolveAssetSource(require('../Images/trigger.png')).uri}}
  style={styles.triggerStyle}
/>

Incorporate the following component into your UI definition at the desired location to display the trigger (typically represented by a microphone icon), often positioned alongside the search bar.

Container(
       height: 45,
       width: 45,
       child:SlangConvaTrigger()
)

To specify a custom image for the microphone icon, please specify the imageAsset field in the trigger element.

imageAsset:- A string representing the asset path of the image.

Container(
       height: 45,
       width: 45,
       child:SlangConvaTrigger(imageAsset: AssetImage ('<path to your custom image>'))
)

To specify the circular background

Container(
       height: 45,
       width: 45,
       child:SlangConvaTrigger(enableCircularBackground: true)
)

Note:- There are other options to customize your trigger:-

1- icon => If you want to specify an icon instead of an image asset

2- iconColor => For specifying icon colour

2- backgroundColor => For specifying the background colour of the circle

Add the below HTML element to your HTML markup in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).

<slang-conva-trigger></slang-conva-trigger>

To specify a custom image for the microphone icon, please specify the src attribute in the trigger element.

src: A string representing the remote URL of the image.

Example:

<slang-conva-trigger 
    src="/images/custom-img.png"
></slang-conva-trigger>

If you are using typescript, then you might get the following error -

Property 'slang-conva-trigger' does not exist on type 'JSX.IntrinsicElements'.

To resolve this, create a declaration.d.ts file and add the following to this new file -

declare namespace JSX {
  interface IntrinsicElements {
    "slang-conva-trigger": any;
  }
}

2.3 Handler for Voice Search

Once the app has integrated the trigger, the next step is to register a callback to handle the voice search commands from the end user. CONVA will process the utterance spoken by the end user and if it detects that the user is trying to do a valid search operation, it will invoke the registered callback with the search string. Note that the search string will always be in English irrespective of which language the end user spoke it in. CONVA will automatically translate other language inputs into English.

SlangRetailAssistant.setOnSearchListener(
    new OnSearchListener() {
        @Override
        public void onSearch(SearchInfo searchInfo, final SearchUserJourney searchUserJourney) {
            String searchString = searchInfo.getSearchString();
            // Fire the actual search using the searchString
        }
    }
);

Or, utilize the onSearch callback provided by the SlangConvaTriggerView component.

SlangConvaTrigger convaTrigger = findViewById(R.id.your_id);

convaTrigger.setOnSearchListener(
    new OnSearchListener() {
        @Override
        public void onSearch(SearchInfo searchInfo, final SearchUserJourney searchUserJourney) {
            String searchString = searchInfo.getSearchString();
            // Fire the actual search using the searchString
        }
    }
);

SlangRetailAssistant.onSearch = { (searchInfo, searchUserJourney) in
    let searchString = searchInfo.searchString
    // Fire the actual search using the searchString
    ...
}

SlangRetailAssistant.setAction({
    onSearch: (searchInfo, searchUserJourney) => { 
         String searchString = searchInfo["description"]
         // Fire the actual search using the search string      
    }
});

SlangRetailAssistant.setOnSearchListener((searchInfo, searchUserJourney) {
      String searchString = searchInfo.toString();
      // Fire the actual search using the searchString
});

SlangRetailAssistant.setOnSearchListener({
    onSearch(searchInfo, searchUserJourney) {
        const searchString = searchInfo.item.description;
    }
})

PreviousCode Integration

NextAdvanced Topics

Last updated 2 years ago