# Copilot Mode

Conva.AI Copilot is available for the following platforms:

* Android
* iOS - *Coming soon*
* Flutter (Android only for now)

## Add Dependencies

To use the Copilot, you need to first add the dependencies in your project.

{% tabs %}
{% tab title="Android" %}
First, you need to add the `maven` repository URL to your project-level `build.gradle` file. Open the `build.gradle` file and add the following code inside the `allprojects` > `repositories` block.

```groovy
allprojects {
    repositories {
        maven {
            url "https://pkgs.dev.azure.com/slanglabs-convaai/prod/_packaging/public_prod/maven/v1"
        }
    }
}
```

Next, open your app-level `build.gradle` file and add the`conva-ai-copilot` dependency.

```groovy
dependencies {
    implementation 'in.slanglabs.conva:conva-ai-copilot:1.0.2-beta'
}
```

{% endtab %}

{% tab title="Flutter" %}

```sh
$ flutter pub add conva_ai_copilot
```

Once done, run the command `dart pub get` and ensure Conva.AI Copilot is added to the dependencies.

```yaml
dependencies:
  conva_ai_copilot: ^0.1.3
```

{% hint style="info" %}
Check the latest version of SDK here :- <https://pub.dev/packages/conva_ai_copilot>
{% endhint %}
{% endtab %}
{% endtabs %}

## Setup the Copilot

To setup the Copilot, follow these steps:

1. **Initialize and Setup**:
   * Initialize the ConvaAI SDK.
   * Setup the Copilot using the `setup()` API, passing in a `ConvaAIOptions` object to configure the assistant.
2. **Register Handlers for Agents**:
   * Handle user interactions when they speak to the Copilot.
3. **Register Handlers for Suggestions**:
   * **Assistant-Handled Suggestions**: Suggestions that generate follow-on queries. Clicking these does not invoke the suggestion handler.
   * **App-Handled Suggestions**: Suggestions meant for app-specific actions, such as initiating a search within the app. Clicking these triggers the suggestion handler.

{% tabs %}
{% tab title="Android" %}

```kotlin
// Initialize the ConvaAI SDK
ConvaAI.init(
    id = "assistant_id",
    key = "api_key",
    version = "LATEST",
    application = applicationContext
);
val options = ConvaAIOptions.Builder()
    .setCapabilityHandler(object : ConvaAIHandler {
        override fun onCapability(
            response: ConvaAIResponse, 
            interactionData: ConvaAIInteraction, 
            isFinal: Boolean) {
            // Handle the response from the assistant
        }
    })
    .setSuggestionHandler(object : ConvaAISuggestionHandler {
        override fun onSuggestion(suggestion: ConvaAISuggestion) {
            // Handle the selected suggestion
        }
    })
    .build()
    
ConvaAICopilot.setup(options)
```

{% endtab %}

{% tab title="Flutter" %}

<pre class="language-dart"><code class="lang-dart">ConvaAI.init(
    id: "your_assistant_id",
    key: "your_api_key", 
    version: "your_api_key"
);
<strong>var options = ConvaAIOptions()
</strong>    ..onCapability = (response, interactionData, isFinal) {
       // Handle the response from the assistant
    }
     ..onSuggestion = (suggestion) {
       // Handle the selected suggestion
     };

  ConvaAICopilot.setup(options);
</code></pre>

{% endtab %}
{% endtabs %}

## Attach the Copilot to UI

To prepare the Copilot UI, you need to call the `attach` API of the `ConvaAICopilot`. This step readies the Copilot to appear on top of the current screen but remains invisible until you call `startConversation()`.

{% hint style="info" %}
You only need to call `attach()` once for the initial setup. The Copilot UI will track screen transitions and continue to persist unless dismissed by the user or the app.
{% endhint %}

{% tabs %}
{% tab title="Android" %}

```kotlin
// Attach the Copilot to the current activity
ConvaAICopilot.attach(getActivity())
```

{% endtab %}

{% tab title="Flutter" %}

```dart
ConvaAICopilot.attach();
```

{% endtab %}
{% endtabs %}

## Start the Copilot

To bring up the Copilot experience, call the `startConversation()` method of the `ConvaAICopilot`. This displays the Copilot on top of the current screen, allowing the user to interact with it.

{% tabs %}
{% tab title="Android" %}

```kotlin
ConvaAICopilot.startConversation()
```

{% endtab %}

{% tab title="Flutter" %}

```dart
ConvaAICopilot.startConversation();
```

{% endtab %}
{% endtabs %}

## Handle Agent Responses

When the Copilot is visible and the user either types or speaks to it, the Copilot sends the request to the Conva.AI backend orchestration service. This service determines the appropriate Agent to invoke and returns the response to the SDK.

To handle the Agent, you need to register a handler, as shown in the setup step above.

{% tabs %}
{% tab title="Android" %}

```kotlin
val options = ConvaAIOptions.Builder()
    .setCapabilityHandler(object : ConvaAIHandler {
        override fun onCapability(
            response: ConvaAIResponse, 
            interactionData: ConvaAIInteraction, 
            isFinal: Boolean) {
            // Handle the response from the assistant
        }
    })
    .build()
```

{% endtab %}

{% tab title="Flutter" %}

```dart
var options = ConvaAIOptions()
  ..onCapability = (response, interactionData, isFinal) {
       // Handle the response from the assistant
   };
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
When using the Copilot, the handler receives streaming responses, meaning it will be called multiple times until the `isFinal` parameter is set to `true`.
{% endhint %}

### ConvaAIResponse object

The `ConvaAIResponse` object contains three main fields:

* **message**: Contains either the status message or the answer to the question. This string is displayed in the message area and spoken aloud (unless muted by the user).
* **related\_queries**: Controls the list of suggestions that appear after each user query.
* **params**: A `Map` containing the custom parameters configured in the Agent blueprint.

### ConvaAIInteraction object

The `ConvaAIInteraction` object contains data relevant to the current interaction. By default it contains an array of `Suggestion` objects which are populated with the `related_queries` available in the `ConvaAIResponse` object.

Inside the `onCapability` callback, the app can use the `ConvaAIInteraction` instance for more advanced functionality. This allows the app to customize the message displayed to the user, as well as set the suggestions that should be shown on the surface.

{% tabs %}
{% tab title="Android" %}

```kotlin
val options = ConvaAIOptions.Builder()
    .setCapabilityHandler(object : ConvaAIHandler {
        override fun onCapability(response: ConvaAIResponse, interactionData: ConvaAIInteraction, isFinal: Boolean) {
            // Update the message that is to be displayed to the user
            interactionData.setMessage("Display this message to the user")
        
            // Update the list of suggestions that are shown to the user
            // NOTE: These suggestions would always be handled
            // by the app via the ConvaAISuggestionAction instance provided
            // to the copilot during setup()
            interactionData.setSuggestionStrings(mutableListOf("Potato", "Tomato", "Onion"))
        
            // Set the suggestion index that should be highlighted on the copilot surface UI, here we set it to 0
            interactionData.setSelectedSuggestionIndex(0)
        }
    })
    // other builder method calls
    .build()
);
```

{% endtab %}

{% tab title="Flutter" %}

```dart
var options = ConvaAIOptions()
  ..onCapability = (response, interactionData, isFinal) {

    // Update the list of suggestions that are shown to the user.
    // [suggestionStrings] - A list of strings representing suggestions.
    interactionData.setSuggestionStrings(
      ["Potato", "Tomato", "Onion"], 
      response.capability
    );

    // Set the suggestion index that should be highlighted on the copilot surface UI.
    // [selectedIndex] - The index of the selected suggestion (default is 0 in this example).
    interactionData.setSuggestionStrings(
      ["Potato", "Tomato", "Onion"], 
      response.capability, 
      selectedIndex: 0
    );

    // Specify the target for the suggestions.
    // [suggestionTarget] - Defines whether the suggestions should be handled by the app or the assistant:
    //   - SuggestionTarget.app: Indicates that the suggestions are for local app search or actions.
    //   - SuggestionTarget.assistant: Indicates that the suggestions are derived from an AI model or LLM search.
    interactionData.setSuggestionStrings(
      ["Potato", "Tomato", "Onion"], 
      response.capability, 
      selectedIndex: 0, 
      suggestionTarget: SuggestionTarget.app // This is for local app search.
    );
  };

```

{% endtab %}
{% endtabs %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.conva.ai/conva.ai-docs/integrating-the-assistant/copilot-mode.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.