> For the complete documentation index, see [llms.txt](https://docs.conva.ai/conva.ai-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.conva.ai/conva.ai-docs/integrating-the-assistant/headless-mode.md).

# Headless Mode

Conva.AI supports SDKs for the following platforms

* Android
* Python
* iOS
* Flutter
* Typescript
* Rust - *Coming soon*

Let's walk through the integration process.

## Setting up the build environment

{% tabs %}
{% tab title="Android" %}
First, you need to add the *maven* repository URL to your project-level build.gradle file. Open the *build.gradle* file and add the following code inside the *allprojects > repositories* block

<pre class="language-gradle" data-overflow="wrap" data-full-width="false"><code class="lang-gradle">allprojects {
    repositories {
        maven {
            url "https://pkgs.dev.azure.com/slanglabs-convaai/prod/_packaging/public_prod/maven/v1"
        }
    }
<strong>}
</strong></code></pre>

Next, open your app-level *build.gradle* file and add the *conva-ai-core* dependency inside the dependencies block

```gradle
dependencies {
    implementation 'in.slanglabs.conva:conva-ai-core:1.0.2-beta'
}
```

{% endtab %}

{% tab title="Python" %}

```sh
$ pip install conva-ai
```

{% endtab %}

{% tab title="Flutter" %}

```sh
$ flutter pub add conva_ai_core
```

Once done, run the command 'dart pub get' and ensure Conva.AI is added to the dependencies.

```yaml
dependencies:
  conva_ai_core: ^0.1.3
```

{% hint style="info" %}
Check the latest version of SDK here:- <https://pub.dev/packages/conva_ai_core>
{% endhint %}
{% endtab %}

{% tab title="iOS" %}
**Add the Cocoapod repository path to your Podfile**

```
# Add this to your podfile

source 'https://github.com/CocoaPods/Specs.git'
```

Next, open your podspec file and add the ConvaAICore dependency.

```
s.dependency 'ConvaAICore', '1.0.3'
```

Once done, run the command 'pod install'&#x20;
{% endtab %}

{% tab title="Typescript" %}

```bash
npm install conva-ai
```

{% endtab %}
{% endtabs %}

## Initializing the Assistant

First, initialize Conva.AI using the credentials of the Assistant created via Magic Studio.

{% tabs %}
{% tab title="Android" %}

```kotlin
ConvaAI.init(
    id = "replace this string with your_assistant_id",
    key = "replace this string with your_api_key",
    version = "LATEST", // this is a special tag to indicate 
                        // the latest version of the Assistant
    application = applicationContext
);
```

{% endtab %}

{% tab title="Python" %}

```python
client = AsyncConvaAI(
    assistant_id="replace this string with your_assistant_id", 
    assistant_version="LATEST", # this is a special tag to indicate 
                                # the latest version of the Assistant
    api_key="replace this string with your_api_key"
)
```

{% endtab %}

{% tab title="Flutter" %}

<pre class="language-dart"><code class="lang-dart">ConvaAI.init(
<strong>    id: "your_assistant_id",
</strong>    key: "your_api_key", 
    version: "LATEST"
);
</code></pre>

{% endtab %}

{% tab title="iOS" %}

```swift
ConvaAI.initialize(
    with: "<your_assistant_id>", 
    key: "<your_api_key>", 
    version: "<your_assistant_version>"
)
```

{% endtab %}

{% tab title="Typescript" %}

```typescript
import ConvaAI from 'conva-ai';

const client = new ConvaAI({
    assistantID:"<YOUR_ASSISTANT_ID>",
    assistantVersion:"<YOUR_ASSISTANT_VERSION>",
    apiKey:"<YOUR_API_KEY>"});
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
Open the "Integration" tab of your Assistant to get the keys.
{% endhint %}

## Invoking a Agent

Next, pass unstructured text input to Conva.AI and let it invoke the appropriate Agent and return the response.

{% tabs %}
{% tab title="Android" %}

```kotlin
// Pass an input string to Conva.AI and let it determine the 
// relevant Agent to invoke and return the output params
// after that Agent is executed
val response = ConvaAI.invokeCapability(
    input = "book a bus ticket from bangalore to chennai tomorrow at 2pm",
)

// response = {
//    "capability_name" : "ticket_booking",
//    "message" : "Showing you buses from Bangalore to Chennai for tommorrow around 2pm"
//    "params" : {
//         "source": "BLR",
//         "destination" : "MAS",
//         "date" : "6/7/2024",
//         "time" : "14:00"
//         "mode" : "bus"
//     }
// }
```

{% endtab %}

{% tab title="Python" %}

```python
# Pass an input string to Conva.AI and let it determine the 
# relevant Capability to invoke and return the output params
# after that Capability is executed
response = await client.invoke_capability("book a bus ticket from Bangalore to chennai tommorrow at 2pm")
return response

# response = {
#    "capability_name" : "ticket_booking",
#    "message" : "Showing you buses from Bangalore to Chennai for tommorrow around 2pm"
#    "params" : {
#         "source": "BLR",
#         "destination" : "MAS",
#         "date" : "6/7/2024",
#         "time" : "14:00"
#         "mode" : "bus"
#     }
# }
```

{% endtab %}

{% tab title="Flutter" %}

```dart
// Pass an input string to Conva.AI and let it determine the 
// relevant Capability to invoke and return the output params
// after that Capability is executed
Response completion = await ConvaAI.invokeCapability(input: "<input_query>");
// completion = {
//    "capability_name" : "ticket_booking",
//    "message" : "Showing you buses from Bangalore to Chennai for tommorrow around 2pm"
//    "params" : {
//         "source": "BLR",
//         "destination" : "MAS",
//         "date" : "6/7/2024",
//         "time" : "14:00"
//         "mode" : "bus"
//     }
// }
```

**Note**

* Proper error handling should be implemented to manage any potential issues during the invocation.
  {% endtab %}

{% tab title="iOS" %}

<pre class="language-swift"><code class="lang-swift"><strong>// Pass an input string to Conva.AI and let it determine the 
</strong>// relevant Capability to invoke and return the output params
// after that Capability is executed.

// Invoke a capability (non-streaming)
let response: ConvaAICapability? = try await ConvaAI.invokeCapability(
    with: "&#x3C;input_query>"
)
</code></pre>

**Notes**

* Ensure that this method is called from an `async` context.
* Proper error handling should be implemented to manage any potential issues during the invocation.
  {% endtab %}

{% tab title="Typescript" %}

```typescript
// Pass an input string to Conva AI and let it determine the
// capability name and the respective parameters
client.invokeCapability({
    query: 'Hi, how are you doing today?',
    stream: false
  }).then(response => {
    console.log('ConvaAI Response:', response); 
    // TODO: Add application logic here
  }).catch(error => {
    console.error('Error:', error);
  });
  
//Example response:
//{inputQuery: 'Hi, how are you doing today?',
//message: "Hi there! I'm doing well, thank you for asking. How about you? How can I help you today?",
//messageType: 'question',
//parameters: { interaction_type: 'greeting' },
//relatedQueries: [
    //'Ask for a joke',
    //'Get fashion tips',
    //'Inquire about the latest trends'
 // ],
//toolName: 'small_talk'
//}
```

{% endtab %}
{% endtabs %}

## Response object

The response object contains the following key fields

* **capability\_name:** The name of the Agent that was triggered.
* **message:** The string containing either the answer to the original query or a status message. Agent creators can update this via Magic Studio.
* **related\_queries:** An array of strings with queries related to the current query and app context. Agent creators can update this field's description if needed.
* **params**: This object contains custom parameters specific to the invoked Agent. For example, if a Agent is created with `task_name`, `date`, and `time` (as shown in the image below), these fields will be in the params object.
* **history**: The current conversation history. This can be used by the app to continue the current conversation further (more on this later).

<figure><img src="/files/yYFCKVnsMTFvFUyY1Wpp" alt=""><figcaption><p>Agent with custom parameters</p></figcaption></figure>

{% hint style="info" %}
Developers should check if a specific custom parameters exists before accessing its value, as it might not be marked as required in Magic Studio.
{% endhint %}

## Invoking a specific Agent

To invoke a specific Agent, you can refer to it by name and pass the input string. You can use this in streaming (async) or non-streaming (sync) mode.&#x20;

{% tabs %}
{% tab title="Android" %}

```kotlin
// Invoke a specific Agent
val response = ConvaAI.invokeCapabilityWithName(
    input = "book a bus ticket from Bangalore to chennai tommorrow at 2pm", 
    capability = "ticket_booking"
);
```

{% endtab %}

{% tab title="Python" %}

```python
# Invoke a specific Capability
response = await client.invoke_capability_with_name("book a bus ticket from Bangalore to chennai tommorrow at 2pm", capability_name="ticket_booking")
return response
```

{% endtab %}

{% tab title="Flutter" %}

```dart
// Invoke a specific capability
Response completion = await ConvaAI.invokeCapabilityWithName(
    input: "book a bus ticket from Bangalore to chennai tommorrow at 2pm",
    capability: "ticket_booking"
);
```

**Note**

* Proper error handling should be implemented to manage any potential issues during the invocation.
  {% endtab %}

{% tab title="iOS" %}

```swift
// Invoke a capability with name (non-streaming)
let completion: ConvaAICapability? = try await ConvaAI.invokeCapability(
    with: <"input_query">,
    capabilityName: <"capability_name">
)
```

**Notes**

* Ensure that this method is called from an `async` context.
* Proper error handling should be implemented to manage any potential issues during the invocation.
  {% endtab %}

{% tab title="Typescript" %}

```typescript
client.invokeCapabilityName({
    query: 'What are the latest shopping trends?',
    stream: false,
    capabilityName: 'domain_faq'
  }).then(response => {
    console.log('ConvaAI Response:', response); 
    // TODO: Add application logic here
  }).catch(error => {
    console.error('Error:', error);
  });
  
//Example response:
//{inputQuery: 'What are the latest shopping trends?',
//message: 'The latest shopping trends include increased online shopping, personalized shopping experiences, and the use of AI in retail.',
//messageType: 'statement',
//parameters: { query: 'What are the latest shopping trends?' },
//relatedQueries: [
    //'Explore popular products this season',
    //'Learn about AI in retail',
    //'Discover personalized shopping experiences'
//],
  //toolName: 'domain_faq'}
```

{% endtab %}
{% endtabs %}

### Handling Streaming Response

Conva.AI supports a streaming version of these APIs. The response is sent back incrementally, with the listener invoked multiple times. Use the `is_final` flag in the Response object to check if all responses are received.

{% tabs %}
{% tab title="Android" %}

```kotlin
// Invoke a Agent asynchronously with a streaming response
ConvaAI.invokeCapability(
    input = "Hello, how are you?",
    listener = object : ResponseListener {
        override fun onResponse(response: Response, isFinal: Boolean) {
            // Check for is_final
            if (isFinal) {
                // Handle the final response
            }
        }
        override fun onError(error: Throwable) {
            // Handle the error
        }
    }
);
```

{% endtab %}

{% tab title="Python" %}

```python
# streaming version
response = await client.invoke_capability_stream(query)
out = ""
async for res in response:
    out = res
    
# At this point the "is_final" flag in the out object will be set
return out
```

{% endtab %}

{% tab title="Flutter" %}

```swift
// Invoke a capability asynchronously with a streaming response
Stream<Response> completionStream =
            ConvaAI.invokeCapabilityStream(input: "Hello, how are you?");
completionStream.listen(
      (event) {
      // Use the response object
  },
  onError: (error) {
    // Handle error
  },
);
```

{% endtab %}

{% tab title="iOS" %}

```swift
// Invoke a capability (streaming)
for try await response in ConvaAI.invokeCapabilityStream(
    with: "<input_query>"
) {
   // Handle the streaming response here
}
```

**Notes**

* Ensure that this method is called from an `async` context.
* Proper error handling should be implemented to manage any potential issues during the invocation.
  {% endtab %}
  {% endtabs %}

## Invoking a Agent Group

When creating Agents via Magic Studio, it places all Agents into the  "default" group.  But users can create their own groups and put specific Agents into it.  If you want to limit the Agent invocation to specific groups, you can pass the group name to the APIs.

{% tabs %}
{% tab title="Android" %}

```kotlin
// Invoke a Agent group synchronously (non-streaming)
val response = ConvaAI.invokeCapability(
    input = "Hello, how are you?",
    capabilityGroup = "default"
);

// Invoke a Agent group asynchronously to get a streaming response
ConvaAI.invokeCapability(
    input = "Hello, how are you?",
    capabilityGroup = "general_conversation",
    listener = object : ResponseListener {
        override fun onResponse(response: Response, isFinal: Boolean) {
            // Handle the response
        }
        override fun onError(error: Throwable) {
            // Handle the error
        }
    }
);
```

{% endtab %}

{% tab title="Python" %}

```python
# 'client' is the initialized ConvaAI object
# Streaming
response = await client.invoke_capability_stream(query, capability_group="default")
out = ""
async for res in response:
    out = res
return out

# Non-streaming
response = await client.invoke_capability(query, capability_group="default")
return response
```

{% endtab %}

{% tab title="Flutter" %}

```dart
// Invoke a capability group synchronously (non-streaming)
Response completion = await ConvaAI.invokeCapability(
    input: "Hello, how are you?",
    capabilityGroup: "default"
);

// Invoke a capability group asynchronously to get a streaming response
Stream<Response> completionStream =
ConvaAI.invokeCapabilityStream(
  input: "Hello, how are you?",
  capabilityGroup: "general_conversation"
);
completionStream.listen(
      (event) {
      // Use the response object
  },
  onError: (error) {
    // Handle error
  },
);
```

**Note**

* Proper error handling should be implemented to manage any potential issues during the invocation.
  {% endtab %}

{% tab title="iOS" %}

```swift
// Invoke a capability with name (non-streaming)
let response: ConvaAICapability? = try await ConvaAI.invokeCapability(
    with: <"input_query">,
    capabilityGroup: <"capability_group">
)
```

**Notes**

* Ensure that this method is called from an `async` context.
* Proper error handling should be implemented to manage any potential issues during the invocation.
  {% endtab %}
  {% endtabs %}

## Handling Conversational Context

Conva.AI supports maintaining context across different conversations, allowing the AI to build upon prior interactions for more relevant responses.

**Parameters**

* **`history`** (Optional): Available after the first response. Pass this from the previous response if you want to maintain conversational context in subsequent calls.
* **`capabilityContext`** (Optional): Context related to a specific Agent, refining AI responses for that particular Agent.

**Usage**

* On the first invocation, call the API without `history`.
* On subsequent invocations, pass the `history` from the previous response if you want to maintain the conversation's context.
* You can also pass the `capabilityGroup` or `capabilityName` along with the context to refine the AI’s responses further.

{% tabs %}
{% tab title="Android" %}

```kotlin
// First Response
val response1 = ConvaAI.invokeCapability(
    input = "Blue jeans",
);

// Second response. Pass the history from the first response
// to the second one to maintain conversational context
val reponse2 = ConvaAI.invokeCapability(
    input = "show me in red",
    context = ConvaAIContext(
        history = response1.history,
        capabilityContext = <"Map of String and Any">
    )
);
```

```
(Any): The values can only include the following types:
String: For textual data.
Int: For integer numbers.
Double: For floating-point numbers.
Boolean: For true/false values.
Array: For lists of items (e.g., Array<String>, Array<Int>, etc.).
Map: For nested maps, allowing for hierarchical data structures.
```

```kotlin
val items = listOf("green tea", "freshly squeezed juice", "herbal tea", "smoothies", "coconut water", "coffee")

// Example for passing capability context
val capabilityContext = mapOf(
    "product_search" to mapOf(
        "best beverages" to items
    )
)

// Note: Here <product_search> is the Agent name.
// Use your own Agent name and describe Agent data according to it

// Optional Context: CapabilityContext is optional.
// If omitted, the AI will function based on the default 
// or previously provided context.
```

{% endtab %}

{% tab title="Python" %}

```python
# First Response
response1 = await client.invoke_capability("blue jeans")
return response

# Second response. Pass the history from the first response
# to the second one to maintain conversational context
response2 = await client.invoke_capability("show me in red", 
 history=response1.conversation_history,
 capability_context={ <capability_name>: <knowledge> })
return response
```

```
(Knowledge): The values can only include the following types:
String: For textual data.
Int: For integer numbers.
Double: For floating-point numbers.
Boolean: For true/false values.
Array: For lists of items (e.g., Array<String>, Array<Int>, etc.).
Map: For nested maps, allowing for hierarchical data structures.
```

```python
items = ["green tea", "freshly squeezed juice", "herbal tea", "smoothies", "coconut water", "coffee"]

# Example for passing capability context
capabilityContext = {
    "product_search": {
        "best beverages": items
    }
}

# Note: Here <product_search> is the capability name. 
# Use your own capability name and describe capability data according to it
```

{% endtab %}

{% tab title="Flutter" %}

```dart
// First Response
var firstResponse = ConvaAI.invokeCapability(
  input: "Blue jeans",
);

// Second response. Pass the history from the first response
// to the second one to maintain conversational context
var response = ConvaAI.invokeCapability(
  input: "show me in red",
  context: ConvaAIContext(
    history: firstResponse.history,
    capabilityContext: <"Map of String and Any">
  ),
);
```

```
(Any): The values can only include the following types:
String: For textual data.
Int: For integer numbers.
Double: For floating-point numbers.
Boolean: For true/false values.
Array: For lists of items (e.g., Array<String>, Array<Int>, etc.).
Map: For nested maps, allowing for hierarchical data structures.
```

```dart
var items = ["green tea", "freshly squeezed juice", "herbal tea", "smoothies", "coconut water", "coffee"];

// Example for passing capability context
var capabilityContext = {
  "product_search": {
    "best beverages": items
  }
};

// Note: Here <product_search> is the capability name.
// Use your own capability name and describe capability data according to it

// Optional Context: CapabilityContext is optional.
// If omitted, the AI will function based on the default 
// or previously provided context.
```

{% endtab %}

{% tab title="iOS" %}

<pre class="language-swift"><code class="lang-swift"><strong>// First Response
</strong>let firstResponse: ConvaAICapability? = try await ConvaAI.invokeCapability(
    with: &#x3C;"input_query">
)

let response: ConvaAICapability? = try await ConvaAI.invokeCapability(
    with: &#x3C;"input_query">,
    context: ConvaAIContext(
        history: firstResponse.history,
        capabilityContext: &#x3C;"Map of String and Any">
    )
)
</code></pre>

```
(Any): The values can only include the following types:
String: For textual data.
Int: For integer numbers.
Double: For floating-point numbers.
Boolean: For true/false values.
Array: For lists of items (e.g., Array<String>, Array<Int>, etc.).
Map: For nested maps, allowing for hierarchical data structures.
```

```swift
let items = ["green tea", "freshly squeezed juice", "herbal tea", "smoothies", "coconut water", "coffee"]

// Example for passing capability context
capabilityContext: [
    "product_search": [
        "best beverages": items
    ]
]
// Note :- Here <product_search> is the capability name. 
// Use your own capability name and describe capability data according to it

// Optional Context: CapabilityContext is optional.
// If omitted, the AI will function based on the default 
// or previously provided context.
```

**Notes**

* Ensure that this method is called from an `async` context.
* Proper error handling should be implemented to manage any potential issues during the invocation.
  {% endtab %}

{% tab title="Typescript" %}

<pre class="language-typescript"><code class="lang-typescript">client.invokeCapability({
    query: 'What can I wear for my graduation party?',
    stream: false,
    history: conversationHistory
}).then(response => {
    console.log('ConvaAI Response (First Query):', response);
    
    if (response &#x26;&#x26; 'conversationHistory' in response) {
        conversationHistory = JSON.stringify(response.conversationHistory);
    }

    return client.invokeCapability({
        query: 'Can you tell me what you do?',
        stream: false,
        history: conversationHistory 
    });
}).then(response => {
    console.log('ConvaAI Response (Second Query):', response);
});

//ConvaAI Response (First Query): ConvaAIResponse {
<strong>//inputQuery: ‘What can I wear for my graduation party?’,
</strong>//message: ‘Searching for graduation party outfit ideas’,
//parameters: {
//search_term: ‘graduation party outfit’,
//category: ‘clothing’,
<strong>//filter_price_range: ‘’
</strong>//},
//relatedQueries: [
<strong>// ‘Explore formal dresses’, ‘Check out party accessories’, ‘View trending outfits for events’, ‘Browse new arrivals in clothing’, ‘Filter by color or style’ ],
</strong>
<strong>// “conversationHistory”: { “items”: [ { “user_input”: “What can I wear for my graduation party?”, “assistant_response”: { “thought”: “Let’s think about this step by step. The user is looking for outfit ideas for a graduation party”, “category”: “clothing”
</strong>// … } }
// … ] }
//toolName: ‘product_search’ }
//ConvaAI Response (Second Query): ConvaAIResponse {
//inputQuery: ‘Can you tell me what you do?’,
//message: “I am here to help you explore the latest fashion trends and find stylish clothing and beauty products on Myntra! How can I assist you today?”,
//parameters: { interaction_type: ‘greeting’ },
//relatedQueries: [
//‘Ask about the latest fashion trends’, ‘Inquire about beauty products’, ‘Get style advice’ ],

// “conversationHistory”: { “items”: [ { “user_input”: “Can you tell me what you do?”, “assistant_response”: { “thought”: “Let’s think about this step by step. The user is asking about my role and what I do. I should provide a friendly and informative response without going into task-oriented details.”,
<strong>// “message”: “I’m here to help you explore the latest fashion trends and find stylish clothing and beauty products on Myntra! How can I assist you today?”
</strong>// … } }
// … ] }
//toolName: ‘small_talk’ }
</code></pre>

{% endtab %}
{% endtabs %}

## Using the Copilot UI

The Copilot experience includes a well designed bottom sheet that appears on top of the app and provides the following elements:

* **Text box:** For users to type their queries.
* **Mic icon:** To trigger Conva.AI's highly accurate speech recognition feature&#x20;
* **Message Area:** Displays the message (the `message` field in the `Response` object)
* **Voice Feedback:** Speak back the message, can be optionally muted by the user
* **Mute icon:** To allow end-users to enable or disable the voice feedback
* **Feedback Icons:** Automatically appear after each response.&#x20;
* **Dynamic Suggestions:** Buttons showing related queries (the `related_queries` field in the `Response` object) or custom suggestions provided by the app via an API.  &#x20;

Here are the high level steps to use the Copilot:

* Setup the Copilot.
  * Theme the Copilot&#x20;
  * Register handlers for Agent handling and Suggestion handling
    * Agent Handling: Handle responses generated by the Copilot&#x20;
    * Suggestion Handling: Handle user interactions with suggestion buttons directed to the app
* Attach the Copilot to an Activity&#x20;
* Start the Copilot
* Handle the Agents and Suggestion clicks


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.conva.ai/conva.ai-docs/integrating-the-assistant/headless-mode.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.