Conva.AI Overview

This gives a quick overview of the core concepts behind Conva.AI and what it can be used for

Overview

Conva.AI is a versatile low-code platform for creating, maintaining, and integrating AI Assistants using Meta Prompts. This full-stack platform provides everything needed to seamlessly add AI Assistants to your apps.

Conva.AI Assistants are software components that process unstructured text inputs to generate structured outputs, triggering actions within the app. These inputs can come from users or the app itself. The actions can be:

Visual Actions: Navigate screens, display search results, or show AI-generated content.
Direct Actions: Provide responses through speech or text.
Combined Actions: Mix of visual and direct responses.

Assistants can appear as interactive visual elements (e.g., a Conversational Overlay) or work behind the scenes, with outputs integrated into the app's existing screens.

Conva.AI simplifies the creation, integration, and maintenance of Assistants by breaking them into Agents. Each Agent is a fundamental unit of an Assistant and acts like an AI function, processing input strings into well-defined JSON outputs.

There are two key steps in the journey of a Conva.AI Assistant

Assistant Creation: Using Meta Prompts to define Agents.
Assistant Integration: Easily integrating these Agents into your app.

More about Agents later.

Components of Conva.AI

It comprises of three core components:

Magic Studio
Conva.AI SDK
Conva.AI orchestration service

Magic Studio

A web-based platform, where developers can create, test, update and maintain AI Assistants and their Agents.

Magic Studio offers:

Offers a simple way for developers (or rather PMs) to define the scope and specification (the output) of each Agents using Meta prompts.
Pipeline management for compiling Agents (which generates the system prompts).
A playground to test the Assistant or specific Agents.
A log viewer
Customization options to control the Copilot (more on this later) and Inline experience of the Assistant.

Conva.AI SDK

Platform-specific SDKs that offer simple but flexible ways for app developers to integrate the Assistant into their app. It comes in two variants:

Core SDK

Allows developers to trigger Agents and process responses, suitable for headless integration or custom UI creation. This is available in streaming and non-streaming variants.

// Pass an input string to Conva.AI and let it determine the 
// relevant Agents to invoke and return the output params
// after that Agent is executed
val response = ConvaAI.invokeCapability(
    input = "book a bus ticket from bangalore to chennai tomorrow at 2pm",
)

// response = {
//    "capability_name" : "ticket_booking",
//    "message" : "Showing you buses from Bangalore to Chennai for tommorrow around 2pm"
//    "params" : {
//         "source": "BLR",
//         "destination" : "MAS",
//         "date" : "6/7/2024",
//         "time" : "14:00"
//         "mode" : "bus"
//     }
// }

# Pass an input string to Conva.AI and let it determine the 
# relevant Capability to invoke and return the output params
# after that Agent is executed
response = await client.invoke_capability("book a bus ticket from Bangalore to chennai tommorrow at 2pm", capability_name=capability_name)
return response

# response = {
#    "capability_name" : "ticket_booking",
#    "message" : "Showing you buses from Bangalore to Chennai for tommorrow around 2pm"
#    "params" : {
#         "source": "BLR",
#         "destination" : "MAS",
#         "date" : "6/7/2024",
#         "time" : "14:00"
#         "mode" : "bus"
#     }
# }

// Pass an input string to Conva.AI and let it determine the 
// relevant Capability to invoke and return the output params
// after that Agent is executed
Response completion = await ConvaAI.invokeCapability(input: "<input_query>");

Copilot SDK

Provides an intuitive Conversational Overlay UI with integrated ASR and TTS, supported only for mobile platforms like Android and iOS. The Copilot UI interacts directly with end-users and notifies the app upon receiving responses from Agents.

It also provides APIs to theme the UI and control various elements and also get appropriate notifications when the user interacts with various elements in the UI.

This is supported only for mobile platforms.

// Register handlers for the Copilot
val options = ConvaAIOptions.Builder()
    .setCapabilityHandler(object : ConvaAIAction {
        override fun onCapability(
            response: ConvaAIResponse, 
            interactionData: ConvaAIInteraction, 
            isFinal: Boolean
        ) {
            // Handle the response from the assistant
        }
    })
    .build()
ConvaAICopilot.setup(options)

// Start the Copilot
ConvaAICopilot.startConversation()

Conva.AI orchestration service

The orchestration service operates within Conva.AI's cloud, communicating with the SDK to manage and route queries effectively.

Key functions

Context Maintenance: Rewrites the incoming query to maintain context with the previous queries, eg. rewrites the query "show me in red" to "show me red shirts" if the previous query was "blue shirts".
Agent Routing: Routes the request to the right Agent, for example,
- Given a travel assistant with two Agents called "Flight Ticket Booking" and "Train Ticket booking", for a query "book a flight ticket from Bangalore to Chennai", the router will pick "Flight Ticket booking" Agent to process this input
- If the input query is unrelated to the context of the app, then it will route it to a special built-in Agent called "Unsupported"
Runtime Data Augmentation: Enhances Agent prompts with Retrieval Augmented Generation (RAG).
LLM Selection: Chooses the relevant Large Language Model (LLM) for executing the Agent.
Response Processing: Processes and returns responses from the selected Agent to the SDK.
Semantic Caching: Implements a semantic cache to improve speed and efficiency.

Agents

The role of an AI Assistant is to convert unstructured input into structured outputs that the app can use to perform actions. This process is managed by the Agents configured for the Assistant.

Assistant Creators can create a Agent by writing a short description of its intended function in Magic Studio.

Magic Studio generates a "Agent Blueprint" based on the outline, which is a structured specification of the Agent.

The blueprint contains the following sections:

Agent Properties

This section includes details for compiling the prompts for the Agent. It contains the following fields:

Display Name - Display name of this Agent as shown in Studio.
Name - Name of this Agent when used in code during integration.
Description - Brief description of the Agent.
Agent Topics: Phrases that inform the LLM about the various topics the Agent can handle.
Groups - Agent Group the Agent belongs to. Users can add it to an existing group or create a new one.
Example Utterances - Some sample utterances that can be provided as input to the Agent. These are auto generated and can be edited by the user.

Agent Parameters

The goal of every Agent is to generate a set of well-formed output parameters, that will be consumed by the app via the Conva.AI SDK.

Magic Studio automatically generates these parameters from the Agent outline, though users can also specify parameters explicitly.

Creators can edit this specification directly or by editing the outline and regenerating the Agent blueprint.

Conva.AI currently supports only the following data types for the output parameters

String
Integer
Boolean
Float
Array
Object

Required fields

Sometimes Conva.AI (or rather than underlying LLM) might not generate the expected parameter even if the input contains the required details. To force the parameter to be generated, you can set that parameter to be "Required".

Another use of the "Required" parameter is to get Conva.AI to prompt the user to provide some missing input, eg. if a "date" field is required for booking a ticket, but the user hasn't provided it, Conva.AI will automatically ask for the date.

Default Parameters

Every Agent by default comes with two default parameters:

message: Contains the answer to the user query or a status message (wherein the answer is inside other parameters)
related_queries: Contains AI-suggested follow-on queries based on the original query.

When the Assistant is used in Copilot mode (details further below), this field populates the suggestion chips by default. Developers can override the suggestion chips with other fields if needed.

Agent Groups

Multiple Agents can be grouped together as a logical unit and given a name. This is called Agent Group.

The Conva.AI router will restrict its choice of Agents to those within the selected group.

This is useful in scenarios where only specific Agents are relevant based on the app context. For example, in a travel app, there might be two Agents: "Flight FAQ" and "Bus FAQ". To avoid conflicting data, you can ensure that only the relevant Agent is active based on whether the user is in the flight booking or bus booking section of the app.

By default all Agents are added to the built-in group called "default".

// Limit the Agent to invoked to a specific group
val response = ConvaAI.invokeCapability(
    input = "Hello, how are you?",
    capabilityGroup = "general_conversation"
)

# Limit the Capability to invoked to a specific group
response = await client.invoke_capability(query, capability_group="general_conversation")
return response

Agents patterns

Agents provide a flexible and easy way to define various use cases. Here are some common patterns for building Agents:

Classification - Classify the given input into one or more buckets. E.g. Navigation Agents ("show my account statement")
Extraction - Extract relevant attributes from the given input. E.g. Search Agents ("show me buses from Bangalore to Chennai for 1st December" -> {"source": "Bangalore", "destination": "Chennai", "date": "1/12/2024")
Transformation - Convert extracted attributes into a different output. E.g. Replace locations into code ("Bangalore to Chennai tomorrow" -> {"source":"BLR", "destination": "MAA", "date":"10/10/2024")
Generation - Generate new content based on the given input ("What is your return policy?" -> "Amazon.in allows you to return your products within 15 days...")

And of course its possible to do a combination of these patterns. A single Agent can have different parameters that are classified, extracted, transformed and even generated.

Agent Data sources

When generating an output, Conva.AI typically relies on the underlying LLM's internal knowledge, guided by the Agent outline and blueprint. However, apps may have internal data source that can help the LLM to enhance the output accuracy. Conva.AI allows Agent creators to add this data source to their Agents.

Data sources can be added to Agents by either creating a new data source or importing an existing data source. Creators can give it a name and describe it.

Data source types

HTML URL: Provides the content of the URL as context to the LLM.
CSV/TSV: Provides static data in CSV/TSV format to the LLM.
Text Content: Text blob that can be passed to the LLM.
PDF: Static data extracted from PDF file. Note: Non-text content aren't supported for now.

Parameter grounding

A unique aspect of the data based grounding option provided by Conva.AI is the ability for the creator to limit it to specific parameters. By default the data source group will impact any or all parameters (based on the input and the context).

Data Grounding Policies

When providing data sources, creators can set one of three grounding Policies

Strict: Conva.AI relies solely on the provided data sources when the Agent is invoked.
Best Effort: Conva.AI uses the data source when possible but falls back on the underlying LLM's internal knowledge if necessary.

Agent Compilation

Once Agents are created, they are in an abstracted form that is easy for Assistant Creators to understand but not yet in a format the underlying LLM can consume. The Magic Studio Compiler converts these into prompts optimized for the LLM.

Magic Studio Compiler is an AI agent that is built using a collaborative agent architecture. It comes with 3 smaller agents

Prompt Compiler - This uses the Agent blueprint and the App details to generate the first set of prompts
Prompt Evaluator - This generates a test plan to evaluate the generated prompts in the context of the app. This also analyzes the test plan results and provides feedback for optimization which will be used by the Compiler in the next phase.
Testplan Executor - This takes the test plan and runs it and generates the output

The above agents work together until they converge on an optimal metric.

Consuming the Assistant

Conva.AI Assistants can be added to apps in two ways:

An interactive Copilot

As an interactive Copilot that end-users can interact with both via voice or text. Conva.AI offers a Copilot SDK for mobile platforms, which comes with a themeable Conversational Overlay UI and other components like ASR and TTS.

Inline Component

As an inline component that fits into the workflow of the app. Developers can use the Conva.AI SDK to build these experiences and build out their own UI to consume the output.

Now that we have the overview of the key concepts of Conva.AI, lets jump right in and build an Assistant and how to go about integrating it to an app

Let the fun begin!

NextCreating an Assistant

Last updated 1 month ago