Tags:

• atria-rag
• generative-ai

Categories:

• functionality

LLM/LMM Experiences Builder

Introduction

ATRIA can integrate third-party AI technologies via API through the LLM/LMM Experiences Builder to create interactive, personalized, and dynamic user interactions, while establishing control mechanisms to ensure security and data privacy.

To do that, the LLM/LMM Experiences Builder allows the creation of LLM chains, which are defined as structured workflows that involve several interconnected steps, each of them using diverse LLM technologies to process, generate, or transform text data. Each step feeds into each other, with the ultimate goal of understanding a request expressed in natural language and providing an accurate response to it.

In the current release, two predefined LLM chains are included in ATRIA, offering two key capabilities:

• Simple flows that call to an LLM: Generative AI capability for understanding and generating human-like texts through LLMs.

• Complex flows: General RAG capability through RAG (retrieval-augmented-generation) processing techniques that combine different AI models.

Currently, only these two predefined chains can be used. In further ATRIA versions, constructors will have the flexibility of creating customized LLM chains.

ATRIA also includes a testing UI interface to test the behavior of the LLM/LMM Experiences Builder when using both Generative and RAG capabilities, before publishing into production. In further versions, the solution will include an interface to configure different parameters easily and a mechanism to load data.

Functional components

The following diagram schematically shows the functional components into play in the LLM/LMM Experiences Builder.

Figure 9. LLM/LMM Experiences Builder

Chain builder and orchestration layer

Currently, this layer allows:

• Using a predefined LLM chain for specific use cases, that corresponds to a RAG (Retrieval Augmented Generation) pipeline integrated using LangChain.
• Manual configuration of parameters.
• Integration of new components (vector databases, document loaders, text splitters, etc.) by Aura Global Team.
• Simple fallback mechanism: flag set in configuration.
• Conversation history, taking into account past interactions for the enrichment of responses.

Control layer

The components of this layer have the following roles:

• Providing mechanisms for ensuring security and data protection.
  • Heuristics blacklists.
  • Prompt injection.
  • Templates.
• Including the control of tokens consumption.

Model layer

The model layer can include both internally and externally hosted models.

Models currently integrated into ATRIA

The AI models currently integrated in ATRIA are:

• Azure OpenAI embeddings model: text-embedding-ada-002

• Hugging face models: paraphrase-multilingual-MiniLM-L12-v2, Multi-qa-distilbert-cos-v1

• Azure OpenAI GPT models: gpt-4-turbo, gpt-4o, gpt-4o-mini, o3-mini

In further releases, the model manager will integrate other state-of-the-art models from different providers, avoiding lock-in and making easy for constructors to choose, try and select the one that fits better with their needs.

Analytics layer

The analytics layer currently includes two features:

• Feedback functionality, for the estimation of the accuracy in the response, in which the user can provide feedback by clicking on a thumbs-up icon if the quality and appropriateness of the answer is correct or selecting the thumbs-down icon if the response misses the point, contains hallucinations, or is unclear.

• Simple RAG monitoring to check how the RAG chain performs.