ATRIA

• 1: Introduction
• 2: ATRIA capabilities
• 2.1: NLP as a Service
• 2.1.1: NLP Apps
• 2.1.2: Semantic Search
• 2.2: LLM/LMM Experiences Builder
• 2.2.1: Generative AI
• 2.2.2: RAG
• 2.2.2.1: General RAG
• 2.2.2.2: Aura SQL RAG pipeline
• 2.3: ATRIA multibrand feature
• 2.4: ATRIA multi-language feature
• 2.5: Generative feedback
• 3: Guidelines for ATRIA use cases constructors
• 3.1: Key ATRIA concepts
• 3.2: Configure experience using Generative/RAG
• 3.2.1: Design preset configuration: Generative
• 3.2.2: Design preset configuration: RAG
• 3.2.3: Calls to API
• 4: Technical components
• 4.1: ATRIA application
• 4.2: Aura Gateway API
• 4.2.1: Architecture and components
• 4.2.1.1: Plugins
• 4.2.1.1.1: aura-generative-api plugin
• 4.2.1.1.2: aura-generative-process-service plugin
• 4.2.1.1.3: aura-nlp-resolution-api plugin
• 4.2.1.1.4: aura-nlp-resolution-process-service plugin
• 4.2.1.1.5: aura-development-api plugin
• 4.2.1.1.6: aura-feedback-api plugin
• 4.2.1.1.7: aura-agents-api plugin
• 4.2.1.1.8: aura-operations-api plugin
• 4.2.2: Operational overview
• 4.2.3: Environment variables
• 4.2.4: API definition
• 4.2.4.1: Aura Gateway API
• 4.2.4.2: Aura Gateway Core API
• 4.2.4.3: Aura Agents API
• 4.2.4.4: Aura Generative API
• 4.2.4.5: Aura NLP resolution API
• 4.2.4.6: Aura Feedback API
• 4.2.4.7: Aura Gateway Development API
• 4.2.4.8: Aura Operations API
• 4.3: ATRIA Model Gateway
• 4.3.1: Architecture and components
• 4.3.2: Operational overview
• 4.3.3: API definition
• 4.3.4: Environment variables
• 4.4: ATRIA Models Manager
• 4.5: AURA GenAI Libs
• 4.6: ATRIA RAG Server
• 4.6.1: Architecture and components
• 4.6.2: Operational overview
• 4.6.3: Environment variables
• 4.7: ATRIA RAG generate DB
• 4.7.1: Architecture and components
• 4.7.2: Operational overview
• 4.8: Agents Manager
• 4.8.1: Architecture and components
• 4.8.1.1: Plugins
• 4.8.1.1.1: agent-dispatcher-api plugin
• 4.8.1.1.2: Agent Deployment Plugin
• 4.8.1.1.3: development-api plugin
• 4.8.1.1.4: configuration-service plugin
• 4.8.1.1.5: redis-connector-service plugin
• 4.8.1.1.6: session-service plugin
• 4.8.2: Communication protocol
• 4.8.3: Environment variables
• 4.8.4: API definition
• 4.8.4.1: Agents Manager API
• 4.8.4.2: Agents Manager Development API
• 4.8.4.3: Agents Manager Dispatcher API
• 4.9: Agents Server
• 4.9.1: Architecture and components
• 4.9.2: Environment variables
• 4.9.3: API definition
• 4.10: ATRIA APIs documentation
• 5: ATRIA operational workflows
• 5.1: NLP as a Service
• 5.2: Agent AI operational workflow
• 5.3: Generative AI
• 5.4: General RAG
• 5.5: Germany General RAG
• 5.6: Ingestion Process Automation operational workflow
• 5.7: Reload config by Redis events
• 5.8: Feedback capability
• 6: Technical guidelines
• 6.1: Build e2e experiences
• 6.1.1: Build experiences that call Generative AI
• 6.1.2: Build experiences that use General RAG
• 6.1.3: Build experiences that call NLP apps
• 6.1.4: Build experiences that call Semantic Search
• 6.2: Publish an API in Kernel
• 6.3: Configure an application
• 6.4: Hot swapping of applications
• 6.5: Agents server
• 6.5.1: Guidelines for technical developers
• 6.5.1.1: Define and configure agent
• 6.5.1.2: Create a Docker image
• 6.5.1.3: Deployment of an agent
• 6.5.1.4: Errors management
• 6.5.2: Guidelines for use cases constructors
• 6.5.2.1: Create and configure an agent
• 6.5.2.2: Create and configure an agent base
• 6.6: ATRIA configuration
• 6.6.1: ATRIA components default configuration
• 6.6.2: Create and configure a preset
• 6.6.3: Import documents into ATRIA
• 6.6.4: Create and configure an agent
• 6.7: Get Kernel access token
• 6.8: Request to Aura NLP Resolution API
• 6.9: Best practices for prompts generation
• 6.10: Request to Aura Generative API
• 6.11: Use ATRIA web interface
• 6.12: O365 Authentication
• 6.13: ATRIA error management
• 6.13.1: atria-model-gateway error management
• 6.14: Tutorial: Create new Copilot preset
• 6.15: Adjust timeouts in ATRIA
• 6.16: Create new Copilot preset (previous to Metallica)
• 6.17: Update ATRIA configuration using ConfigMap (previous to Metallica)
• 6.18: Modify prompts (previous to Metallica)

1 - Introduction

Introduction to ATRIA

What is ATRIA?

The world of Artificial Intelligence (AI) is revolutionizing virtual assistants by integrating advanced AI-based algorithms to understand, anticipate and respond to the users’ needs.

In this framework, ATRIA is a platform that integrates multiple AI technologies, both proprietary and third-party ones, enabling products and services to build experiences flexibly, leveraging the advanced functionalities of AI in a secure and controlled manner.

Figure 1. What is ATRIA?

The innovative solution of ATRIA as a platform that integrates different AI technologies empowers our users to effortlessly incorporate advanced capabilities based on Artificial Intelligence into their products or services.

This can be done in an efficient and secure way through AI-driven capabilities included in ATRIA that are accessible via APIs exposed in Kernel, the Telefónica digital ecosystem, which is founded on the principle of maintaining privacy and security.

Key ATRIA features

The design of ATRIA provides the following main features:

ATRIA functional overview

The following diagram shows how ATRIA operates and its communication flow at a high level:

Figure 2. ATRIA overview

• The user accesses ATRIA through one of the brands that the OB uses to commercialize its services and sends a request.

• First, the request passes through Kernel (Telefónica Digital Ecosystem) for authentication and security purposes.

• ATRIA derives the input information to the corresponding AI-driven capability to be used, previously set in configuration.

• The capability uses the specific AI technology to generate the most relevant response, which is provided back to the user.

ATRIA benefits

ATRIA platform provides key advantages from the use of its AI-powered capabilities, both for constructors and end users, which are summarized below:

• Innovation and integration
  ATRIA facilitates an easy and straightforward integration of AI technologies, powered by our in-house team of experts dedicated to continuous research and implementation of innovations in AI and prompt engineering.

• Accuracy
  Leveraging advanced AI capabilities, ATRIA delivers precise and coherent responses that align closely with user’s requests, enhancing satisfaction and fostering greater interaction.

• Security in interactions
  ATRIA guarantees security against malicious attacks aiming to change the purpose of AI and, additionally, provides mechanisms that ensure privacy and prevent access to, or leakage of, sensitive data from the system.

• Compatibility with Kernel
  ATRIA is compatible with Telefónica Kernel, the technological service that integrates data and APIs from Telefónica services, assuring efficiency, as well as data privacy and control.

• Ethical and responsible AI
  ATRIA preserves the ethical uses of AI by not allowing misuse by individuals or systems, implementing control mechanisms in requests and verifying AI responses.

• Efficiency and time saving
  ATRIA automates tasks, enabling users to complete actions more efficiently. This leads to significant time savings as routine and time-consuming activities can be handled seamlessly.

2 - ATRIA capabilities

ATRIA capabilities

ATRIA AI-driven functionalities

ATRIA is conceived as an AI technologies aggregator and currently includes key capabilities, based both on proprietary and third-party technologies.

The current capabilities and AI technologies in this first ATRIA are shown in the following figure and introduced below.

Figure 3. ATRIA capabilities

NLP as a Service

NLP as a Service enables the connection with Aura cognitive services to leverage different NLP technologies via API for understanding users’ requests and providing accurate responses.

Find detailed information regarding NLP as a Service or access directly to its associated capabilities:

• NLP Apps

  • Semantic Search

LLM/LMM Experiences Builder

The LLM/LMM Experiences Builder allows ATRIA to integrate third-party AI technologies via API to create interactive, personalized, and dynamic user interactions while establishing control mechanisms to ensure security and data privacy.

Find detailed information regarding the LLM/LMM Experiences Builder or access directly to its associated capabilities:

• Generative AI

• RAG

ATRIA features

Additionally, ATRIA contains key features focused on improving the generation of experiences, that allow advanced customization options and adaptability to user preferences.

• Multibrand feature: Users can access ATRIA through the different Telefónica brands available in their country.

• Multi-language feature: ATRIA RAG includes a multi-language feature, to deliver service to a global audience in multiple languages, as it automatically detects the input language and provides the response accordingly.

2.1 - NLP as a Service

NLP as a Service

Introduction to NLP technologies

Natural language processing (NLP) refers to the branch of AI concerned with giving computers the ability to process, understand and generate human language. NLP combines computational linguistics (rule-based modelling of human language) with statistical, machine learning and deep learning models to bridge the communication gap between humans and machines.

Figure 4. NLP technologies

NLP encompasses a wide spectrum of technologies designed to be integrated into diverse user experiences. It includes deterministic and probabilistic techniques, syntax and semantics methods, named entity recognition (NER), etc.

Nowadays, the use of NLP in virtual assistants is not limited to understand and respond to simple utterances but also to derive meaning and use data behind user queries, allowing them to provide relevant and precise responses resulting in more accurate and natural interactions through different NLP technologies.

Application of NLP as a Service in ATRIA

NLP as a Service enables the use of different technologies through NLP Apps (Natural Language Processing recognition stages) for understanding users’ requests and providing back accurate responses.

Currently, these technologies, both proprietary and third-party ones, are included in the Aura NLP component but, in future releases, external NLP methods will be used.

In this framework, constructors have two approaches depending on the utilized stages:

• Using NLP Apps (NLP recognition stages) different from Semantic Search
  Find here detailed information regarding NLP Apps capability

• Using Semantic Search technology, a specific NLP App that overcomes traditional keyword-based searches through the use of embeddings.
  Find here detailed information regarding Semantic search functionality

Figure 5. NLP as a Service in ATRIA

Benefits from the use of NLP as a Service

NLP as a Service offers several benefits both for constructors and end-users:

Benefits for use cases constructors

• Less time and complexity of use cases development
• It is possible to create and configure tailored NLP pipelines, choosing from a variety of available stages and connectors that cover different aspects of Natural Language Processing.
• No need for the manual generation of training phrases (aliases) for generic questions knowledge bases.
• Knowledge bases can be updated continuously in an easy and quick way.
• Accessibility: any application, both internal and external to Aura Platform, can consume this service.

Benefits for Aura end users

• End-users can interact with Aura in a natural and conversational way, using their own words and expressions and even informal language, slang, abbreviations, misspellings, etc.
• Improved Aura’s understanding capabilities, leading to fast and reliable responses and results.
• Easy update of the knowledge bases, so the users can receive reliable responses based on up-to-date data.
• The NLP service can be leveraged as capabilities offered to end-users or as internal features used by other Telefonica teams in order to streamline specific internal processes.

2.1.1 - NLP Apps

NLP Apps capability

Introduction to NLP Apps technology

Within Natural language processing (NLP) technologies, NLP Apps refers to NLP pipelines (chains) that combine different technologies for language processing with several tools for combining them.

Currently, these technologies, both proprietary and third-party ones, are included in the Aura NLP component and can be categorized in the following groups:

• Deterministic recognizers, such as Exact Match or Grammars.

• Probabilistic recognizers, such as OpenAI embeddings: Semantic Search, that uses Azure OpenAI capabilities based on embeddings and Microsoft CLU.

• Classifiers, from the traditional Domain Classifier to the embeddings-based Domain Classifier.

• Entities recognition stages, Standard NER or Gazetteer NER, based on deterministic entity detection.

• Other tools for executing different normalization tasks, connectors, adapters, etc.

The technical description of all the available NLP technologies in included in the document Components for NLP pipelines

Application of NLP Apps in ATRIA

ATRIA enables the generation of experiences (use cases) through the use of Aura cognitive capabilities as stand-alone NLP Apps for sending a request expressed in natural language and receiving back an accurate response without the need for a conversational bot.

Figure 6. NLP Apps in ATRIA

Interaction with NLP Apps in ATRIA

This service is accessible via API, enabling its consumption both from Aura Platform or any external application.

Functional overview

A simple overview of the process is provided below:

• A user sends a request to ATRIA, indicating which specific NLP App she wants to use for the recognition of the request. These Apps are available in Aura NLP, a module of Aura Cognitive Services in charge of processing and understanding human natural language.

• The NLP technologies in the pre-selected specific App resolves the use case and generates a response.

• The response is sent back to the user.

2.1.2 - Semantic Search

Semantic Search capability

Introduction to Semantic Search technology

Within Natural Language Processing technologies, Semantic Search goes beyond the traditional keyword-based search methods, as it delves into the intent and the meaning behind a query, interpreting the meaning of words and phrases.

This leads to the generation of more accurate and relevant search results that align closely with the user’s intent.

For this purpose, semantic search uses neural network embeddings: a representation of words or phrases in a continuous vector space that captures the semantic relationships between them. This information is crucial for semantic search to interpret the user’s intent accurately.

Figure 7. Semantic Search technology

Application of Semantic Search in ATRIA

Semantic Search is a specific NLP App, included in the NLP as a Service capability.

ATRIA benefits from the Semantic Search capability based on embeddings for the development of generic questions experiences (grounded in FAQs).
It allows achieving an accurate understanding of requests and the generation of highly reliable answers, fully aligned with the user's expectations.

Figure 8. Semantic Search in ATRIA

Interaction with Semantic Search in ATRIA

This service is accessible via API, enabling its consumption both from Aura Platform and any external application.

Semantic search technology is available in Aura through a specific Aura NLP stage: OpenAI embeddings.

Current available models

Semantic Search currently uses Azure OpenAI embeddings technology.

Check the version of the model here.

Functional overview

The use of this capability encompasses three different stages:

• Preparation, for the creation of the use case knowledge bases with the required FAQs and associated answers, and the subsequent generation of embeddings with this information.

• Identification, in which a user sends a request to ATRIA, selecting as the specific NLP App the one that includes the semantic search technology (OpenAI embeddings). This app recognizes the user’s request.

• Answer generation: the best response to the user request is identified and sent back to the user.

2.2 - LLM/LMM Experiences Builder

LLM/LMM Experiences Builder

Introduction

ATRIA can integrate third-party AI technologies via API through the LLM/LMM Experiences Builder to create interactive, personalized, and dynamic user interactions, while establishing control mechanisms to ensure security and data privacy.

To do that, the LLM/LMM Experiences Builder allows the creation of LLM chains, which are defined as structured workflows that involve several interconnected steps, each of them using diverse LLM technologies to process, generate, or transform text data. Each step feeds into each other, with the ultimate goal of understanding a request expressed in natural language and providing an accurate response to it.

In the current release, two predefined LLM chains are included in ATRIA, offering two key capabilities:

• Simple flows that call to an LLM: Generative AI capability for understanding and generating human-like texts through LLMs.

• Complex flows: General RAG capability through RAG (retrieval-augmented-generation) processing techniques that combine different AI models.

Currently, only these two predefined chains can be used. In further ATRIA versions, constructors will have the flexibility of creating customized LLM chains.

ATRIA also includes a testing UI interface to test the behavior of the LLM/LMM Experiences Builder when using both Generative and RAG capabilities, before publishing into production. In further versions, the solution will include an interface to configure different parameters easily and a mechanism to load data.

Functional components

The following diagram schematically shows the functional components into play in the LLM/LMM Experiences Builder.

Figure 9. LLM/LMM Experiences Builder

Chain builder and orchestration layer

Currently, this layer allows:

• Using a predefined LLM chain for specific use cases, that corresponds to a RAG (Retrieval Augmented Generation) pipeline integrated using LangChain.
• Manual configuration of parameters.
• Integration of new components (vector databases, document loaders, text splitters, etc.) by Aura Global Team.
• Simple fallback mechanism: flag set in configuration.
• Conversation history, taking into account past interactions for the enrichment of responses.

Control layer

The components of this layer have the following roles:

• Providing mechanisms for ensuring security and data protection.
  • Heuristics blacklists.
  • Prompt injection.
  • Templates.
• Including the control of tokens consumption.

Model layer

The model layer can include both internally and externally hosted models.

Models currently integrated into ATRIA

The AI models currently integrated in ATRIA are:

• Azure OpenAI embeddings model: text-embedding-ada-002

• Hugging face models: paraphrase-multilingual-MiniLM-L12-v2, Multi-qa-distilbert-cos-v1

• Azure OpenAI GPT models: gpt-4-turbo, gpt-4o, gpt-4o-mini, o3-mini

In further releases, the model manager will integrate other state-of-the-art models from different providers, avoiding lock-in and making easy for constructors to choose, try and select the one that fits better with their needs.

Analytics layer

The analytics layer currently includes two features:

• Feedback functionality, for the estimation of the accuracy in the response, in which the user can provide feedback by clicking on a thumbs-up icon if the quality and appropriateness of the answer is correct or selecting the thumbs-down icon if the response misses the point, contains hallucinations, or is unclear.

• Simple RAG monitoring to check how the RAG chain performs.

2.2.1 - Generative AI

Generative AI capability

Introduction to Generative AI

What is Generative AI?

Generative Artificial Intelligence is a subset of Machine Learning that focuses on the creation of new content, such as text, images, or music, based on patterns learned from large volumes of data.

This technology has advanced significantly in recent years, fueled by the development of Deep Learning models that can understand and replicate complex data structures.

Figure 10. Generative AI technology

Below are the main steps in how Generative AI works:

• Training: The model is fed with extensive datasets containing examples of the target content, allowing the system to identify patterns, structures and relationships among different elements.

• Instruction input: The user provides an instruction or “prompt,” which can be a question, a topic, or any indication of what is expected from the model output.

• Content generation: Based on the information obtained during training, the model applies complex algorithms to generate a relevant and coherent response or new content aligned with the user’s request.

• Response delivery: The model presents the generated output to the user quickly and efficiently. It can be a text, an image, or any other type of content.

Within Generative AI, the Large Language Models (LLMs) are advanced AI models designed to understand and generate human-like text, typically trained on vast amounts of text data, enabling them to predict and produce coherent and appropriate text. They are the ones integrated into ATRIA.

Benefits and limitations

The main benefits from the use of Generative AI are summarized below:

• Creativity and Originality: Generative AI generates new and original content that can inspire creators.
• Efficiency: Generative AI automates content generation tasks, allowing humans to focus on more complex activities.
• Personalization: Generative AI generated content is tailored to the specific needs of users.
• Access to information: Generative AI provides quick answers to complex questions thanks to its extensive access to data.

Despite these advantages, Generative AI has certain limitations that led ATRIA to integrate other complementary technologies:

• Hallucinations: Generative AI can generate inaccurate responses that seem plausible, leading to misinformation.
• Temporal Limitations: Generative AI models are limited to the information available at the time of their last training, meaning they cannot access real-time or recent data updates.

Application of Generative AI in ATRIA

Generative AI is a key ATRIA capability provided by a predefined chain designed with the LLM/LMM Experiences Builder.

ATRIA enables the generation of experiences (use cases) to resolve users' requests expressed in natural language by supporting simple calls to AI models.
This is done through an easy integration of advanced Generative AI technologies while guaranteeing security and privacy in interactions.

Figure 11. Generative AI in ATRIA

Example case

Imagine that our platform, ATRIA, operates like a restaurant with different chefs, each specialized in a unique approach to meeting customers' needs.

A traditional generative model can be compared to Chef Manuel, a chef who spent several years mastering in traditional Spanish cuisine.

Manuel’s expertise encompasses a wide range of recipes and cooking techniques, but some of his knowledge may be outdated since he hasn’t pursued further training in recent years.

When a customer requests for a nutritious and hearty meal, Manuel relies solely on his internal knowledge to prepare a classic dish: lentils with vegetables. He does not need to search for additional information because his prior expertise is sufficient to offer a consistent and reliable answer.

A traditional generative model operates like Manuel, generating responses based solely on the implicit knowledge learned during the model's training, without consulting external sources.

Interaction with ATRIA Generative AI

This service is accessible via API, enabling its consumption both from Aura Platform and any external application.

Current available models

The AI-driven models currently integrated into ATRIA are included here.

Functional overview

The use of this capability encompasses different stages:

• When a user sends a request to ATRIA, it is sent to an auto-generative content generator, the one that best aligns with the use case considering different factors such as latencies, costs, etc.

• Additionally, specific instructions upon which the model must base its response are also included. These instructions can be configured to meet specific channel-level business and experience requirements but, at the same time, to ensure that the provided responses retain the nuances of tone and personality that characterize Aura.

• In addition, ATRIA provides a layer of security to avoid prompt injection, that is, to prevent misuse by third-party services that can create malicious prompts as inputs and cause the model to act in unintended ways.
  For example, it can prevent a user from modifying the instructions on how the system should behave or the invalidation of instructions from a predefined block of the prompt (Aura personality), if contradictory instructions are given.

• The Generative AI model recognizes the request and generates the most appropriate response for it. This response is sent back to the user.

Benefits from the use of Generative AI in ATRIA

There are clear benefits derived from the integration of Generative AI in ATRIA:

Benefits for constructors

• It streamlines the use cases development process, since there is no need to generate specific responses or undergo specific trainings.
• Other types of experiences, not directly related to Aura, can be generated. For example: data analysis tasks, development of new products, etc.

Benefits for end-users

• Our customers’ satisfaction will increase, as Aura can offer enhanced understanding capabilities.
• Aura can incorporate new areas of interest for users in a more agile manner and explore new types of users for whom to develop services based on natural language recognition technologies.
• ATRIA interactions guarantee security and privacy for our users.

Generative feedback functionality

When testing how Generative AI/RAG capabilities work with the ATRIA web interface aura-manager, it is possible to use the feedback functionality to estimate the user’s satisfaction regarding the quality and appropriateness of the generated answer to her request. This can be done easily by clicking the thumbs-up or thumbs-down icons.

2.2.2 - RAG

RAG capability

Introduction to RAG technology

RAG (Retrieval Augmented Generation) is a technique for augmenting LLM knowledge with additional data. It provides a way to optimize the output of an LLM with targeted and updated information without retraining it; thus, providing more appropriate answers based on specific and latest data.

The process includes three differentiated parts:

• Retrieval: it searches and extracts relevant information from a KB database using information retrieval techniques, such vector representations (embeddings) to find text blocks that contain the appropriate information to resolve the input request.
• Augmented: the RAG model augments the user input (or prompts) by adding the relevant retrieved data. This step uses prompt engineering techniques to communicate effectively with the LLM.
• Generation: the enriched prompt is sent to an LLM, that generates the most accurate response for the user.

Figure 12. RAG technology

Application of RAG in ATRIA

As explained before, the LLM/LMM Experiences Builder enables the generation of LLM chains that integrate different AI technologies.

Within this capability, complex flows based on the RAG technology can be integrated.

Example case

Imagine that our platform, ATRIA, operates like a restaurant with different chefs, each specialized in a unique approach to meeting customers' needs.

A RAG model can be compared to Chef Sara, a chef who combines her traditional culinary experience with the real-time consultation of resources to enhance her recipes with the latest culinary trends worldwide, as she likes to be continuously up-to-date.

When a customer requests a nutritious and hearty meal, Sara goes beyond her own knowledge, based on already learnt techniques and recipes. Instead, she consults innovative cuisine resources: Indian cookbooks and her recent notes on advanced molecular cooking techniques. These external sources allow her to innovate and propose a unique dish: a curry foam, light and airy, with an intense spice flavor and a touch of coconut milk.

In technical terms, the RAG approach combines:
a. Generation based on prior knowledge (the internal model): equivalent to Sara's knowledge of cooking.
b. Real-time retrieval of external information: consulting cookbooks and notes represents how a RAG system looks up information in databases or dynamic sources during the response process.

This integration allows the model to provide more contextualized responses, tailored to specific needs, especially when the stored knowledge is limited or insufficient.

Currently, ATRIA incorporates the following RAG chains:

• General RAG: Complex AI-driven flow for resolving generic questions experiences based on FAQs

• SQL RAG: RAG-based pipeline for resolving SQL queries

In upcoming versions, constructors will be able to design their own LLMs chains based on RAG.

Benefits from the use of RAG technologies

• Updated and targeted information: RAG allows developers to provide the latest data to the generative models, targeted to the specific use case.

• Cost-effective implementation: Data in the knowledge repository can be continually updated without incurring significant costs.

• Enhanced user trust: The data sources contributing to the RAG’s vector database are identifiable. This transparency allows for the correction or removal of any inaccuracies present in RAG and clearly improves users’ confidence.

• Improved developers control: With RAG, developers can test and improve their applications more efficiently, control and change the LLM’s information sources to adapt to changing requirements, restrict sensitive information retrieval to different authorization levels and ensure the LLM generates appropriate responses.

2.2.2.1 - General RAG

General RAG capability

Application in ATRIA: General RAG

ATRIA enables the generation of generic questions experiences (use cases) to resolve users' requests expressed in natural language and based on FAQs by supporting complex calls to AI models.
This is done through the integration of a predefined RAG (Retrieval Augmented Generation) chain while guaranteeing security and privacy in interactions.

Figure 13. General RAG in ATRIA

The predefined RAG chain defined in ATRIA is called General RAG. It includes additional steps that overcome the potential of Retrieval Augmented Generation technologies by optimizing the input prompt and generating more accurate responses. See details in section Functional overview.

In upcoming versions, constructors will be able to design their own LLMs chains based on RAG.

Interaction with ATRIA General RAG capability

This service is accessible via API, enabling its consumption both from Aura Platform and any external application.

Current available models

The AI-driven models currently integrated into ATRIA are included here.

Functional overview of General RAG

The use of the General RAG capability encompasses three different stages:

• Data ingestion, that includes uploading the knowledge bases used for lexical (keywords) and semantic search (embeddings) search.
  Discover the underlying processes for that in the document Import documents into *ATRIA, as well as tips for data curation, a process recommended before the documents uploading.

• RAG chain: If a request enters ATRIA, the General RAG capability executes the predefined steps in its chain, which are described in the following figure.

• Aura answer: The generated response is sent to the user.

Figure 14. General RAG stages

Making a zoom in the stages of the General RAG pipeline, the following steps are included:

Figure 18. General RAG chain

1. Security: the request is analyzed to improve security and prevent prompt injection.
2. Multi-language: The multi-language feature allows users to receive responses in their own language. The system automatically detects the language in the user’s request in the multi-language step of the RAG pipeline, and this language is afterwards used in the response generation stage to provide the response back to the user.
3. Conversation history: If there is information from previous interactions, they are now analyzed to check if they are relevant for the current query. In this case, the query is rewritten using this context information.
4. Retrieval: Lexical and semantic retrieval from databases that return text blocks with key information to compose the response.
5. Post-filtering: The retrieved text blocks are compared with the user query to determine if they are relevant or not to answer the question.
6. Response generation: If so, the fragments are reordered and used to compose an augmented prompt which is resolved through LLMs technology.

Benefits from the use of ATRIA General RAG

• The General RAG predefined chain enables all the advantages of RAG technologies to the resolution of use cases. Specifically for generic questions use cases based on FAQs.

• Moreover, General RAG capability integrates other extra features that lead to more accurate responses:

  • Features to avoid prompt injection
  • Conversation history
  • Filtering steps

• The use of Retrieval Augmented Generation techniques enables the use of continually updated information, every time an up-to-date knowledge base is uploaded into the system.

Generative feedback functionality

When testing how Generative AI/RAG capabilities work with the ATRIA web interface aura-manager, it is possible to use the feedback functionality to estimate the user’s satisfaction regarding the quality and appropriateness of the generated answer to her request. This can be done easily by clicking the thumbs-up or thumbs-down icons.

Do you need a more detailed explanation on how Generative feedback capability works?

• Access the document Generative feedback functional description
• Access the document Use ATRIA web interface (aura-manager) to discover how to utilize this functionality.

2.2.2.2 - Aura SQL RAG pipeline

Aura SQL RAG pipeline

Introduction

ATRIA currently integrates one RAG pipeline for the conversion of a request from natural language to an SQL query.

Steps in the SQL RAG chain

The use of the SQL RAG chain encompasses different stages, which are explained and schematically represented below.

Figure 15. SQL RAG chain

1. Injection checking

• Detects the presence of anomalies in the user’s query that may affect the resolution process.
• Currently, a set of checks, based on heuristics, are made:
  • Detects overly long questions.
  • Detects suspicious substrings in the query.

2. Question translation (currently deactivated)

• Optional step for the translation of the user’s query into English.
• Currently, it is not activated.

3. Candidate table retrieval

• The system searches the candidate tables for relevant documents. This is currently done using a hybrid search, through the combination of lexical and semantic search (embeddings).
• The table retrieval is currently based on the similarity between the user’s query and the tables high level description.

4. SQL query generation

• The top-2 results (tables) are selected.
• In them, the user’s request is converted from natural language to an SQL query.

2.3 - ATRIA multibrand feature

Introduction to ATRIA multibrand feature

Introduction

ATRIA, just like Aura Virtual Assistant, is designed as a multibrand platform, meaning that users can access ATRIA through the different Telefónica brands available in their country.

This multibrand feature is based on a multitenant architecture, with a tenant defined as the deployment associated to a specific brand.

Functional multitenant architecture in ATRIA

An overview of the functional operation of the multibrand feature in ATRIA is shown and explained below:

Overview of multitenant architecture in ATRIA

• ATRIA supports different brands
• Each brand is associated to several channels
• Each channel allows accessing to use cases in a specific domain
• When a user send a request, it passes through Kernel and is managed by a specific Kernel tenant
• This Kernel tenant sends the query to a particular ATRIA tenant
• ATRIA calls the required AI-driven models for its resolution

Technical documents

Multitenant configuration in Aura installer

Aura installer: Multitenant configuration: Guidelines for the configuration of different tenants when several brands are available in the OB.

Descriptive documents and guidelines

Once the user accesses through a specific Telefónica brand, the technical behavior of the corresponding Aura tenant is similar to the one before the implementation of the multitenant architecture. Therefore, there are no specific descriptive documents or guidelines for the multitenant architecture.

2.4 - ATRIA multi-language feature

Introduction to ATRIA multi-language feature

Introduction

ATRIA RAG now includes a multi-language feature, to deliver service to a global audience in multiple languages.

This multi-language capability allows users to make a request to ATRIA and receive back the response in their own language through a technology that automatically detects and adapts to the input language.

The multi-language feature provides multiple benefits:

• The information provided by ATRIA is easier to understand, as it is generated in the user’s language.
• A wide range of languages is supported, allowing ATRIA to reach a global audience.
• The user experience is also optimized by reducing the need for external translation tools, making communication more seamless and natural.

From a technical point of view, the model for text identification and classification fastText is used, that supports more than 176 languages.

Functional overview

This feature is only available for RAG stages

A high-level overview on how the ATRIA multi-language feature works is included below.

• A users sends a request to ATRIA in a specific language. ATRIA automatically detects the input language and sends the response back to the user in the same language.

• In case the user’s request includes a mixture of languages (for example, “por favor, dame feedback”), ATRIA detects the predominant language of the query and uses it in the response.

• In case ATRIA is not capable of identifying the request language, then the system generates the response in a language previously configured by default (that should be the region/country primary one).

• The multi-language feature can be activated or deactivated by ATRIA constructors, as well as configured to meet their requirements and needs.

Technical guidelines

How constructors can configure ATRIA multi-language feature?

Constructors can configure this feature through different parameters of the prompt:

• Best practices for ATRIA configuration: Use the multi-language feature

ATRIA server internal configuration

The ATRIA server configuration responsible for managing the multi-language feature includes these fields:

• ATRIA default configuration: language identification fields

2.5 - Generative feedback

Generative feedback functional description

Introduction

Within the use of the ATRIA AI-driven Generative AI or RAG capabilities, we have developed a feedback functionality.

This feedback functionality allows the estimation of the user’s satisfaction regarding the obtained response.

The user can provide feedback by clicking on a thumbs-up icon if the quality and appropriateness of the answer is correct or selecting the thumbs-down icon if the response misses the point, contains hallucinations, or is unclear.

Functional operation

The underlying process is summarized in the following lines and schematically shown in the figure below:

1. An application sends a request to aura-gateway-api generative with a correlator.
2. Firstly, it passes through Kernel (Telefónica Digital Ecosystem) for authentication and security purposes.
3. aura-gateway-api processes the received request and sends the request to the auto-generative content generator atria-model-gateway to obtain an appropriate response.
4. atria-model-gateway generates the most appropriate response and sends it back to aura-gateway-api.
5. aura-gateway-api sends the response back to the service that initiated the request with the same correlator and a session identifier.
6. An application sends a request to aura-gateway-api feedback with:
   • A new header correlator
   • The sessionId received in the path
   • The field msg_corrid, in the body, that indicates the correlator of the message the feedback is about.
7. aura-gateway-api processes the received request and communicates with atria-model-gateway to send this request.
8. atria-model-gateway stores the feedback.
9. aura-gateway-api communicates a 204 to the application.

3 - Guidelines for ATRIA use cases constructors

Guidelines for ATRIA use cases constructors

Scope

ATRIA foundations are based on technical components and processes involving different teams with varying levels of technical expertise.

With the specific goal of supporting ATRIA use cases constructors, the current documents aim to describe the main procedures for creating experiences in a practical and simplified way.

Navigate through the section

3.1 - Key ATRIA concepts

Key ATRIA concepts

Preset

The preset is the key entity for the configuration of ATRIA. It is like a recipe for your experience, that defines which ingredients to use and in what amounts.

Technically, the preset is a JSON file that defines a hierarchy of both required and optional configuration parameters for the use case configuration.

ATRIA preset

Once the constructors have defined a preset, it must be included in an application to be used by ATRIA.

Prompt

Within an ATRIA preset, a prompt is defined as an input instruction given to an AI model to generate a response. It guides the AI in the required kind of output we want as constructors.

Application

From the constructor’s point of view, an application can be defined as a container for ATRIA presets.

When developing a use case, constructors should use an existing application or create a new one in order to:

• Define the specific ATRIA capabilities that can be used, currently Generative AI, RAG, NLP as a Service and Semantic Search.
• Declare the presets that this application can use.

Highlights

• Every preset and application is identified by both a name and an ID.

• To be used and tested, a preset must be placed inside an application (folder).

• Nested applications are not allowed. The hierarchy is: application → preset.

• An application can contain one or more presets.

• A single preset can belong to multiple applications. When you update the preset, it will be updated in all applications it is associated with.

  
  Definition of presets in an application

• Each environment can be considered as an independent container of applications and presets. Consequently, if you update a preset in one environment, another preset with the same ID will not be updated in a different environment.

Access technical documentation

Do you need more technical information regarding ATRIA components? Access here:

ATRIA technical components

3.2 - Configure experience using Generative/RAG

Configure your experience using Generative/RAG

Overall process workflow

The current section shows the sequence of required tasks for building a use case in ATRIA, whether using Generative AI or Retrieval-Augmented Generation (RAG) capabilities.

Check the specific step of your interest and click on the corresponding block to access the corresponding comprehensive guidelines.

---
config:
  theme: base
  look: neo
  layout: dagre
---
flowchart LR
 subgraph s1["Design preset configuration"]
        n1["Generative AI"]
        n2["RAG"]
  end
    s1 --> B{"Is it a <br>new preset?"}
    B -- Yes --> D["Create <br>a new preset"]
    B -- No --> E["Update an <br>existing preset"]
    D --> F{"Already <br>existing <br>application?"}
    F -- Yes --> G["Add preset <br>to an existing <br>application"]
    F -- No --> H["Create <br>an application"]
    H --> G
    style n1 fill:#FFFFFF
    style n2 fill:#FFFFFF
    style s1 fill:#d5dade
    style B fill:#FFF9C4
    style D fill:#d5dade
    style E fill:#d5dade
    style F fill:#FFF9C4
    style G fill:#d5dade
    style H fill:#d5dade
    style H fill:#d5dade

• Design preset configuration

  • Generative AI
  • RAG
    
    

• Create a new preset

• Update an existing preset

• Create an application

• Add preset to an existing application

3.2.1 - Design preset configuration: Generative

Design preset configuration: Generative

Introduction

A simple ATRIA Generative experience performs a basic interaction with a Large Language Model (LLM), generating content from an input user’s query based on predefined instructions (prompts), and including control stages to ensure reliable and appropriate responses.

The first task in the process involves defining the configuration for your preset, that entails selecting the specific parameters tailored to your use case.

These parameters are defined in the aura-configuration-api API swagger » PresetConfiguration

The preset configuration parameters have been divided into two categories:

• Basic configuration: Selection of the most relevant ones from the use case constructors point of view.
• Advanced configuration: Other preset parameters that can also be configured but require greater technical expertise.

From all the available preset parameters, some of them are general ones and others are specific for each step of the predefined Generative pipeline, which is schematically shown below.

ATRIA Generative pipeline

a. Define the basic configuration for your preset

1. Select a preset template for Generative

For this purpose, two options can be used:

1.1. Create a preset from scratch

If you want to create a preset from scratch, use the template below.

This preset JSON file is intended to serve as a base template on which you can make your modifications.

Remember that this preset template only included basic configuration parameters for use cases constructors.

{
    "id": "e27ca464-488a-435d-a508-da8a262d905f",
    "name": "openai",
    "description": "openai model",
    "group": "simple_ai",
    "session": {
        "window": 0
    },
    "generative": {
        "model": {
            "id": "gpt-4o-mini",
            "parameters": {
                "max_tokens": 1024,
                "response_format": {
                    "type": "json_object"
                    },
                "temperature": 0.2,
                "top_p": 0.9
                }
               },
        "injectionMaxLength": 1000,
        "prompts": {
            "template": "{MSG}",
            "preamble": {
                "text": "Speak as if you were {name}",
                "args": {
                        "name": "a pirate"
                        }
             },
            "examples":[
                "Hello, comrades"
                "Hoist the sails"
                       ],
            "promptMaxLength": 10000,
            "promptRegexClean": "[#\\n\"]+",
            }
        }
      }

1.2. Use available presets as templates

If you want to use an existing ATRIA preset and update specific parameters on it or use it as a reference to create a new one, you can access the list of the presets available in your environment: Calls to API: Get info about the available presets or applications

2. General parameters

Configure key general parameters of the Generative experience.

• id: Mandatory. Unique preset identifier in UUID format.
• name: Mandatory. Preset Name.
• description: Optional. Description of the preset functionality.
• group: Mandatory. This parameter is used to group requests regarding the AI technologies used to generate KPIs. Value: simple_ai.
• model and associated parameters: Model to be used in the LLM call of the experience.
  • The following options are available: Models by default.
  • In addition, it is possible to include other model relevant parameters. Specifically, response_format, that allows selecting the format of the response, is interesting.

3. Security

Improve security and prevent prompt injection.

• injectionMaxLength: Optional. Maximum length of the input request. If longer, an error is provided and the request does not enter the following stage.
• promptMaxLength: Optional. Maximum length of the completed prompt. Used to avoid calling LLMS with wrong prompts. If the prompt length exceeds the set value, the prompt will be truncated and the LLM will only use the truncated prompt to generate the response.
• promptRegexClean: Optional. Regex pattern to clean the query before sending it to the model. This is useful to remove unwanted characters or patterns from the query. Type: number.

4. Context

Enrich the response including information from past interactions.

• window: Number of previous interactions from the same session that the model will take into account to generate the response.

5. Response generation

Define the prompt with instructions to be used by the AI model for the generation of the response.

• template: Optional. Template that includes the user’s input. It must include {MSG} for the user’s utterance.
• preamble: Optional. Instructions that the model must follow for the use case.
  - text: Specific instructions sent to the language model. It can include variables as placeholders ({}).
  - args: Specific values for the placeholders defined in the text.
• examples: Optional. Examples to enrich the prompt.

Check the document Best practices for prompts generation that includes practical guidelines for creating a prompt in ATRIA

b. Define advanced configuration for your preset

In addition to the basic parameters for use cases constructors, presets also include other advanced fields that can also be configured but require greater technical expertise.

Discover them here: Create and configure a preset.

3.2.2 - Design preset configuration: RAG

Design preset configuration: RAG

Introduction

An ATRIA RAG experience performs a multiple-stages interaction with a Large Language Model (LLM), generating content from an input user’s query based on predefined instructions (prompts) to ensure reliable and appropriate responses.

The first task in the process involves defining the configuration for your preset, that entails selecting the specific parameters tailored to your use case when using the RAG capability.

These parameters are defined in the aura-configuration-api API swagger » PresetConfiguration

The preset configuration parameters have been divided into two categories:

• Basic configuration: Selection of the most relevant ones from the use case constructors point of view.
• Advanced configuration: Other preset parameters that can also be configured but require greater technical expertise.

From all the available preset parameters, some of them are general ones and others are specific for each step of the predefined ATRIA RAG pipeline, which is schematically shown below.

ATRIA General RAG pipeline

a. Define the basic configuration for your preset

As explained before, the basic configuration does not include all the available preset parameters for these reasons:

a. Presets have both mandatory and optional parameters, so they can be adjusted to a specific experience.

b. Certain preset parameters are defined by default. Parameters defined by default are not included in the preset configuration file. For using these default values, nothing must be done in the preset by the use case constructors.
One of the preset parameters is the prompt. In the ATRIA General RAG pipeline, stages marked as “Implies LLM call” (grey border boxes) require editing a prompt with instructions for the LLM model. Constructors can use the prompts defined by default for each stage or modify them. Detailed information is included in the corresponding RAG stage section.

1. Select a preset template for RAG

For this purpose, two options can be used:

1.1. Create a preset from scratch

If you want to create a preset from scratch, use the template below. This JSON file is intended to serve as a base template on which you can make your modifications.

Remember that this preset template only included basic configuration parameters for use cases constructors.

{
  "id": "1cafcb5c-7951-4645-86d4-055d3b46fe79",
  "name": "atria-rag-gpt-35-turbo",
  "group": "enriched_ai",
  "description": "Atria rag GPT 3.5",
  "session": {
      "window": 3
  },
  "rag": {
      "ragType": "questions-answers",
      "model": {
          "id": "gpt-35-turbo",
          "parameters": {
              "max_tokens": 4000,
              "temperature": 1,
              "top_p": 1
          }
      },
      "stages": {
          "promptSystemLanguage": "es",
          "defaultUserLanguage": "es",
          "SecurityStg": {
              "injectionMaxLength": ""
          },   
          "contextStg": {
              "enabled": true,
              "stickyContext": "ask_llm",
              "prompts": { }
          },
          "retrievalStg": {
              "sources": {
                  "name": "project-gpt-35-turbo",
                  "embeddings": "text-embedding-ada-002",
                  "docs": [
                      {
                      "extension": "pdf",
                      "loader": {
                          "loaderType": "unstructured",
                          "options": {
                              "loaderMode": "single",
                              "postProcessors": ""
                              }
                          }
                      }
                  ],
                  "splitter": {
                        "options": {
                            "chunkSize": 60,
                            "chunkOverlap": 20
                      }
                  },
                  "retrievers": [
                      {
                        "retrieverType": "qdrant",
                        "config": {
                            "numDocs": 2,
                            "loadChunkSize": 10000
                        }
                      }
                  ]
              }
          },
          "postFilteringStg": {
              "enabled": true,
              "candidatesPostFiltering": "llm_filter",
              "prompt": { }  
          },
          "generativeStg": {
              "ragStrategy": "stuff",
              "prompts": { }  
          }
      }
  }
}

1.2. Use available presets as templates

If you want to use an existing ATRIA preset and update specific parameters on it or use it as a reference to create a new one, you can access the list of the presets available in your environment: Calls to API: Get info about the available presets or applications.

2. General parameters

Configure key general parameters for your RAG experience.

• id: Mandatory. Unique preset identifier in UUID format.

• name: Mandatory. Preset Name.

• group: Mandatory. This parameter is used to group requests regarding the AI technologies used to generate KPIs.
  Value: enriched_ai for RAG preset.

• description: Optional. Description of the preset functionality.

• ragType: Optional. Type of RAG. For RAG of documents, the value must be: "ragType": "questions-answers"

• model and associated parameters: Model to be used in all the LLM calls of the experience.

  • The following options are available: Models by default.
  • In addition, it is possible to include other model relevant parameters.
    
    

• promptSystemLanguage: Language of the prompts to be used. Values: es, en.
  If “default” prompts are used, the language is en.

3. Security

Improve security and prevent prompt injection.

• InjectionMaxLength: Maximum number of characters allowed in the user’s query. If longer, an error is provided and the request does not enter the following stage.

4. Multi-language

The multi-language feature allows users to receive responses in their own language.

The system automatically detects the language in the user’s request in the multi-language step of the RAG pipeline, and this language is afterwards used in the response generation stage to provide the response back to the user.

Activate multi-language feature

• No action is required by constructors, as it is activated by default in the preset parameter #.auto.language.user_query, within the args field of the prompt.

  Related parameters in preset

  
  

• Check the prompt by default for this stage here:

  
  

• Additionally, constructors can adjust the parameter defaultUserLanguage to select the language to be used in case the system does not recognize the language of the user’s request. By default, this is set to English.

  Related parameters in preset

  
  

Deactivate multi-language feature

• Constructors must modify the prompts in which the parameter #.auto.language.user_query is included and customize it according to their use case.

• The following example shows a prompt adjusted to provide the response in English:

The associated code for this prompt customization is included below. If you want to modify the default prompt, you can take this one as a reference and update it.

...
  "generativeStg": {
      "prompts": {
          "stuff": {
              "system": {
                  "default": {
                      "text": "Respond in English.\n\nQuestion:\n{question}\n",
                      "args": {}
                  }
              },
              "human": {
                  "default": {
                      "text": "You are going to generate an answer for a user question or query. \nTo generate the answer, take always into account all the information available in the context provided.\n\nContext:\n{context}\n\nQuestion:\n{question}\n\nNever include information by your own using your own knowledge.\n{extra_prompt}\n"
                  }
              }
          },
          "notAnswerResponse": {
              "default": {
                  "text": "You are a question answering agent. You have tried to answer this question: {query}\nHowever you do not have information to answer this.\nPlease, tell the user that you are not able to answer, apologize and invite the user to make other question.\nAvoid any harmful answer, such as sexual, rude, sexist or racist.\nRespond in English.\n\nUser question:\n{query}\n",
                  "args": {}
              }
          }
      },
...

5. Context

The context stage (contextStg) of the RAG pipeline is responsible for rewriting the user’s query if it is related to the previous context.

• window: Set the number of previous interactions taken into account through the size of the session window, in queries.

• enabled: The contextStg can be enabled or disabled. Set to true to enable the use of context information.

• stickyContext: The recommended strategy is ask_llm. With this strategy, the LLM firstly checks if the user query is related to the previous ones. ​Only if it is, the LLM rewrites the question.

• Context prompts: Two prompts can be modified:

  • sameContext: Prompt to check if the query is in the same context.
  • recreatedQuestion: Prompt to rewrite the original question, only if the LLM has evaluated that the request is related to the previous one.

...
"contextStg": {
       "stickyContext": "ask_llm",
       "enabled": true,
       "prompts": {
         "sameContext": {
           "default": {
             "text": "Below is a conversation followed by a question. You must determine if the question corresponds to the same context as the conversation or if it is from a different context.\n Respond only with: [SAME CONTEXT] o [DIFFERENT CONTEXT]\n\nConversation:\n{memory}\n\nQuestion:\n{query}"
           }
         },    
         "recreatedQuestion": {
           "system": {
             "default": {
               "text": "The user text contains a query, plus the previous conversation turn.\n\n- If the previous conversation is relevant for the current query, incorporate it into the query and produce a rewritten query\n- else just repeat the current query.\n\n  Always rewrite the question in the same language as the user's question."  
               }
            },
            "human": {
             "default": {
               "text": "Previous conversation:\n{memory}\n\nCurrent query:\n{query}\n\nRewritten query:\n"
             }
           }
         }
   }
}
...

6. Retrieval

The retrieval stage (retrievalStg) is responsible for retrieving the relevant text blocks for the generation of the final response to the user. There are three configurable steps within this stage, which are shown in the following diagram.

6.1. Configure the knowledge base

Before this task, it is important to curate the data in order to optimize the recognition process. For this purpose, we recommend following the Guidelines for data curation.

The configuration of the knowledge base focuses on the parameters related to the segmentation and processing of the kb.

Below, a selection of parameters that can be configured is included:

• embeddings: Mandatory. Identifier of the embeddings model to be used.

• docs:

  • extension: Mandatory. File types (extensions) of documents that can be processed. The extensions must be separated by a comma.
  • loader and postProcessors: Parameters in charge of reading data from the source and converting it into a usable text or structured text.

• splitter: Optional. Parameter in charge of dividing large text inputs into smaller, manageable chunks to make them suitable for processing.

  • chunkSize: Maximum number of characters or tokens allowed in each chunk.
  • chunkOverlap: Number of overlapping characters or tokens between consecutive chunks to preserve context.

• retrievers: List of retrievers (lexical and semantic) and their associated parameters used for storing and retrieving information from the knowledge base.

6.2. Ingestion process

During this step, the documents are uploaded into the knowledge base. For this purpose, it is required to specify where the documents will be uploaded so that the system can locate them.

• Configure the path in the preset
  The preset must specify the exact path where the knowledge base is located in Microsoft Azure Storage Explorer, taking the value of the following parameters of the preset:

  Preset parameters for ingestion path

  
  

  
  

• Upload document to Azure Blob Storage
  Within the corresponding environment, access the atria-resources folder and insert the documents in the following path:
  <preset_name>/<retrievalStg.sources.name>/<retrievalStg.sources.docs[i].extension>
  For the previous example:
  atria-resources/atria-rag-de-faqs/project-de-faqs/pdf

  
  

• Notify if needed
  Once documentation has been uploaded and the preset has been configured, it is needed to execute the ingestion process (GES, Engineering Team).

6.3. Use your knowledge base

Edit these parameters to indicate how to use the knowledge base:

• numDocs: Optional. Number of documents to be obtained as output from the retrieval stage.

7. Post-filtering

The post-filtering stage postFilteringStg verifies the relevance of retrieved text blocks. For each candidate, the LLM determines if the candidate text is related to the query, and if not, the candidate will be filtered out.

• enabled: This stage can be enabled or disabled. If not enabled, no post-processing will take place.

• candidatesPostFiltering: This parameter is fixed to the value llm_filter

• prompt: Prompt for the post-filtering stage. It is recommended to use the default prompt and not change it.

The default prompt is included below, both schematically and JSON code. It is recommended to use it with no modifications. Nevertheless, if you want to modify it, take the default one as a reference and make your updates.

...
"postFilteringStg": {
          "enabled": true,
          "candidatesPostFiltering": "llm_filter",
          "prompt": {
            "default": {
              "text": "Below is an excerpt of text followed by a question. You must determine if the excerpt is relevant or irrelevant for answering the question.\nRespond only with: [RELEVANT] o [IGNORABLE]\n\nExcerpt:\n{extract}\n\nQuestion:\n{query}\n\n\n"
            }
          }
        }
...

8. Response generation

Stage for the generation of the response based on the retrieved context. This last stage is ideal for adjusting response behavior (e.g., multi-language, tone, etc.).

When the RAG pipeline flow reaches the response generation stage (generativeStg), there are two options:

• If relevant text blocks have been achieved in the previous stage, then:

  • By default, the prompt stuff is used to formulate answers with the retrieved context.

• If no relevant text blocks have been found in the previous stage, then:

  • By default, the prompt notAnswerResponse is used to formulate the response when the question cannot be answer

The above-mentioned prompts by default are included here. If you want to modify them, take the default ones as a reference and make your updates.

...
"generativeStg": {
      "prompts": {
        "stuff": {
            "system": {
                "default": {
                      "text": "Respond in language {user_query_language}. \n\nQuestion:\n{question}\n",
                      "args": {
                              "user_query_language": "#.auto.language.user_query"
                              }
                            }
                        },
            "human": {
                "default": {
                      "text": "You are going to generate an answer for a user question or query. \nTo generate the answer, take always into account all the information available in the context provided.\n\nContext:\n{context}\n\nQuestion:\n{question}\n\nNever include information by your own using your own knowledge.\n{extra_prompt}\n"
                        }
                      }
                    },
        "notAnswerResponse": {
            "default": {
                      "text": "You are a question answering agent. You have tried to answer this question: {query} \nHowever you do not have information to answer this.\nPlease, tell the user that you are not able to answer, apologize and invite the user to make other question.\nAvoid any harmful answer, such as sexual, rude, sexist or racist.\nRespond in language {user_query_language}.\n\nUser question:\n{query}\n",
                      "args": {
                        "user_query_language": "#.auto.language.user_query"
                            }
                        }
                    }
                }
            }

b. Define advanced configuration for your preset

In addition to the basic parameters for use cases constructors, presets also include other advanced fields that can also be configured but require greater technical expertise.

Discover them here:

• ATRIA default configuration: prompts
• Create and configure a preset
• Import documents into ATRIA

3.2.3 - Calls to API

Calls to API

Introduction

Once your preset configuration is designed, whether for Generative AI or RAG, you can proceed to use the API to continue with the process of creating experiences in ATRIA.

The current document includes a specific procedure for calling APIs. You can access more detailed technical content in this document: Create and configure a preset

Prerequisites

These prerequisites are common for all the tasks included in the following sections.

• Download the aura-configuration-api API swagger

• Make sure you have available Postman or a similar tool to call the API.

• Connect to the intended environment for your preset, by modifying the corresponding {{baseUrl}} and {{apiKey}}.

1. Create a new preset

1. Open the Postman application and select the POST call:
   aura-services > v2 > configuration > presets > Adds a new preset

2. Copy and paste the content of your configured preset (JSON file) in the body.

3. Send the POST request.

2. Update an existing preset

1. Do you have your preset on hand?

• No: Go to step 2.
• Yes: Go directly to step 3.
  

2. Access your intended preset:
   2.1. Open the Postman application.
   2.2. Make a GET request to the API with the preset ID.
   aura-services > v2 > configuration > presets > {presetId} > Gets all the info associated to one preset
   This will show you the entire preset to be updated.

3. Select the PATCH call:
   aura-services > v2 > configuration > presets > {presetId} > Updates some info associated to one preset

4. Copy and paste the content of your preset (JSON file) with the required updates in the body.

5. Copy and paste the ID of the preset in the params field.

6. Send the PATCH request.

3. Create an application

Once your preset is created, it must be placed on an application to be used by an experience.

An application must always contain at least one preset, it cannot be empty.

1. Configure the application JSON file

   1.1. Copy the application template below.

   Application template

      ```json
        {
            "brand": "10000",
            "id": "816bdab6-3ea3-4a77-bdea-12945d6d7052",
            "models": {
                "level": "user",
                "presets": [
                    "19c58923-6fc5-4459-8f9e-822af2136ab1"
                ]
            },
            "name": "app"
        }    
      ```
   

   
   1.2. Modify the template below as required by defining the following fields:
   • brand: Identifier of the Telefónica Brand associated to the application. Available values in the document Telefónica brands management.
   • id: Unique identifier of the application in UUID format.
   • name: Unique application name.

   1.3. Include the required presets in your application (presets field in the application template above)

2. Open the Postman application and select the POST call:
   aura-services > v2 > configuration > applications > Adds a new application

3. Copy and paste the content of your new application (JSON file) in the body.

4. Send the POST request.

4. Add a preset to an existing application

1. Do you have your application on hand?

• No: Go to step 2.
• Yes: Go directly to step 3.

2. Access your intended application
   2.1. Open Postman.
   2.2. Make a GET request to the API with the application ID:
   aura-services > v2 > configuration > applications > {applicationId} > Gets all the info associated to one application
   This will show you the entire application to be updated.

3. Select the PATCH call:
   aura-services > v2 > configuration > applications > {applicationId} > Updates some info associated to one application

4. Copy and paste the content of your application (JSON file) in the bodyand add the preset ID to be associated to the application.

5. Copy and paste the ID of the application in the params field.

6. Send the PATCH request.

5. Get info about available presets/applications

If you need to know which presets or applications are available in your environment, follow these steps:

1. Open Postman and select the following GET calls:

   • For presets:
     aura-services > v2 > configuration > presets > Gets all presets info
   • For applications:
     aura-services > v2 > configuration > applications > Gets all application info

2. In order to view all the parameters related to presets or applications, deselect the options excludeParams and includeParams.

3. Send the corresponding GET request.

6. Delete a preset/application

1. Open Postman and select the following DELETE calls:

   • For presets:
     aura-services > v2 > configuration > presets > {presetId} > Delete one preset
   • For applications:
     aura-services > v2 > configuration > applications > {applicationId} > Delete one application

2. Send the corresponding DELETE request.

4 - Technical components

ATRIA technical components

Introduction

ATRIA capabilities are driven by certain technical components, which are fully described in the succeeding documents, together with their role, architecture, communication protocols, sub-components and environment variables.

Index of technical components

• ATRIA application: Configurable entity that allows the connection of channels, services or skills with aura-gateway-api.

• aura-gateway-api: Entry gateway to ATRIA that manages the access to the different AI cognitive capabilities.

• atria-model-gateway: Component that orchestrates the communication with LLM models.

• atria-rag-server: Component that manages a RAG-type server for using RAG (Retrieval Augmented Generation) models.

• atria-rag-generate-db: Component that manages the upload of documents to feed the databases.

• agents-manager: Component that manages input and output requests to the registered agents, storing their conversation history by session.

• agents-server: Component responsible for the execution of different agents’ tasks.

• Aura NLP: Module of Aura Cognitive Services in charge of processing and understanding human natural language. It can be considered as a cross component for Aura Virtual Assistant and ATRIA.

• ATRIA APIs documentation: List of available ATRIA APIs.

4.1 - ATRIA application

ATRIA application

Introduction

Within ATRIA’s framework, an application is defined as an entity that allows the connection of a channel, service or skill with aura-gateway-api, the component in charge of managing the access to the different ATRIA capabilities.

ATRIA application

As a preliminary step for leveraging any ATRIA AI-driven capability, the use case constructor must configure an application, including different parameters:

• To indicate which specific ATRIA capabilities are utilized in the use case.
  In this framework, applications supported by ATRIA can be classified as follows:

  • Applications that make use only of the NLP as a Service capabilities
  • Applications that use Generative capabilities
  • Applications that use RAG capabilities

  The difference between Generative and RAG capabilities relies solely on the definition of the preset that is associated. But the same application can make use of one, several or all of these capabilities, combining their configuration.

• Once the capabilities are selected, to set the required fields for its operation

  For example, if we want to use Generative AI for the use case resolution, firstly, certain parameters must be set corresponding to:

  • Establishing admin accesses.

  • Setting the presets that the application can use, that is, the instructions to work with the AI model, that will be defined in the application configuration.

How to configure an application

An ATRIA application must be configured with specific parameters. For this purpose, follow the guidelines Configure an application in ATRIA, using the specific parameters for the ATRIA AI-driven technology to be used.

Example of application

This is a example of an application that makes use of NLPaaS, Generative and RAG capabilities:

    {
        "id": "8832550f-f03c-4e18-bdbe-7c6fc7adf5ff",
        "name": "app",
        "disabled": false,
        "brand": "0401",
        "nlp": {
            "channelId": "1234"
        },
        "models": {
            "level": "user",
            "presets": ["atria-rag-gpt-4", "generative-preset"]
        }
    }

Applications model

Common fields

NLPaaS fields

Generative/RAG fields

4.2 - Aura Gateway API

Aura Gateway API

Introduction

aura-gateway-api is a server in charge of the access to the different AI cognitive capabilities provided by ATRIA.

A channel, service or skill can send a request through an application (entity that enables the communication with aura-gateway-api). After passing through Kernel for authentication and security purposes, aura-gateway-api sends this request to the corresponding AI-driven technology for providing the most accurate response.

Currently, aura-gateway-api enables the access to ATRIA capabilities.

Associated documentation

Descriptive technical documentation regarding aura-gateway-api includes:

• Architecture and components
• Operational overview
• Environment variables
• Aura Gateway API definition and Aura Gateway API Codebase

Guidelines for working with aura-gateway-api for the development of experiences, depending on the specific ATRIA capability:

• Technical guidelines

4.2.1 - Architecture and components

Aura Gateway API architecture and components

Technical foundations

aura-gateway-api is mainly a web server built on Typescript 4.3 using nodejs as engine. It is api-first designed, using OpenAPI v3 to provide the API definition and openapi-backend to handle swagger specification.

aura-gateway-api server is composed by several plugins, which provide different functionalities to this component. From the different capabilities that aura-gateway-api can manage, certain plugins are used by all of them and others are specific of one capability.

A channel, service, or skill uses an application to connect with aura-gateway-api following this communication protocol.

Architecture overview

The following figure shows the main technical components of the aura-gateway-api.

ConfigurationManager

ConfigurationManager is a handler for configuration, obtained through a configuration file or environment variables.

RedisConnector

RedisConnector is a handler connection to redis.

Kpis-handler

KpisHandler is the module responsible for writing the KPIs entities.

HTTP server

Microservice is implemented as an HTTP server (AuraGatewayServer) that exposes an API to receive the request to be validated.

Middlewares

The route published in the API definition file is handled by a controller but, before a request lands on its controller, it goes through a series of middlewares that provides some common steps needed by all the controllers of the server such as: request validation, common parameters extraction, logging, metrics initialization, etc.

Plugins

Different plugins provide functionality to aura-gateway-api.

Check the available ones together with detailed information in: aura-gateway-api plugins.

4.2.1.1 - Plugins

Aura gateway API plugins

Introduction

aura-gateway-api is composed of plugins, which provide functionality to this component. Plugins work independently, the same way as a service in microservices oriented architecture: isolated, self-contained and without affecting other existing functionalities in the system.

From the different types of plugins, only Api plugins are available in aura-gateway-api.

The following plugins are currently available in aura-gateway-api. From the different ATRIA capabilities that this component can manage, certain plugins are used by all or some of them and others are specific of one capability.

• aura-generative-api plugin
  This module manages request to external OpenAI APIs.

• aura-generative-process-service plugin
  This module manages the communication and data processing with external OpenAI APIs.

• aura-nlp-resolution plugin
  This module manages the request to NLP services.

• aura-nlp-resolution-process-services plugin
  This module manages the communication and data processing with external NLP services.

• aura-development-api plugin
  This module manages configuration in development environments. Only available in development environments.

• aura-feedback-api plugin
  This module manages requests to provide feedback about the generative and RAG services behavior.

• aura-agents-api plugin
  This module manages the communication with agents.

• dapr-pubsub-service plugin This plugin allows the interaction with the Dapr PubSub Service.

• aura-operations-api plugin
  This module allows you to perform operations on now, currently launching the generateDB

Discover detailed information regarding the available plugins in the left index.

Plugins management

aura-gateway-api uses the @architect/architect library for the management of plugins, so it is the architect library that is responsible for managing the dependencies injection in each module.

To create the architect application, aura-gateway-api uses the PluginManager module (located in the modules/plugin-manager folder). This module starts as the rest of modules at the aura-gateway-api start-up.

The PluginManager performs the following tasks:

• It starts the architect application with the plugins defined in plugin-config.json file, located at the root of the aura-gateway-api component.
• It adds the core modules to the IOC context. See the section plugins modules.
• It stores the information of each module defined in the plugins.

Apart from the aura-gateway-api core environment variables, each plugin can define its own specific variables. Access the document aura-gateway-api environment variables and find them in the section corresponding to your plugin.

Plugin basic structure

Currently, aura-gateway-api uses @architect/architect library for plugins management.

A basic plugin must define at least:

• A package.json file defining the library, like any other JavaScript library, with a plugin section defining which modules it consumes and supplies.
• A source code file that defines the modules that it supplies (index.ts for example).

The structure of this basic plugin is as follows:

session-api
├── index.ts
└── package.json

Plugins modules

aura-gateway-api currently adds one module that can be used by the different plugins. To use it, it is only necessary to add the package.json dependencies on plugin.consumes (like any other module/component).

• configurationManager: Module with the aura-gateway-api configuration information.
• redisConnector: Module with the aura-gateway-api Redis connection.

A plugin can provide one or more plugin modules and each plugin module can be of a different type. Each type of module is intended to add a specific functionality to aura-gateway-api.

4.2.1.1.1 - aura-generative-api plugin

aura-generative-api plugin

Introduction

The aura-generative-api plugin manages the endpoints for the communication with atria-model-gateway.

Consumes components (IOC)

Provides components (IOC)

4.2.1.1.2 - aura-generative-process-service plugin

aura-generative-process-service plugin

Introduction

The aura-generative-process-service plugin manages the communication with atria-model-gateway to allow applications to communicate with LLMs.

Consumes components (IOC)

Provides components (IOC)

4.2.1.1.3 - aura-nlp-resolution-api plugin

aura-nlp-resolution-api plugin

Introduction

The aura-nlp-resolution-api plugin manages the communication with NLP APIs to return a recognizer response.

Consumes components (IOC)

Provides components (IOC)

4.2.1.1.4 - aura-nlp-resolution-process-service plugin

aura-nlp-resolution-process-service plugin

Introduction

The aura-nlp-resolution-process-service plugin manages the communication with NLP APIs to return a recognizer response.

Consumes components (IOC)

Provides components (IOC)

4.2.1.1.5 - aura-development-api plugin

aura-development-api plugin

Introduction

aura-development-api plugin manages the configuration in development environments. Only available in development environments.

Consumes components (IOC)

Provides components (IOC)

4.2.1.1.6 - aura-feedback-api plugin

aura-feedback-api plugin

Introduction

The aura-feedback-api plugin manages the communication with atria-model-gateway to provide feedback about the Generative and RAG services behavior.

Consumes components (IOC)

Provides components (IOC)

4.2.1.1.7 - aura-agents-api plugin

aura-agents-api plugin

Introduction

The aura-agents-api plugin manages the endpoints for the communication with the agents-manager.

Consumes components (IOC)

Provides components (IOC)

4.2.1.1.8 - aura-operations-api plugin

aura-operations-api plugin

Introduction

The aura-operations-api plugin manages the endpoints for operations related to Aura.

Consumes components (IOC)

Provides components (IOC)

4.2.2 - Operational overview

Aura Gateway API operational overview

Communication protocol

aura-gateway-api communication protocol is synchronous, depending on the endpoint.

A new component comes here into play: application, defined as an entity used by a channel, service or skill to connect with aura-gateway-api.

An application has a specific configuration and will be the entry point to the aura-gateway-api flow, previous pass through Kernel, meaning that the input parameters for aura-gateway-api come from the application, and no specific information regarding the channel, service or skill is required.

4.2.3 - Environment variables

Aura Gateway API environment variables

Introduction

aura-gateway-api environment variables can be common for all plugins or specific for each of them.

Common properties

aura-development-api plugin

aura-generative-api plugin

aura-update-me-api plugin

aura-nlp-resolution-process-services plugin

| Property | Type | Description | Modifiable by OB? | |:—————-|:—————————————————————————| | AURA_COGNITIVE_ENDPOINT | string | URL of Aura NLP API. | NO. In any case, it must be the internal k8s URL pointing to the api-gw. |

aura-feedback-api plugin

aura-agents-api plugin

aura-operations-api plugin

4.2.4 - API definition

API definition for Aura Gateway API

APIs index

• Aura Gateway API
  Definition of the complete API in aura-gateway-api

• Aura Gateway Core API
  Definition of the core API in aura-gateway-api that handle its operations

• Aura Gateway Agents API
  This API is exposed in Kernel to handle agents services: https://developers.baikalplatform.com/apis/aura-aiservices/

• Aura Generative API
  This API is exposed in Kernel to handle Generative and RAG services: https://developers.baikalplatform.com/apis/aura-aiservices/

• Aura NLP Resolution API
  This API is exposed in Kernel to handle NLP as a Service: https://developers.baikalplatform.com/apis/aura-aiservices/

• Aura Feedback API
  This API is exposed in Kernel to provide feedback about the behavior of Generative and RAG services: https://developers.baikalplatform.com/apis/aura-aiservices/

• Aura Gateway Update-Me API

• Aura Gateway Development API
  Only available in development environments.

• Dapr PubSub API

• Aura Operations API
  API that include a set of endpoints related to operations over Aura.

4.2.4.1 - Aura Gateway API

Aura Gateway API

4.2.4.2 - Aura Gateway Core API

Aura Gateway Core API

4.2.4.3 - Aura Agents API

Aura Agents API

4.2.4.4 - Aura Generative API

Aura Generative API

4.2.4.5 - Aura NLP resolution API

Aura NLP resolution API

4.2.4.6 - Aura Feedback API

Aura Feedback API

4.2.4.7 - Aura Gateway Development API

Aura Gateway Development API

4.2.4.8 - Aura Operations API

Aura Operations API

4.3 - ATRIA Model Gateway

ATRIA Model Gateway

Introduction

atria-model-gateway is an ATRIA component in charge of managing the communication with different AI models.

Currently, this component receives a request from aura-gateway-api, together with other input data, and makes a call to the LLM/LMM Experiences Builder and use its capabilities.

atria-model-gateway is also in charge of security and privacy control and allows users to provide feedback on their experience.

If the selected AI model is RAG, then atria-model-gateway calls the atria-rag-server, which is in charge of executing the RAG chain and making the corresponding calls to the LLM models and databases.

The functional components of atria-model-gateway are described in the document LLM/LMM Experiences Builder

Associated documentation

Descriptive technical documentation regarding atria-model-gateway includes:

• Architecture and components
• Operational overview
• API definition

4.3.1 - Architecture and components

ATRIA Model Gateway architecture and components

Technical foundations

atria-model-gateway is responsible for managing the communication with different AI models. This component receives a request from aura-gateway-api, together with other input data, and makes a call the corresponding AI models.

If the selected AI model is RAG, then atria-model-gateway calls the atria-rag-server, which is in charge of executing the RAG chain and making the corresponding calls to the LLM models and databases.

Functional components

The functional components of atria-model-gateway are described in the document LLM/LMM Experiences Builder

Architecture overview

The following diagram schematically shows the main technical components integrated into atria-model-gateway.

A brief description of these components is included below:

Access module

Module for the management of different profiles to access atria-model-gateway.

Context module

Module in charge of the storage of a conversation history in a cache (currently, Redis is used) over a period of time, grouped by session ID. These conversations are taken into account when calling the generative LLM models.

Model manager

Module that includes the available models and presets. It is in charge of receiving the info from aura-gateway-api and calling the corresponding model.

Models

Available AI models integrated into the atria-model-gateway.

Presets

Presets are configurable entities to define the specific model to work with and certain parameters associated to it: model Id, name, description, model parameters, etc.

Constructors can use the default presets or build new ones: Go to document ATRIA configuration.

When configuring an application, all the presets that can be used for this application must be previously defined.

4.3.2 - Operational overview

ATRIA Model Gateway operational overview

Operational workflow

The operational flow between an application (for the communication with aura-gateway-apì), atria-model-gateway, atria-rag-server and atria-rag-generate-db is schematically shown in the following figure:

• Application

  • Constructors must configure an application for a channel, skill or service to communicate with aura-gateway-api.
  • In the application, the constructor must set the access grants for this application and all the presets that this application can use, from the ones configured in atria-model-gateway.
    
    

• atria-model-gateway It contains:

  • The different accesses that are defined here for each preset.
  • The available presets. Each of them is associated to an AI model with specific parameters.
  • The available AI models.
    
    

• atria-rag-server

  • When using RAG (Retrieval Augmented Generation), atria-rag-server is in charge of managing the requests made to the RAG model.
  • The available projects that contain information required for the execution of the RAG pipeline are included here.
    
    

• atria-rag-generate-db

  • atria-rag-generate-db is in charge of feeding the databases the RAG works with.
  • The available projects that contain the data required for reading the information sources and feed the databases are included here.

Configuration

atria-model-gateway includes a default configuration. Constructors can use it as is or they can modify it to be adapted to their requirements or business models: Go to document ATRIA configuration.

4.3.3 - API definition

Atria Model Gateway API definition

4.3.4 - Environment variables

ATRIA Model Gateway environment variables

Introduction

The atria-model-gateway depends on these environment variables to be set. None of them are modifiable by the OBs.

• The environment variables related to Redis are used to connect to the Redis database. This database is used to refresh the agent’s configuration, because every time the configuration of an agent is changed, it publishes this change in the corresponding channel so we can detect this change in order to refresh it.

• The environment variables related to DAPR are used to connect to the DAPR components. These components are used to refresh the agent’s configuration, because every time the configuration of an agent is changed, it publishes this change in a corresponding topic, so we can detect this change in order to refresh it.

• In order to use DAPR, it is necessary to have the DAPR pubsub component configured in the DAPR configuration file, besides the DAPR environment variables.

4.4 - ATRIA Models Manager

ATRIA Models Manager

Introduction

atria-models-manager is a ATRIA library responsible for managing all the models that we can use in the atria-model-gw model. The objective is for the atria.model-gw to only have to import the models from this component, avoiding defining the logic of the models in the atria-model-gw. For the first version, it will only feature Perplexity’s Sonar model, in the future, we will be able to migrate the atria-model-gw models here.

It contains all the logic necessary to call an LLM. Prompt construction, query tokenization, etc. It also contains all the definitions of the exceptions that an LLM can throw.

How to use it

To use the atria-models-manager, you need to install it in your Python environment and import the model into the code.

4.5 - AURA GenAI Libs

AURA GenAI Libs

Introduction

aura-genai-libs is a ATRIA that contains common components for agent development, such as logs and the langchain call to the atria_model_gw service.

Includes the following modules:

• logger module, whose purpose is to implement a callback handler to log the different stages of model, chain, and tool processing within the LangChain framework, integrating it with the AURA PyTraces logging system and the ATRIA Agents context. The module is based on the use of BaseCallbackHandler (langchain_core) and logs information at different stages of the execution cycle of language models (LLM), chats, chains, and tools. Each logged event uses m_headers.

• chat_models module, which provides a custom implementation of the ChatOpenAI class from LangChain. This custom implementation is designed to facilitate the integration with the atria_model_gw service.

How to use it

To use the aura-genai-libs, you need to install it in your Python environment and import and use the module in your code.

4.6 - ATRIA RAG Server

ATRIA RAG Server

Introduction

atria-rag-server is an ATRIA component that manages a RAG-type server. It is called by atria-model-gateway when RAG (Retrieval Augmented Generation) is used.

atria-rag-server manages the request made to the RAG model following the predefined RAG chain (pipeline) and making continuous requests combining Generative AI technology (LLMs) with semantic and lexical searches to retrieve the required information.

Associated documentation

Descriptive technical documentation regarding atria-rag-server includes:

• Architecture and components
• Operational overview

4.6.1 - Architecture and components

ATRIA RAG Server architecture and components

Architecture overview

The following diagram schematically shows the main technical components integrated into atria-rag-server.

A brief description of the technical components is included below:

Project

A project contains information required for the execution of the RAG pipeline: specific models for semantic search and lexical search; path where the documents to feed the LLMs are located; allowed file extensions, etc.

Semantic search (embeddings)

Qdrant database that stores the embeddings generated through semantic search (OpenAI embeddings) technology.

Lexical search (LLMs)

Database that stores the required documentation for making lexical searching, based on keywords.

Configuration

atria-rag-server includes a default configuration. Constructors can use it as is or they can modify it to be adapted to their requirements or business models: Go to document ATRIA configuration.

4.6.2 - Operational overview

ATRIA RAG Server operational overview

Operational workflow

The operational flow between an application (for the communication with aura-gateway-api), atria-model-gateway, atria-rag-server and atria-rag-generate-db is schematically shown in the document atria-model-gateway: operational flow.

Configuration

atria-rag-server includes a default configuration. Constructors can use it as is or they can modify it to be adapted to their requirements or business models: Go to document ATRIA configuration.

4.6.3 - Environment variables

ATRIA RAG Server environment variables

Introduction

The atria-rag-server depends on these environment variables to be set. None of them are modifiable by the OBs.

• The environment variables related to Redis are used to connect to the Redis database. This database is used to refresh the agent’s configuration, because every time the configuration of an agent is changed, it publishes this change in the corresponding channel so we can detect this change in order to refresh it.

• The environment variables related to DAPR are used to connect to the DAPR components. These components are used to refresh the agent’s configuration, because every time the configuration of an agent is changed, it publishes this change in a corresponding topic, so we can detect this change in order to refresh it.

• In order to use DAPR, it is necessary to have the DAPR pubsub component configured in the DAPR configuration file, besides the DAPR environment variables.

4.7 - ATRIA RAG generate DB

ATRIA RAG Generate DB

Introduction

atria-rag-generate-db is an ATRIA component that manages a RAG-type database. This component is launched when you want to feed the document database for the first time or when you want to update the database with new information. See more information about these processes in the guidelines Import documents into ATRIA.

atria-rag-generate-db is in charge of handling the information coming from different sources and feeding the databases the RAG works with.

Associated documentation

Descriptive technical documentation regarding atria-rag-generate-db includes:

• Architecture and components
• Operational overview

Launch atria-rag-generate-db

To launch atria-rag-generate-db, there are two suitable options:

Option 1

Send a request to the API for it to launch the atria-rag-generate-db. The endpoint responsible for this is:
/aura-services/v2/operations/data

curl -X POST "https://<your-atria-domain>/aura-services/v2/operations/data" \
-H "Content-Type: application/json"
-d '{
  "presetId": "<name of the project>"
}'

Option 2

Execute the following command to update the data in the environment. This command is in charge of launching the generation of the database for all the projects, but we can launch this generation for a specific project.

PROJECT='project-copilot-reduced'
kubectl patch configmap/atria-rag-generate-db-project --type merge -p "{\"data\":{\"ATRIA_PROJECT\":\"${PROJECT}\"}}" -n <namespace>
kubectl create job --from=cronjob/atria-rag-generate-db $(date +%Y%m%d%H%M%S)-atria-rag-generate-db-${PROJECT} -n <namespace>

(Change <namespace> by the specific one)

4.7.1 - Architecture and components

ATRIA RAG Generate DB architecture and components

Architecture overview

The following diagram schematically shows the main technical components integrated into atria-rag-generate-db.

A brief description of the technical components is included below:

Data sources

A project contains information required for the execution of the generation of the databases: specific path of documents to feed the databases, allowed file extensions, etc. It can read from different sources, this source type is defined in the extensions field.

Before the information from the documents is stored in the corresponding database, the documents are processed, e.g., they are cut up and cleaned.

Retrievers

The retrievers are in charge of reading the information from the documents and feeding the databases.

The retrievers are defined in the retrievers field of the project. Each retriever is associated with a database in order to feed or retrieve information from it.

4.7.2 - Operational overview

ATRIA RAG Generate DB operational overview

Operational flow

The operational flow between an application (for the communication with aura-gateway-api), atria-model-gateway, atria-rag-server and atria-rag-generate-db is schematically shown in the document atria-model-gateway: operational flow.

Configuration

atria-model-gateway includes a default configuration. Constructors can use it as is or they can modify it to be adapted to their requirements or business models: Go to document ATRIA configuration.

Data persistence feature

Now ATRIA enables data persistence in knowledge bases across releases: After the installation of a new release, all existing data in the knowledge base (currently, Qdrant) remains fully available and accessible for every ATRIA experience. Thus, information is completely independent of the deployed version.

This feature provides key advantages:

• Guaranteed continuity of ATRIA experiences.
• No need for data re-ingestion after each release.
• No need to recalculate embeddings.
• Data ingested after the installation of a release (through hot swapping) is now automatically consolidated and carried forward to subsequent releases.

Tracking and clean-up processes

atria-rag-generate-db keeps a record of the current state of documents and related configuration for data sources, so it only feeds documents that have been modified or added since the last update.

atria-rag-generate-db also cleans up any resources that are left behind and no longer used after new ones are introduced.

Preset management

Preset report

After generation-db is executed, a report is logged with the following information for each preset:

• The preset name.
• The status of the execution (success, skipped or error).
• A descriptive message with the reason for the status.
• Date and time of the execution start.
• Date and time of the execution end.
• The configured documents for the preset

Preset availability

When a new preset is created, it is necessary to launch the database generation process by executing the atria-rag-generate-db component. This process may take several minutes to complete. Once the generation is finished, the atria-rag-generate-db component is automatically restarted.

While these processes are running, a message is shown to the user indicating that the preset is not yet available.

When both processes are finished, the preset becomes available for use.

Data migration between ATRIA releases

The data persistence feature is implemented by a migration tool between environments or releases integrated in the atria-rag-generate-db component. This tool moves the trained data from one release to another, to avoid generating preset data that has been previously created in a release.

The process for migrating data must be triggered manually by launching a command (similar to the aura-rag-generate-db job), where both source and target environments should be indicated.

After executing this command, data will be migrated from one environment to the other automatically.

The migration flow is executed as follows:

1. Process the hashes file and, for each preset we want to migrate, we will do the following steps:
   • Check that the preset from the source environment is in the config of the target environment
   • Move the trained_data files from the source environment to the respective training folder of the target environment
   • Duplicate the collections from the source environment to the target environment
   • Move the TFIDFs files from the source environment to the target environment
2. Move the hashes file from the source environment to the target environment
3. Add the new presets training files to the respective training folder in the target environment.
4. Launch atria-rag-generate-db. Only new presets will be reloaded.

Data migration flow

In the migration process described above, the following folders are generated and stored in an Azure blob storage after atria-rag-generate-db is finished:

Shared data

This folder contains the trained data shared between atria-rag-server and atria-rag-generate-db. This is used to store the files that the atria-rag-generate-db generates and then the atria-rag-server uses to be able to process the request.

At the moment, only the files generated by the TFIDF (Term Frequency–Inverse Document Frequency) exist in this folder.

This folder is used for migration, as we can take the TFIDFs of a trained preset to the blob of a specific release where that preset has not been trained and save the training afterward.

Trained data

This folder contains the files that have been used in the atria-rag-generate-db for each preset.

The folder structure is defined with a hash of the contents of all the files for each preset, to facilitate migration.

Atria RAG project hashes

This is a file containing all the information for each preset, to facilitate migration.

It contains the following information for each preset:

• config_hash: Hash of the preset configuration at the time the atria-rag-generate-db was launched.

• source_files_hash: Hash of the source files used to generate the preset. This hash should exist in one folder into the trained data folder.

• metadata: Metadata of the preset, including the date of atria-rag-generate-db launching.

• retrievers: Info that retrievers used to generate the preset. It contains the name of the Qdrant collection and the path where it holds the TFIDF files, which would correspond to the shared data.

  {
    "5905dece-433d-47f4-a78c-72366bcd1473": {
      "config_hash": "28f837d56079f30c59a419292d129bc3",
      "source_files_hash": "cda3afcd8e74ede0d23065e897d55fae",
      "metadata": {
        "date": "2025-04-01 11:25:59"
      },
      "retrievers": {
        "qdrant_collection_name": "rag-ap-eight-9100-dev-project-copilot",
        "tfidf_path_file": "project-copilot/tfidf"
      }
    }
  }
  

In addition to using this data for migration, it also speeds up the launch of the atria-rag-generate-db.

The config_hash and source_files_hash values are used to verify if, at the moment of launching the atria-rag-generate-db, something has been changed in the configuration or in the training data. If changes are detected, all the data for that preset is regenerated. Otherwise, if the preset has not changed, we will save that generation.

Launch migration process

The process to persist data between releases has to be launched manually through the execution of the following command: To run this script, we just need the output files with the environment configuration info generated by the installer in the output_install directory from the source and destination environment. With this info, run the script as shown below, using the corresponding files names for the desired environment:

  ./migrate-data --source-file ${SOURCE_ENVIRONMENT_INFO_FILE} --dest-file ${DEST_ENVIRONMENT_INFO_FILE}

Where:

• source-file: Source environment info file where the data is stored.
• dest-file: Target environment info file where the data is going to be migrated.

4.8 - Agents Manager

Agents Manager

Introduction

agents-manager is a server in charge of managing and orchestrating access to the different agents within the AURA ecosystem.

The main purpose of agents-manager is to facilitate orchestration between the different agents registered in ATRIA, also to enrich the conversation by managing the message history, before sending the requests to the agent responsible for their processing.

Associated documentation

Descriptive technical documentation regarding agents-manager includes:

• Architecture and components
• Communication protocol
• Environment variables
• Agents Manager API definition and Agents Manager Codebase

4.8.1 - Architecture and components

Agents Manager architecture and components

Technical foundations

agents-manager is mainly a web server built on Typescript using nodejs as engine. It is api-first designed, using OpenAPI v3 to provide the API definition and openapi-backend to handle swagger specification.

agents-manager server is composed by several plugins, which provide different functionalities to this component.

A channel, service, or skill uses an application to connect with agents-manager following this communication protocol.

Architecture overview

The following figure shows the main technical components of the agents-manager, which are described below.

agents-manager components

ConfigurationManager

ConfigurationManager is a handler for configuration, obtained through a configuration file or environment variables.

RedisConnector

RedisConnector is a handler connection to Redis.

HTTP server

Microservice is implemented as an HTTP server (AuraServer) that exposes an API to receive the request to be validated.

Middlewares

The route published in the API definition file is handled by a controller but, before a request lands on its controller, it goes through a series of middlewares that provides some common steps needed by all the controllers of the server such as: request validation, common parameters extraction, logging, metrics initialization, etc.

Controllers and Services

agents-manager is composed of plugins, which provide functionality and uses specific modules from aura-configuration-api.

Check the available plugins together with detailed information in: agents-manager plugins.

4.8.1.1 - Plugins

Agents Manager API plugins

Introduction

agents-manager is composed of plugins, which provide functionality to this component. Plugins work independently, the same way as a service in microservices oriented architecture: isolated, self-contained and without affecting other existing functionalities in the system.

From the different types of plugins, only Api plugins are available in agents-manager.

The following plugins are currently available in agents-manager. Certain plugins are used by all or some of the other plugins and others are specific for one capability.

• agent-dispatcher-api plugin
  This plugin manages request to Agent Dispatcher API.

• development-api plugin
  This plugin manages configuration in development environments. Only available in development environments.

• configuration-service plugin
  This plugin manages requests to Agent Configuration.

• redis-connector-service plugin
  This plugin manages requests to Redis.

• session-service plugin
  This plugin manages the sessions.

• agent-deployment plugin, that provides deployment functionalities for agents within the agents-manager. Plugin currently deactivated.

Discover detailed information regarding the available plugins in the left index.

Plugins management

agents-manager uses the @architect/architect library for the management of plugins, so it is the architect library that is responsible for managing the dependencies injection in each module.

To create the architect application, agents-manager uses the PluginManager module (located in the modules/plugin-manager folder). This module starts as the rest of modules at the agents-manager start-up.

The PluginManager performs the following tasks:

• It starts the architect application with the plugins defined in plugin-config.json file, located at the root of the agents-manager component.
• It adds the core modules to the IOC context. See the section plugins modules.
• It stores the information of each module defined in the plugins.

Apart from the agents-manager core environment variables, each plugin can define its own specific variables. Access the document agents-manager environment variables and find them in the section corresponding to your plugin.

Plugin basic structure

Currently, agents-manager uses @architect/architect library for plugins management.

A basic plugin must define at least:

• A package.json file defining the library, like any other JavaScript library, with a plugin section defining which modules it consumes and supplies.
• A source code file that defines the modules that it supplies (index.ts for example).

The structure of this basic plugin is as follows:

session-api
├── index.ts
└── package.json

Plugins modules

agents-manager currently adds one module that can be used by the different plugins. To use it, it is only necessary to add the package.json dependencies on plugin.consumes (like any other module/component).

• configurationManager: Module with the agents-manager configuration information.
• redisConnector: Module with the agents-manager Redis connection.

A plugin can provide one or more plugin modules and each plugin module can be of a different type. Each type of module is intended to add a specific functionality to agents-manager.

4.8.1.1.1 - agent-dispatcher-api plugin

agent-dispatcher-api plugin

Introduction

agent-dispatcher-api plugin manages and dispatches the requests to the agent API.

Consumes components (IOC)

Provides components (IOC)

4.8.1.1.2 - Agent Deployment Plugin

Agent Deployment Plugin

Introduction

The agent-deployment plugin is designed to manage the deployment of agents in a flexible and scalable manner. It allows multiple agents to be deployed within a single microservice, and the same agent can be deployed with different configurations across different microservices or within the same microservice.

Plugin Management

The agent-deployment plugin uses the @architect/architect library for managing dependencies and plugin lifecycle. The PluginManager module, located in the modules/plugin-manager folder, is responsible for initializing the plugin at the start-up of the agents-manager.

The PluginManager performs the following tasks:

• Initializes the architect application with the plugins defined in the plugin-config.json file, located at the root of the agents-manager component.
• Adds core modules to the IOC context. See the section plugin modules.
• Stores information about each module defined in the plugins.

Plugin Basic Structure

A basic plugin must define at least:

• A package.json file defining the library, with a plugin section specifying which modules it consumes and supplies.
• A source code file that defines the modules it supplies (index.ts for example).

The structure of this basic plugin is as follows:

agent-deployment-plugin
├── index.ts
└── package.json

Example index.ts

import { PluginType, registerPlugin } from '@telefonica/agents-manager-common';
import { Services } from './deployment-consume-services';

export = registerPlugin([
    {
        type: PluginType.Service,
        name: 'agentDeploymentService',
        instance: {
            deployAgent(agentConfig) {
                // Implementation for deploying an agent
            }
        },
        services: Services
    }
]);

Example package.json

{
    "name": "@telefonica/agent-deployment-plugin",
    "version": "1.0.0",
    "main": "index.js",
    "private": true,
    "plugin": {
        "consumes": [
            "configurationManager"
        ],
        "provides": [
            "agentDeploymentService"
        ]
    }
}

Plugin Modules

The agent-deployment plugin can utilize the following modules provided by the agents-manager:

• configurationManager: Module with the agents-manager configuration information.
• redisConnector: Module with the agents-manager Redis connection.

A plugin can provide one or more plugin modules, and each plugin module can be of a different type. Each type of module is intended to add specific functionality to the agents-manager.

4.8.1.1.3 - development-api plugin

development-api plugin

Introduction

The development-api plugin manages the configuration in development environments. Only available in development environments.

Consumes components (IOC)

Provides components (IOC)

4.8.1.1.4 - configuration-service plugin

configuration-service plugin

Introduction

The configuration-service plugin manages the configuration of agents.

Consumes components (IOC)

Provides components (IOC)

4.8.1.1.5 - redis-connector-service plugin

redis-connector-service plugin

Introduction

The redis-connector-service plugin manages requests to Redis.

Consumes components (IOC)

Provides components (IOC)

4.8.1.1.6 - session-service plugin

session-service plugin

Introduction

The session-service plugin manages the sessions.

Consumes components (IOC)

Provides components (IOC)

4.8.2 - Communication protocol

Agents Manager communication protocol

Communication protocol

The agents-manager uses a synchronous communication protocol, although the specific behavior may vary depending on the endpoint. The following diagram illustrates the protocol:

agents-manager communication protocol

4.8.3 - Environment variables

Agents Manager environment variables

Introduction

agents-manager environment variables can be common for all plugins or specific for each of them.

Common properties

development-api plugin

configuration-service plugin

redis-connector-service plugin

session-service plugin

4.8.4 - API definition

API definition for Agents Manager

APIs index

• Agents Manager API

• Agents Manager Development API
  Only available in development environments.

• Agents Manager Dispatcher API

• Dapr PubSub API

4.8.4.1 - Agents Manager API

Agents Manager API

4.8.4.2 - Agents Manager Development API

Agents Manager Development API

4.8.4.3 - Agents Manager Dispatcher API

Agents Manager Dispatcher API

4.9 - Agents Server

Agents Server

Introduction

The agents-server component is a microservice that provides the capability to create server API and manage agents. It is responsible for executing the agent’s tasks.

To launch the agents-server component, you need install the agents package you prefer to use and deploy the server using this agents package.

Associated documentation

Descriptive technical documentation regarding agents-server includes:

• Architecture and components
• Environment variables
• Agents Manager API definition

4.9.1 - Architecture and components

Agents Server architecture and components

Technical foundations

The agents-server component will start a server that listens for incoming requests and executes the agent’s tasks based on the received input.

The server can be configured to use an agent by specifying the agent package name.

This server creates a REST API that can be used to interact with different agents. The API allows sending requests to the agent and receiving responses.

Within the agents-server, the agents package component enables information processing tasks to be performed and acted upon to achieve specific objectives. This information can come from a database and respond to the user’s request based on that acquired information.

Architecture overview

The following figure shows the main technical components of the agents-server, which are described below.

agents-server components

HTTP server

Microservice is implemented as an HTTP server (fastapi) that exposes an API to receive the request to be validated.

Config Api Handler

The agents-server uses the aura-configuration-api to get the configuration of the agent. This handler is responsible for retrieving the configuration of the agent from the aura-configuration-api and providing it to the server.

ConfigurationManager

ConfigurationManager is a handler for configuration, obtained through a configuration file or environment variables.

Agents Package

The server is configured to use the agent of a specific agent package name. This package contains the agent’s code and its dependencies. The server will load the agent package and use it to execute the agent’s tasks.

Event Subscribers

Event subscribers are implementations for managing event registration and handling in the agents-server. They allow the server to listen for events and trigger actions based on those events.

Redis Async Subscriber

RedisAsyncSubscriber is a handler for subscribing to Redis channels asynchronously. It allows the server to receive messages from Redis and process them in real time.

DAPR Async Subscriber

DaprAsyncSubscriber is a handler for subscribing to DAPR topics asynchronously. It allows the server to receive messages from DAPR and process them in real time.

4.9.2 - Environment variables

Agents server environment variables

Introduction

The agents-server depends on these environment variables to be set. None of them are modifiable by the OBs.

• The environment variables related to Redis are used to connect to the Redis database. This database is used to refresh the agent’s configuration, because every time the configuration of an agent is changed, it publishes this change in the corresponding channel so we can detect this change in order to refresh it.

• The environment variables related to DAPR are used to connect to the DAPR components. These components are used to refresh the agent’s configuration, because every time the configuration of an agent is changed, it publishes this change in a corresponding topic, so we can detect this change in order to refresh it.

• In order to use DAPR, it is necessary to have the DAPR pubsub component configured in the DAPR configuration file, besides the DAPR environment variables.

4.9.3 - API definition

Agents Server API definition

4.10 - ATRIA APIs documentation

ATRIA APIs documentation

ATRIA APIs index

• aura-gateway-api API definition

• atria-model-gateway API definition

5 - ATRIA operational workflows

ATRIA operational workflows

Index of technical operational workflows

• NLP as a Service operational workflow

• Generative API operational workflow

• General RAG operational workflow

• Germany ATRIA General RAG operational workflow

• Feedback capability operational workflow

• Reload config by Redis events

5.1 - NLP as a Service

NLP as a Service operational workflow

Operational flowchart

The sequence diagram of the process executed by the NLP Apps capability is shown below:

@startuml
title NLP resolution API diagram
participant Application
participant Kernel #1add4d
participant AuraGatewayApi #76bbe7
Application -> Kernel: Create two-legged token with scope aura-ai-services:nlp-messaging:write
Note right of Kernel: this token needs refreshing
Kernel -> Application: Response two-legged token
Application -> Kernel: Request to aura-aiservices/nlp/query with token
Kernel -> AuraGatewayApi: Request to aiservices/nlp/query with token-info header
AuraGatewayApi -> AuraGatewayApi: Validate request
AuraGatewayApi -> AuraNLPApp: Request recognition
AuraGatewayApi -> AuraGatewayApi: generate response
AuraGatewayApi -> Kernel: response 200 and message
Kernel -> Application: response 200 and message
@enduml

NLP Apps operational sequence diagram

5.2 - Agent AI operational workflow

Agent AI operational workflow

Operational flowchart

The sequence diagram of the process executed by the agents AI capability is shown below:

@startuml
title Agent API diagram
participant Application
participant Kernel #1add4d
participant AuraGatewayApi #76bbe7
participant AgentsManager #0796f5
participant MongoDeviceRecommender #11f5cf
participant AtriaModelGateway #f58e11
participant AzureOpenAI #9476e7
Application -> Kernel: Create two-legged token with scope aura-ai-services:agents-messaging:write
Note right of Kernel: this token needs refreshing
Kernel -> Application: Response two-legged token
Application -> Kernel: Request to aura-aiservices/agents/messages with token and correlatorId (x-correlator)
Kernel -> AuraGatewayApi: Request to aiservices/agents/messages with token-info header and correlatorId
AuraGatewayApi -> AuraGatewayApi: Validate request
AuraGatewayApi -> AgentsManager: Send Request to an agent
AgentsManager -> AgentsManager: Check if the context exists && retrieves it
AgentsManager -> AgentsManager: routing to the agents
AgentsManager -> MongoDeviceRecommender: Send Request to an agent with the context
MongoDeviceRecommender -> MongoDeviceRecommender: process the Request
group Agents process
  loop XX times
    MongoDeviceRecommender -> AtriaModelGateway: Send prompt to atria-model-gateway
    AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
    AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
    AtriaModelGateway -> MongoDeviceRecommender: Response prompt
    MongoDeviceRecommender -> MongoDeviceRecommender: Analyze the response
  end
end
MongoDeviceRecommender -> AgentsManager: response 200 and message
AgentsManager -> AgentsManager: Store the context
AgentsManager --> AuraGatewayApi: response 200 and message
AuraGatewayApi -> AuraGatewayApi: process response
AuraGatewayApi -> AuraGatewayApi: generate response
AuraGatewayApi -> Kernel: response 200 and message
Kernel -> Application: response 200 and message
@enduml

5.3 - Generative AI

Generative AI operational workflow

Operational flowchart

The sequence diagram of the process executed by the Generative AI capability is shown below:

@startuml
title Generative API diagram
participant Application
participant Kernel #1add4d
participant AuraGatewayApi #76bbe7
participant AtriaModelGateway #f58e11
participant AzureOpenAI #9476e7
Application -> Kernel: Create two-legged token with scope aura-aiservices:messaging:write
Note right of Kernel: this token needs refreshing
Kernel -> Application: Response two-legged token
Application -> Kernel: Request to aura-aiservices/generative/prompts with token and correlatorId (x-correlator)
Kernel -> AuraGatewayApi: Request to aiservices/generative/prompts with token-info header and correlatorId
AuraGatewayApi -> AuraGatewayApi: Validate request
AuraGatewayApi -> AuraGatewayApi: Generate prompt
AuraGatewayApi -> AtriaModelGateway: Send prompt to atria-model-gateway
AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
AtriaModelGateway -> AuraGatewayApi: Response prompt
AuraGatewayApi -> AuraGatewayApi: process atria-model-gateway response
AuraGatewayApi -> AuraGatewayApi: generate response
AuraGatewayApi -> Kernel: response 200 and message with session_id
Kernel -> Application: response 200 and message with session_id
@enduml

5.4 - General RAG

ATRIA General RAG operational workflow

Flow diagram

Calls to the atria-rag-server component (AtriaRAG in the sequence diagram) executes the predefined RAG chain General RAG.

@startuml
title RAG API diagram
participant Application
participant Kernel #1add4d
participant AuraGatewayApi #76bbe7
participant AtriaModelGateway #f58e11
participant AtriaRAG #f5de11
participant AzureOpenAI #9476e7

Application -> Kernel: Create two-legged token with scope aura-aiservices:messaging:write
Note right of Kernel: this token needs refreshing
Kernel -> Application: Response two-legged token
Application -> Kernel: Request to aura-aiservices/generative/prompts with token and correlatorId (x-correlator)
Kernel -> AuraGatewayApi: Request to aiservices/generative/prompts with token-info header and correlatorId
AuraGatewayApi -> AuraGatewayApi: Validate request
AuraGatewayApi -> AuraGatewayApi: Generate prompt
AuraGatewayApi -> AtriaModelGateway: Send prompt to atria-model-gateway
activate AtriaModelGateway
AtriaModelGateway -> AtriaRAG: 1.0: Enrich request 
activate AtriaRAG
AtriaRAG -> AtriaRAG: securityStg

opt translateStg.enabled == true
    AtriaRAG -> AtriaRAG: 1.1: Translate user query 
    AtriaRAG -> AtriaModelGateway: Send request to LLM 
    AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
    AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
    AtriaModelGateway --> AtriaRAG: LLM response with translated query
end
opt cleanStg.enabled == true
    AtriaRAG -> AtriaRAG: 1.2: Clean the user query 
    AtriaRAG -> AtriaModelGateway: Send request to LLM 
    AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
    AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
    AtriaModelGateway --> AtriaRAG: LLM response with new cleaned query

end
opt contextStg.enable == true
    alt Ask LLM
        AtriaRAG -> AtriaModelGateway: 1.3: Request LLM to validate the conversational context
        AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
        AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
        AtriaModelGateway --> AtriaRAG: LLM response [SAME CONTEXT] or [DIFFERENT CONTEXT]
        AtriaRAG -> AtriaRAG: Recreate Query
    end
    alt Recreate Query 
        AtriaRAG -> AtriaModelGateway: 1.4: Call LLM to generate new question 
        AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
        AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
        AtriaModelGateway --> AtriaRAG: Response with new question
    end
end

AtriaRAG -> AtriaRAG: retrievalStg

opt postFilteringStg.enable == true
    AtriaRAG -> AtriaRAG: Post Filtering 
    note right: Batch request
    AtriaRAG -> AtriaModelGateway: 1.5: Request LLM for each chunk 
    AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
    AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
    AtriaModelGateway --> AtriaRAG: LLM response [RELEVANT] or [IGNORABLE]
end

AtriaRAG -> AtriaModelGateway: 1.6: Request LLM generativeStg 
AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI 
AtriaModelGateway --> AtriaRAG: LLM response
AtriaRAG --> AtriaModelGateway: 2: Final response 
deactivate AtriaRAG
deactivate AtriaModelGateway

AtriaModelGateway -> AuraGatewayApi: Response Model Gateway
AuraGatewayApi -> AuraGatewayApi: process atria-model-gateway response
AuraGatewayApi -> AuraGatewayApi: generate response
AuraGatewayApi -> Kernel: response 200 and message with session_id
Kernel -> Application: response 200 and message with session_id

@enduml

5.5 - Germany General RAG

Germany ATRIA General RAG operational workflow

Flow diagram

Calls to the atria-rag-server component (AtriaRAG in the sequence diagram) executes the predefined RAG chain General RAG.

@startuml
title Germany RAG API diagram
participant Application
participant Kernel #1add4d
participant AuraGatewayApi #76bbe7
participant AtriaModelGateway #f58e11
participant AtriaRAG #f5de11
participant AzureOpenAI #9476e7

Application -> Kernel: Create two-legged token with scope aura-aiservices:messaging:write
Note right of Kernel: this token needs refreshing
Kernel -> Application: Response two-legged token
Application -> Kernel: Request to aura-aiservices/generative/prompts with token and correlatorId (x-correlator)
Kernel -> AuraGatewayApi: Request to aiservices/generative/prompts with token-info header and correlatorId
AuraGatewayApi -> AuraGatewayApi: Validate request
AuraGatewayApi -> AuraGatewayApi: Generate prompt
AuraGatewayApi -> AtriaModelGateway: Send prompt to atria-model-gateway
activate AtriaModelGateway
AtriaModelGateway -> AtriaRAG: 1.0: Enrich request 
activate AtriaRAG
AtriaRAG -> AtriaRAG: securityStg


    alt Ask LLM
        AtriaRAG -> AtriaModelGateway: 1.3: Request LLM to validate the conversational context
        AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
        AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
        AtriaModelGateway --> AtriaRAG: LLM response [SAME CONTEXT] or [DIFFERENT CONTEXT]
        AtriaRAG -> AtriaRAG: Recreate Query
        AtriaRAG -> AtriaModelGateway: 1.4: Call LLM to generate new question 
        AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
        AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
        AtriaModelGateway --> AtriaRAG: Response with new question
    end


AtriaRAG -> AtriaRAG: retrievalStg

AtriaRAG -> AtriaRAG: Post Filtering 
note right: Batch request
AtriaRAG -> AtriaModelGateway: 1.5: Request LLM for each chunk 
AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI
AtriaModelGateway --> AtriaRAG: LLM response [RELEVANT] or [IGNORABLE]


AtriaRAG -> AtriaModelGateway: 1.6: Request LLM generativeStg 
AtriaModelGateway -> AzureOpenAI: Send Request to ChatCompletation endpoint
AzureOpenAI --> AtriaModelGateway: Response from AzureOpenAI 
AtriaModelGateway --> AtriaRAG: LLM response
AtriaRAG --> AtriaModelGateway: 2: Final response 
deactivate AtriaRAG
deactivate AtriaModelGateway

AtriaModelGateway -> AuraGatewayApi: Response Model Gateway
AuraGatewayApi -> AuraGatewayApi: process atria-model-gateway response
AuraGatewayApi -> AuraGatewayApi: generate response
AuraGatewayApi -> Kernel: response 200 and message with session_id
Kernel -> Application: response 200 and message with session_id

@enduml

5.6 - Ingestion Process Automation operational workflow

Ingestion Process Automation

Flow diagram

Flow of calls made to launch the generate-db process.

@startuml
title Ingestion Process Automation Flow

' Define participants with themed colors and clear names
actor User

participant "Azure Blob Storage" as AzureStorage #A2C4E0
participant "Gateway API" as GatewayAPI #bfb1f2
participant "Config Watcher" as ConfigWatcher #f296ee
participant "Deployment API" as DeploymentAPI #f77cbc
participant "Generate DB Process" as GenerateDBProcess #D9EAD3


' === Upload Files Stage ===
User -> AzureStorage : Upload training files
AzureStorage --> User : Response 200 OK

' === Launch generate-db ===
User -> GatewayAPI : Request to /aura-services/v2/operations/data to launch ingestion process
GatewayAPI -> ConfigWatcher : Request to Config Watcher
ConfigWatcher -> GenerateDBProcess : Start generate-db process
GenerateDBProcess --> ConfigWatcher : Response 200 OK
ConfigWatcher --> GatewayAPI : Response 200 OK
GatewayAPI --> User : Response 200 OK


' === Processing Stage ===
GenerateDBProcess -> AzureStorage : Read training files
AzureStorage --> GenerateDBProcess : Response 200 OK
GenerateDBProcess -> GenerateDBProcess : Processing training files

' === Logs Querying ===
... Logging queries can occur anytime ...


User -> GatewayAPI : Request to /aura-services/v2/operations/data/{presetId}/logs
GatewayAPI -> ConfigWatcher : Request to get logs
ConfigWatcher -> DeploymentAPI : Response 200 OK
DeploymentAPI --> ConfigWatcher : Response 200 OK
ConfigWatcher --> GatewayAPI : Response 200 OK
GatewayAPI --> User : Response 200 OK

' === Status Query ===
... Status queries can occur anytime ...

' === Status process ===
User -> GatewayAPI : Request to /aura-services/v2/operations/data/{presetId}/status to get status
GatewayAPI -> ConfigWatcher : Request to get status
ConfigWatcher -> DeploymentAPI : Response 200 OK
DeploymentAPI --> ConfigWatcher : Response 200 OK
ConfigWatcher --> GatewayAPI : Response 200 OK
GatewayAPI --> User : Response 200 OK


@enduml

5.7 - Reload config by Redis events

Reload config by Redis event in RAG and Model Gateway

Introduction

Once a preset or application configuration has been modified, a Redis event is automatically triggered to the PresetConfiguration and ApplicationConfiguration channels.

The components atria-model-gateway and atria-rag-server are subscribed to these channels and once an event arrives, they launch the configuration update process.

This process is transparent to the user.

5.8 - Feedback capability

Feedback capability operational workflow

Flow diagram

@startuml
title Feedback API diagram
participant Application
participant Kernel #1add4d
participant AuraGatewayApi #76bbe7
Note right of Application: The application has made a previous request to `aiservices/generative/prompts` on which it will give feedback. Use the correlatorId sending in the request and the session_id received in the response. View Generative API diagram
Application -> Kernel: Request to aiservices/{session_id}/feedback with token and msg_corrId: correlatorId
Kernel -> AuraGatewayApi: Request to aiservices/{session_id}/feedback with token-info header and msg_corrId: correlatorId
AuraGatewayApi -> AuraGatewayApi: Validate request
AuraGatewayApi -> AuraGatewayApi: Covert message to format atria-model-gateway
AuraGatewayApi -> AtriaModelGateway: Send feedback to atria-model-gateway
AtriaModelGateway -> AuraGatewayApi: Response feedback
AuraGatewayApi -> AuraGatewayApi: process atria-model-gateway response
AuraGatewayApi -> AuraGatewayApi: generate response
AuraGatewayApi -> Kernel: response 204
Kernel -> Application: response 204


@enduml

Request

curl --location 'https://api.environment.baikalplatform.com/aura-aiservices/v1/{session_id}/feedback' \
--header 'x-correlator: <uuid2>' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: Bearer {token}' \
--data '{
  "application": {    
    "name": "app-name",    
  },
  "value": true,  
  "msg_corrId": "{previous-x-correlator}"
}'

6 - Technical guidelines

ATRIA technical guidelines

Index of ATRIA technical guidelines

• Build e2e experiences
• Publish an API in Kernel
• Configure an application
• Hot swapping of applications
• ATRIA configuration
• Get Kernel access token
• Request to Aura NLP Resolution API
• Request to Aura Generative API
• Use ATRIA web interface (aura-manager)
• ATRIA error management
• Adjust timeouts in ATRIA

Other technical guidelines

These guidelines are common to ATRIA and Aura Virtual Assistant

• Deploy Aura: Deployment Task Center

• Develop and operate ATRIA: Developers Workspace

6.1 - Build e2e experiences

Build end-to-end experiences in ATRIA

Introduction

In order to leverage the available ATRIA capabilities within a use case development, specific technical tasks must be performed by different Aura teams.

The current document provides a schematic overview of the end-to-end workflow and links to the corresponding technical guidelines tailored to each team responsible for specific aspects of the project.

Build experiences that call Generative AI

Build experiences that use General RAG

Build experiences that call NLP Apps

Build experiences that call Semantic Search

6.1.1 - Build experiences that call Generative AI

Build experiences that call Generative AI

Introduction

Generative AI in Aura benefits from the auto-generative capabilities of Azure OpenAI GPT models for an accurate understanding of requests and the generation of highly reliable answers.

Steps in the process

a. Prerequisites: Install and enable

b. Build experience

6.1.2 - Build experiences that use General RAG

Build experiences that use General RAG

Introduction

General RAG capability enables the implementation of RAG (Retrieval Augmented Generation) techniques to surpass the capabilities of LLMs in the development of generic questions use cases (based on FAQs).

Steps in the process

a. Prerequisites: Install and enable

b. Build experience

6.1.3 - Build experiences that call NLP apps

Build experiences that call NLP apps

Introduction

Within NLP as a Service, the NLP Apps capability enables channels, services or skills to connect with Aura cognitive capabilities for sending a request expressed in natural language and receiving back an accurate response via API, without the need for a conversational bot.

Steps in the process

a. Prerequisites: Install and enable

b. Configure

c. Build & test

6.1.4 - Build experiences that call Semantic Search

Build experiences that call Semantic Search

Introduction

Within [NLP as a Service], the Semantic Search capability enables the use of Azure OpenAI embeddings for the development of generic questions experiences (grounded in FAQs).

Steps in the process

a. Prequisites: Install and enable

b. Configure

c. Build & test

6.2 - Publish an API in Kernel

Publish an API in Kernel

Guidelines

As a prerequisite for building an experience in ATRIA, the aura-gateway-api API must be published in Kernel.

For this purpose, follow the instructions below:

• Request Kernel team to configure in the corresponding Kernel environment the scopes needed to call this API:

  • aura-ai-services:messaging:write: Permission to send Generative / RAG and feedback messages to Aura.
  • aura-ai-services:nlp-messaging:write: Permission to send NLP as a Service messages to Aura.
    
    

• Access from Kernel to aura-gateway-api will be done by APIKey. It is necessary to create this APIKey in each environment following these instructions Generate an APIKey.
  
  

• Request Kernel team to configure in the corresponding Kernel environment this API Aura AI Services.

  • For that, the following settings are needed:

    • URL: https://{{aura-services-environment}}.auracognitive.com/aura-services/v2/aiservices/

    • Authorization header: APIKEY {{api-key}}

      Where:

      • {{aura-services-environment}} should look like svc-[country]-[environment], for instance svc-es-pre
      • {{api-key}} is a specific APIKey created for Kernel to access this endpoint. This APIKey must be requested to the team in charge of operating the corresponding Aura environment:
        • Aura Global team for development and staging environments
        • GES for certifications, pre-production and production environments
          
          

• This process must be executed in each Kernel deployment.