Aura – aura-groot

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Channeldata validator middleware

Description of channeldata-validator-middleware, in charge of the validation of the request channelData for normalized versions

Introduction

The source code of this middleware is included in this Github repository.

This middleware will check if the channelData of the incoming request is valid or not. This validation is made:

If the environment variable AURA_CHANNELDATA_VALIDATION is set to true.
If the incoming channelData version is greater or equal than AURA_CHANNELDATA_VALIDATION_MIN_VERSION.
If a validator for this version exists. Validators are loaded using schemas stored in aura-models package.

If the validation is ok or is not executed because one or more of the previous conditions are not satisfied, aura-groot continues the normal flow, executing the rest of middlewares.

If the validation fails, the error message core:core.model-validation.request.error will be returned to the user and no more middlewares will be executed.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura development architecture

Description of the Aura Virtual Assistant development architecture, its operation and main components

Introduction

A bot is an application, particularly an HTTP server, that allows interaction with users in a conversational way. Different means of interaction are available, such as text, cards with images, video or audio and speech.

Users interact with the bot through a given channel, that is the final application used directly by the users. For instance, the bot can be accessible from Facebook, WhatsApp or from any private application, such as My O2 or Mi Movistar. Every message exchanged between the user and the bot is called an activity.

In Aura distributed architecture, the user interacts directly with the root bot, here named aura-groot, and the root bot delegates some of its conversational logic to the skill aura-bot.

Skills and root bot communicate over HTTP using the Bot Framework protocol.

Both aura-bot and aura-groot are separate bots and are published independently.

Main technical components

The key Aura technical components that executes all the functionalities of Aura Virtual Assistant are defined below.

Take into account the mapping between the functional architecture with development architecture:

aura-groot
aura-groot is the main component in charge of handling the assistant. It is responsible for the communication between each channel and its corresponding skill, keeping track of all the communication process, but introducing a minimal interference on it.

Access detailed technical documentation regarding aura-groot component
aura-bot
aura-bot is a skill based on Microsoft Bot Framework that constitutes Aura’s neuronal network. It handles the requests received from aura-groot and interacts with every Aura system or component in order to provide Aura users with the intended answer or requested service.

Access detailed technical documentation regarding aura-bot component

Operational flow

The main stages in tasks in Aura Virtual Assistant conversational process are described below and shown in the following flowchart.

aura-groot adapter receives activities from the user and forwards them to the aura-groot activity handler. (Activities from the user are received at the aura-groot messaging endpoint.)
The aura-groot uses a skill HTTP client to send an activity to the skill. The client gets the consumer-skill conversation information from a skill definition and a skill conversation ID factory. This includes the service URL that the skill will use to reply to the activity.
The aura-bot adapter receives activities from aura-groot and forwards them to the skill’s activity handler. (Activities from the consumer are received at the skill bot’s messaging endpoint).
When aura-bot responds, the aura-groots skill handler receives the activity. It gets the root-user conversation information from the skill conversation ID factory. It then forwards the activity to the aura-groot’s adapter. (Activities from the skill are received at the aura-groot’s skill host endpoint).
The aura-groot adapter sends messages from the skill to the user.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Domain recognizer

Description of domain-recognizer in charge of recognizing the user’s domain

Introduction

domain-recognizer is the aura-groot recognizer in charge of identifying the domain in the user’s input request.

This is possible thanks to the mapping of the list of channels and skills that is done at server start-up. A skill can have several associated channels, but a channel can only have one associated skill.

Example:

GOOD
channel1 => skill1
channel2 => skill1
channel3 => skill2

BAD
channel1 => skill1
channel2 => skill2
channel2 => skill1 -> This is not possible

When a user writes through a channel:

The domain to which the channel belongs is identified
The skill associated with that domain is executed
The domain is registered in KPIs

If the recognition of the domain fails, the user will be returned the error message root:skill.not.found.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Groot environment variables

Updated list of aura-groot environment variables

Properties marked in bold are mandatory.
Properties marked in italics are optional.

Property	Type	Description	Modifiable by OB?
AURA_ACTIVATE_MINI_BOT	boolean	Flag to activate minibot mode. By default: `false`.	NO. Only valid during development to run Aura minibot.
AURA_ALLOWED_CALLERS	string	List of allowed callers separated by commas. By default: [`*`]	YES
AURA_AUTHORIZATION_APIKEY_PATH	string	Path to validate channel APIKey authorization	NO
AURA_AUTHORIZATION_ENDPOINT	string	aura-authentication-api base endpoint.	NO. In any case, it must be the internal k8s URL pointing to the aura-authentication-api.
AURA_AUTHORIZATION_HEADER	string	Complete authorization header to be sent to aura-authentication-api, with the following format: `APIKEY xxxxxx`.	YES, but only if the previous APIKey was deprecated.
AURA_CHANNELDATA_CURRENT_VERSION	string	`channelData` version used internally by aura-groot. By default, `3`.	NO
AURA_CHANNELDATA_DEFAULT_VERSION	string	`channelData` version returned by default in all messages. By default, `1.0.0`.	NO
AURA_CHANNELDATA_RESPONSE_VALIDATION	boolean	Flag to indicate that response validation should be done. By default, `true`.	NO
AURA_CHANNELDATA_VALIDATION	boolean	Flag to indicate that request validation should be done. By default, `true`.	NO
AURA_CHANNELDATA_VALIDATION_VERSIONS	string	`channelData` versions that will be validated. By default, `3`.	NO
AURA_CHANNELDATA_VALIDATION_MIN_VERSION	string	Minimal version to validate. By default, `3`.	NO
AURA_CHANNELS_CONFIGURATION_API_ENDPOINT	string	Endpoint for aura-configuration-api.	NO
AURA_DEFAULT_LOCALE	string	Culture code to be used by default in the current deployment: `de-de`, `en-gb`, `es-es`, `pt-br`.	NO
AURA_DEFAULT_TIME_ZONE	string	Default time zone. For example: `Europe/Madrid`	NO
AURA_ENCRYPTION_ALGORITHM	string	Encryption algorithm used to validate the APIKey	NO. It would break database encrypted data and APIKey validation.
AURA_ENCRYPTION_IV_LENGTH	number	Size for the initialization vector used by the encryption algorithm that validates the APIKey	NO. It would break database encrypted data and APIKey validation.
AURA_ENCRYPTION_IV_POSITION	number	Position where to insert the initialization vector in the final string with the encrypted payload. Future use.	NO. It would break database encrypted data and APIKey validation.
AURA_ENCRYPTION_KEY	string	Encryption key or comma-separated list of encryption keys to be used in the environment. It is mainly used to decrypt the APIKeys.	NO. It would break database encrypted data and APIKey validation.
AURA_ENVIRONMENT_NAME	string	Name of the environment where the aura-authentication-api is deployed. For example: `ap-next`, `es-dev` or `de-pre`.	NO
AURA_EXTERNAL_HOST_SERVICE_URL	string	URL of the external host service.	NO. In any case, it must be the internal k8s URL pointing to the `host-service`.
AURA_HOST_SERVICE_URL	string	URL of the host service.	NO. In any case, it must be the internal k8s URL pointing to the `host-service`.
AURA_HTTP_PATHS_DISABLED_FOR_LOGS	stringArray	Paths which will be filtered in logs. Used in HTTPMonkeyPatcher module.	NO
AURA_HTTP_KEEP_ALIVE	boolean	Use of keep-alive in HTTP connections. Used in HTTPMonkeyPatcher module.	NO
AURA_HTTP_KEEP_ALIVE_MSECS	number	Number of milliseconds to keep alive HTTP connections. Used in HTTPMonkeyPatcher module.	NO
AURA_HTTP_MONKEY_PATCHER_ENABLED	boolean	Enable or disable the monkey patching of the HTTP and HTTPs modules. By default, `false`.	YES
AURA_HTTP_REQUEST_MAX_SOCKETS	number	Maximum number of HTTP sockets open in the server. By default, `100`	NO. Only if indicated by Aura Global Team.
AURA_IGNORE_LOGGING_PATH	string	Comma-separated string with all the incoming paths which requests should not be logged. By default, `/healthz`	YES
AURA_INTERNAL_DELAY_MS	number	The time in milliseconds between retries. By default, `10000`	NO, except if requested
AURA_INTERNAL_RETRIES	number	Number of retries for internal calls. By default, `1`	NO, except if requested
AURA_KPI_ENABLED	Boolean	Boolean value, indicating whether aura-groot writes entity files or not. By default: `true`	NO, excepting if requested by Product or Operations teams
AURA_KPI_FILE_PREFIX	string	String with the prefix used in the KPIs entities files of this service. By default, `groot/GROOT`	NO
AURA_KPI_REMOVE_SPECIAL_CHARACTERS	string	Regular expression for removing special characters Default:`\n\r`	NO
AURA_KPI_STORE_MODE	string	It indicates which is the destination of the KPIs entities files. Default: `blob`. If `file`, they will be stored locally to the instance, in the folder shown in `KPI_TO_DSV_LOCAL_FILES_DIRECTORY`. For development purposes. If `blob`, they will be stored remotely in the Azure blob container shown in `KPIS_STORE_CONTAINER`. Mandatory in environments running on k8s.	NO, only configurable when running aura-groot locally.
AURA_KPI_TO_DSV_CACHE_TTL	number	Number with the amount of milliseconds to cache existing requests to calculate their duration. Default: 1800.	NO
AURA_KPI_TO_DSV_DELIMITER	string	Field delimiter to be used in KPIs entities files. Default: \|	NO. It will break all the analysis and processes running on top of these files.
AURA_KPI_TO_DSV_EXTENSION	string	Extension to be used in KPIs entities files.Default: `txt`	NO. If changed without changing aura-kpi-uploader, the files will not be uploaded to Kernel.
AURA_KPIS_BLOB_STORE_INTERVAL	number	Time interval in milliseconds to upload asyncrhonously logs to the `KPIS_STORE_CONTAINER`. Default: `60000`. Only needed if `KPI_STORE_MODE==blob`.	NO. In pre/production, it must be blob so the files will be uploaded to Kernel instance afterwards. Setting file for development is recommended.
AURA_KPIS_LOG_API_REQUEST_BODY	boolean	Flag to log or not the request body of the API calls. Default: `true`	YES. Once disabled, to enable run performance tests to validate if it is possible to write them.
AURA_KPIS_LOG_API_RESPONSE_BODY	boolean	Flag to log or not the response body of the API calls. Default: `true`	YES. Once disabled, to enable run performance tests to validate if it is possible to write them.
AURA_KPIS_STORE_CONTAINER	string	The name of the Azure Blob container to store KPIs entities files. By default, `aura-kpis`. It MUST be the same than the one configured in `KPIS_UPLOADER` module. Only needed if `KPI_STORE_MODE==blob`.	NO. If changed without changing aura-kpi-uploader, the files will not be uploaded to Kernel.
AURA_LOCALE_FOLDER	string	Full path to the local folder where locale files are stored. By default, `./locale`.	NO
AURA_LOCALE_FORCE_IMPORT	boolean	`true` if locale remote loading must be carried out although there were validation errors. By default, `false`.	YES
AURA_LOCALE_REMOTE_CONTAINER	string	Name of the Azure Storage Blob Container where the locale files will be stored. By default, `static-resources`.	NO
AURA_LOCALE_REMOTE_CONTAINER_PREFIX	string	Path within `AURA_LOCALE_REMOTE_CONTAINER` where the locale files are stored. By default, `locale`.	NO
AURA_LOGGING_FORMAT	string	Format to be used in monitoring logs: `json` or `dev`(more visual format). By default, `json`.	NO. Only for development, set it to `dev`.
AURA_LOGGING_LEVEL	string	Level to be used in monitoring logs, from more to less verbose: `'TRACE', 'DEBUG', 'INFO', 'WARN', 'ERROR', 'FATAL', 'OFF'`. By default, `INFO`.	YES, for development set it to `DEBUG`. In pre/production should be `INFO` or `ERROR`. For analysis of an issue in pre/production it may be changed to `DEBUG`.
AURA_MAKEUP_MODE	string	Allow dev mode of the make-up with the value `local`. By default, `full`.	NO, only for development, set it to `local`.
AURA_MAX_HISTORY_CHAT_ITERATIONS	number	Number of history chat iterations saved in conversation history. By default, `6`	NO
AURA_MESSAGES_REQUEST_PATH	string	Path exposed to receive incoming messages. By default, `/api/messages`	NO
AURA_MICROSOFT_APP_ID	string	ID of the Microsoft bot application to be used in the deployment.	NO. Only if Operations Team changes it.
AURA_MICROSOFT_APP_PASSWORD	string	Password of the Microsoft bot application to be used in the deployment.	NO. It must be changed by Operations Team.
AURA_MICROSOFT_APP_TYPE	string	Type of the Microsoft bot application to be used in the deployment. By default: `MultiTenant`	NO. Only if Operations Team changes it.
AURA_MICROSOFT_APP_TENANT_ID	string	Tenant ID of the Microsoft bot application to be used in the deployment.	NO. Only if operations team changes it.
AURA_MICROSOFT_AZURE_STORAGE_COMMON_ACCESS_KEY	string	Microsoft Storage password of the common storage. Currently used for KPI storing.	NO. Only if Operations Team changes it.
AURA_MICROSOFT_AZURE_STORAGE_COMMON_ACCOUNT	string	Microsoft Storage account of the common storage. Currently used for KPI storing.	NO. Only if Operations Team changes it.
AURA_MICROSOFT_AZURE_STORAGE_ACCESS_KEY	string	Microsoft Storage password of the deployment.	NO. It must be changed by Operations Team.
AURA_MICROSOFT_AZURE_STORAGE_ACCOUNT	string	Microsoft Storage account of the environment.	NO. It must be changed by Operations Team.
AURA_MICROSOFT_AZURE_STORAGE_CONFIGURATION_CONTAINER	string	Name of the container where basic bot configuration files are stored: Mongo indexes descriptor, aura-behavior-manager configuration, etc. By default, `aura-configuration`.	NO
AURA_MIDDLEWARE_SPEAK_PROCESSOR	string	Used to enable speak property text substitution on the outgoing messages, to produce speakable text. Regular expression of substitution format example: `[["\\*"," asterisk "],["\\#"," hash "]]`	YES. It includes all the needed changes to be done automatically between the written and the spoken text.
AURA_MONGODB_BOT_COLLECTION_CONTEXT_INDEX_TTL	number	Number of seconds to keep the context data available in aura-groot context cache. By default, `3600`.	NO
AURA_MONGODB_BOT_COLLECTION_CONTEXT	string	MongoDB collection name where aura-groot will store the context of the users. By default, `aura-context`.	NO
AURA_MONGODB_BOT_DATABASE	string	MongoDB database name where aura-groot will store all the data needed to handle requests. By default, `aura-groot`.	NO
AURA_MONGODB_CACHE_TTL	number	Lifetime of cache used by aura-behavior-manager. It MUST be disabled in PRE and PRO environments. By default, `3600` seconds (1h).	YES, only for development environments.
AURA_MONGODB_CACHE_DATABASE	string	Database used in cache of aura-behavior-manager. By default, `aura-groot-caches`.	YES, only for development environments.
AURA_MONGODB_CACHE_COLLECTION_DL	string	Collection used in cache of aura-behavior-manager. By default, `dev-cache`.	YES, only for development environments.
AURA_MONGODB_CACHE_INDEX_DL	string	Index used in TwoLevelsCache of aura-behavior-manager. By default, `id`.	YES, only for development environments.
AURA_MONGODB_CACHE_METRIC_TTL	number	Lifetime of metric cache. By default, `60` seconds (1 minute).	YES.
AURA_MONGODB_CACHE_METRIC_COLLECTION	string	Collection used in metric-cache. By default, `metric-cache`.	YES
AURA_MONGODB_CACHE_METRIC_INDEX	string	Index used in TwoLevelsCache of metric-cache. By default, `correlator`.	YES
AURA_MONGODB_PASSWORD	string	MongoDB password to access `AURA_MONGODB_BOT_DATABASE` in `AURA_MONGODB_URI`.	NO
AURA_MONGODB_POOL_SIZE	number	Number of connections available in MongoDB pool. By default, `60`.	NO
AURA_MONGODB_SSL	boolean	`true` if access to `AURA_MONGODB_URI` is `SSL`. By default, `false`.	NO
AURA_MONGODB_URI	string	MongoDB URI of the server handling. `AURA_MONGODB_BOT_DATABASE`.	NO
AURA_MONGODB_USERNAME	string	MongoDB username to access `AURA_MONGODB_BOT_DATABASE` in `AURA_MONGODB_URI`.	NO
AURA_MONGODB_MAX_POOL_SIZE	number	Maximum number of connections in the MongoDB pool. By default, `60`.	NO
AURA_MONGODB_MIN_POOL_SIZE	number	Minimum number of connections in the MongoDB pool. By default, `2`.	NO
AURA_MONGODB_MAX_IDLE_TIME_MS	number	Maximum idle time in milliseconds for connections in the MongoDB pool. By default, `30000` (30 seconds).	NO
AURA_MONGODB_RETRY_WRITES	boolean	`true` if retry writes are enabled in the MongoDB connection. By default, `true`.	NO
AURA_MONGODB_RETRY_READS	boolean	`true` if retry reads are enabled in the MongoDB connection. By default, `true`.	NO
AURA_MONGODB_HEARTBEAT_FREQUENCY_MS	number	Frequency in milliseconds of the heartbeat in the MongoDB connection. By default, `2000` (2 seconds).	NO
AURA_MONGODB_SERVER_SELECTION_TIMEOUT_MS	number	Timeout in milliseconds for server selection in the MongoDB connection. By default, `5000` (5 seconds).	NO
AURA_MONGODB_SOCKET_TIMEOUT_MS	number	Timeout in milliseconds for socket operations in the MongoDB connection. By default, `45000` (45 seconds).	NO
AURA_CONNECTION_TIMEOUT_MS	number	Timeout in milliseconds for establishing a connection to the MongoDB server. By default, `10000` (10 seconds).	NO
AURA_MONGODB_READ_PREFERENCE	string	Read preference for the MongoDB connection. By default, `primaryPreferred`. Possible values: `primary`, `primaryPreferred`, `secondary`, `secondaryPreferred`, `nearest`.	NO
AURA_REQUEST_ENABLE_NAGLE	boolean	Boolean to enable Nagle. By default: `false`.	NO
AURA_REQUEST_TIMEOUT	number	Number of milliseconds to wait a request. By default: `30000`, 30 seconds.	YES, in case of network issues.
AURA_SERVER_BODY_LIMIT	string	Maximum size in bytes of the request body. It is a string because the allowed values must indicate the units: 10 mb, 200 kb, etc. By default, `20 mb`.	YES, in order to decrease it, if it is considered too high. To increase it, both local and global Operational Teams must review it, because it could lead to DDoS attacks easily.
AURA_SERVER_PORT	number	Port where aura-groot will be listening to requests. By default, `8080`.	NO
AURA_SERVICE_ENVIRONMENT	string	Type of environment: `'DEV', 'PRE', 'PRO'`. By default, `DEV`. It is used during locale translation, to get the correct text reference and, in development environments, to allow aura-behavior-manager execution.	NO
AURA_SERVICE_URL_DIRECTLINE	string	Default URL for Direct Line services. By default, `https://directline.botframework.com/`.	NO
AURA_SHUTDOWN_GRACEFUL_TTL	number	Number of milliseconds to wait until all aura-groot tasks are done before shutting each pod down. By default `25000` (25 seconds)	NO
AURA_SKILLS_BLOCKED_TIMEOUT	number	Number of seconds to block a skill after a timeout. By default, `600` (10 minutes).	NO
AURA_SKILLS_RESPONSE_PATH	string	Path where the skills will return the answers. By default, `/api/skills`.	NO
AURA_SUPERAGENT_DEADLINE_TIMEOUT	number	Number of milliseconds to wait until the `superagent` deadline timeout, as specified here. It is optional and, by default, is not configured.	No, only changed if the Global Team or the LCDO Dev Team requires it.
AURA_SUPERAGENT_RESPONSE_TIMEOUT	number	Number of milliseconds to wait until the `superagent` response timeout, as specified here. It is optional and, by default, it is not configured.	No, only changed if the Global Team or the LCDO Dev Team requires it.
AURA_SUPERAGENT_RETRY_ATTEMPTS	number	Number of retry attempts (not counting the first attempt) to be applied on `superagent` requests, as specified here. It is optional and, by default, it is not configured.	No, only changed if the Global Team or the LCDO Dev Team requires it.
AURA_RETRIES_CODES	string[]	Array of strings with error codes. By default, [‘ECONNRESET’]	NO, except if requested
AURA_VERSION	string	Number of the Aura release being executed.	NO
AURA_HTTP_PATHS_LOG_DISABLED	string	HTTP paths separated by commas which requests would not be logged. By default: `aura-kpis,static-resources`. Used in http-monkey-patcher.	YES, if there is a path that is not wanted to be logged. The default values are always added to the provided list.
DEV_AURA_BEHAVIOR_MANAGER_ACTIVE	boolean	Flag to indicate whether or not aura-behavior-manager module should be activated in the current deployment. It is only valid for development environments.	YES, in development environments.
DEV_AURA_GROOT_BEHAVIOR_CRON_PATTERN	string	Cron pattern to set the refresh time of aura-behavior-manager configuration. By default, `/5 * * *` (every 5 minutes). It is only valid for development environments.	YES, in development environments.
DEV_AURA_GROOT_BEHAVIOR_MICROSOFT_AZURE_STORAGE_SETTINGS_FILE	string	Name of the configuration file route used by aura-behavior-manager. By default, `aura-groot/aura-groot-behavior-manager.json`. It is only valid for development environments.	YES, in development environments

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Intent result recognizer

Description of intent-result-recognizer in charge of registering the intent result

Introduction

intent-result-recognizer is the recognizer in charge of the registration of the intent result.

This recognizer is executed after the execution of the middlewares and before sending a message to user. It is also in charge of registering the result of the intent in KPIs.

It receives information from the skill in the following way:

activity: {
    channelData: {
        payload: {
            skill: {
                intentResult: {
                    name: string,
                    entities: [
                        entity: string,
                        type: string
                    ]
                }
            }
        }
    }
}

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Groot middlewares

Description of middlewares included in aura-groot

Introduction

The current document describes the different middlewares that are part of aura-groot

init-middleware

This middleware is in charge of setting the initial context values (userData accessor, conversationData accessor, correlator) for the rest of the middlewares/handlers.

user-middleware

In this middleware, the user is checked and stored in context for the following middlewares/handlers.

Find detailed information in user-middleware documentation.

channeldata-validator-middleware

This middleware validates input channelData activities and throws an error if something is wrong.

Find more information in channeldata-validator-middleware documentation.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

User middleware

Description of user-middleware in charge of the validation of the user’s data

Introduction

user-middleware is the middleware in charge of validating auraId and channel.

This is needed in aura-groot because routing is currently done by channel, so if the request does not include channelId (mandatory only in channelData v3), the user must be checked in aura-authentication-api to obtain the channel of this user.

This middleware throws an error if there is not activity.from.id or if some error is returned by aura-authentication-api (except 404, because anonymous users should continue the flow).

This middleware also handles a basic user’s cache to avoid unnecessary calls to aura-authentication-api with every user’s request. This cache is implemented using bot UserState.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Groot

aura-groot is the router bot that manages the behavior of Aura Root ecosystem functionalities acting as a router for the request to a specific skill.
Find in the current documents the description of this component, its architecture, components and processes.

Aura Virtual Assistant component

Introduction

aura-groot is the main component in charge of handling the distributed architecture of Aura. It is responsible for the communication between each channel and its corresponding skill, keeping track of all the communication process, but introducing a minimal interference on it.

aura-groot main role includes:

It is the component in charge of making the decision about what is the best bot to answer the user’s request and routing the request to the corresponding skill (bot).
It is able to map configurable input parameters such as the channel that sent the request, user type, previous requests stored in the context, utterance, etc. with a specific domain. The OB will be able to configure the parameters used for the routing process.

Currently, routing is done only by channels or group of channels (no configuration options are available).

Aura Groot components

aura-groot contains two main components:

Middlewares: software components in charge of the message flow.
Recognizers: specific type of middlewares that are executed in a certain stage of the message flow.

Access to the corresponding sections to find detailed information regarding these components.

Aura Groot architecture

The first diagram shows all the components running in aura-groot.

The following diagram includes the messages flow through aura-groot.

Aura Groot configuration and operation

Find out the environment variables included in the aura-groot component here.

The operation of aura-groot is described in the document Aura Groot operational flows.

ConversationId handling

aura-groot uses groot-channel conversation identifier to track the conversation with the user and the use case with the skill. It is a string generated by the channel and is unique for each conversation. The groot-channel is sent in the incoming request to aura-groot in the conversationId field of the message object.

Additionally, it adds conversationReference inside. In this way, aura-groot only has to manage a single conversation and avoids synchronizing two different conversations. Doing so, accesses from aura-groot to the database are reduced.

This change implies a modification in the process the BotFramework works with a conversation, in two different documents which are updated separately. But, on the contrary, the advantage is a most efficient and secure way of working.

sequenceDiagram
title: BEFORE-DIRE-STRAITS

    actor Channel
    participant AuraGroot #ebdff7
    participant AuraBot #ebdff7

    Channel ->> AuraGroot: request message1
    AuraGroot ->> AuraGroot: create targetSkill in groot-channel
    AuraGroot ->> AuraGroot: save conversationReference in groot-channel
    AuraGroot ->> AuraBot: request message1
    AuraBot -->> AuraGroot: response message1
    AuraGroot ->> AuraGroot: load conversationReference in groot-channel
    AuraGroot -->> Channel: response message1
    AuraBot -->> AuraGroot: response endOfConversation
    AuraGroot ->> AuraGroot: load conversationReference in groot-channel
    AuraGroot ->> AuraGroot: delete targetSkill

    Channel ->> AuraGroot: request message2
    AuraGroot ->> AuraGroot: create targetSkill in groot-channel
    AuraGroot ->> AuraGroot: save conversationReference in groot-channel
    AuraGroot ->> AuraBot: request message2
    AuraBot -->> AuraGroot: response message2
    AuraGroot ->> AuraGroot: load conversationReference in groot-channel
    AuraGroot -->> Channel: response message2
    AuraBot -->> AuraGroot: response endOfConversation
    AuraGroot ->> AuraGroot: load conversationReference in groot-channel
    AuraGroot ->> AuraGroot: delete targetSkill

Aura Groot endpoints

Path	Method	Description
/api/messages/	POST	Incoming requests of channels or aura-bridge
/refresh/	POST	Update skills and channels. Data: `{"changes": ["skills", "channels"]}`. You can use “skills” and/or “channels” or neither of them. In the last case, all will be updated.
/api/skills/*	POST	Endpoints of Direct Line API for skills can exchange messages with aura-groot. The path where the skills will return the answers is configured in the environment variable `AURA_SKILLS_RESPONSE_PATH`, by default, `/api/skills`.
/healthz	GET	Monitoring endpoint. Healthcheck endpoint to validate if the server is up and running, only available from local network
/shutdown	GET	Monitoring endpoint. Wait until shutdown (Kubernetes lifecycle).
/heapSnapshot	GET	Endpoint that must be called internally directly accessing to the aura-groot pod. It stores in Aura Azure Storage Account a copy of the heap memory of the pod. It is useful to debug memory leaks.
/heapStatistics	GET	Endpoint that must be called internally directly accessing to the aura-groot pod. It returns the heap memory statistics of the pod.

These endpoints do not need to be defined in the plugin swagger, since they are supplied by the aura-groot core. Specific routes are defined in https://github.com/Telefonica/aura-distributed-bot/blob/main/master/packages/aura-groot/src/modules/server/routes.ts.

Postman collection

Aura Platform Team has generated a Postman collection for simulations over Aura distributed architecture: Aura distributed Postman collection.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Groot recognizers

Description of recognizers included in aura-groot

Introduction

The current documents describe the different recognizers that are part of aura-groot.

They are listed below:

domain-recognizer
This recognizer is in charge of identifying the domain of the user’s input.
intent-result-recognizer
This recognizer is in charge of registering of intent result.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Virtual Assistant development architecture and operation

Documents from this point forward contain low-level technical information regarding Aura Virtual Assistant, aimed at technical profiles.

We highly recommend reading these basic Aura Virtual Assistant documents first to have a clear overview of its foundation, benefits and functional behavior:

Introduction

Aura Virtual Assistant architecture is conceived as a multi-bot system based on Microsoft skills architecture.

In this framework, Aura is divided into isolated modules (skill-based bots) that, currently, are independent domain bots able to provide a specific function or capability in a coordinated and efficient manner.

These self-supported skills work with an orchestrator at the top, responsible for connecting channels with bots and managing the conversational flow with the customer.

Mapping functional architecture with development architecture

As explained before, Aura Virtual Assistant documentation contains both:

Low-level technical documents, aimed at not-technical profiles, that describe the functional behavior of our assistant
High-level technical documents, for highly-skilled technical profiles, that include detailed technical information, system architecture and technical processes

In order to ease comprehension for readers, take as a basis the Aura Virtual Assistant functional flowchart and check the mapping between functional and technical components.

Functional component	Technical component
Channel	Channel
Aura Root	Routing module	aura-groot
Channel adapter	aura-bridge
Skill	Skill

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Migrate Aura Bot and Aura Groot context to Linkin Park

Guidelines for the migration of aura-bot and aura-groot database context schemas from releases previous to Linkin Park to Linkin Park onwards. Additionally, the rollback procedure is included.

Prerequisites

Mandatory requisite: A valid kubeconfig for the environment must be available.
Recommendation: Create a snapshot/backup of the current DB data.

Process for context DB migration

Access MongoDB console

To access and open MongoDB console we can use the kubectl command below to export the intended environment, with MongoDB adminuser password. Use the right environment and database name in the commands.

$ export ATLAS_DEPLOYMENT_NAME=aura-br-pro-70 # Get from kubectl get -n aura-system atlasdeployments.atlas.mongodb.com
$ export ATLAS_DATABASE_NAME=aura-groot-br-pro # Set with the correct DB of your deployment
$ export MONGODB_USER=$(kubectl get -n aura-system atlasdatabaseusers.atlas.mongodb.com mongodb-adminuser-password -o jsonpath='{.spec.username}')
$ export MONGODB_PASSWORD=$(kubectl get -n aura-system secret mongodb-adminuser-password -o go-template='{{ .data.password | base64decode }}')
$ export MONGODB_URI="$(kubectl get -n aura-system atlasdeployments.atlas.mongodb.com $ATLAS_DEPLOYMENT_NAME -o jsonpath='{.status.connectionStrings.standardSrv}'|sed "s|srv://|srv://$MONGODB_USER:$MONGODB_PASSWORD@|g")/${ATLAS_DATABASE_NAME}"
$ kubectl -n aura-system run fix-script -i -q --rm --restart=Never --image=mongo:latest --command mongosh ${MONGODB_URI}

Migrate aura-bot context collection to new schema

Update documents to the new format:

> db["aura-context"].updateMany(
      {
        document: {
          $exists: true
        }
      },
      [
        {
          $set: {
            eTag: "$document.eTag",
            conversationData: { $mergeObjects: ["$conversationData", "$document.conversationData"] },
            dialogState: { $mergeObjects: ["$dialogState", "$document.dialogState"] },
            userData: { $mergeObjects: ["$userData", "$document.userData"] },
            userAuthData: { $mergeObjects: ["$userAuthData", "$document.userAuthData"] }
          }
        },
        { $unset: "document" }
      ]
    );

Migrate aura-groot context collection to new schema

Update documents to the new format:

> db["aura-context"].updateMany(
      {
        document: {
          $exists: true
        }
      },
      [
        {
          $set: {
            eTag: "$document.eTag",
            activeSkill: { $mergeObjects: ["$activeSkill", "$document.activeSkill"] },
            conversationHistory: { $concatArrays: [{ $ifNull: ["$conversationHistory", []] }, { $ifNull: ["$document.conversationHistory", []] }] },
            conversationDataProperty: { $mergeObjects: ["$conversationDataProperty", "$document.conversationDataProperty"] },
            dialogContext: { $concatArrays: [{ $ifNull: ["$dialogContext", []] }, { $ifNull: ["$document.dialogContext", []] }] },
            skillConversationReference: { $mergeObjects: ["$skillConversationReference", "$document.skillConversationReference"] }
          }
        },
        { $unset: "document" }
      ]
    );

Rollback aura-bot to the previous schema

If a rollback is needed, documents should be modified to have the previous schema and maintain the users’ context data:

> db["aura-context"].updateMany(
      {},
      [
        {
          $set: {
            document: {
              eTag: "$eTag",
              conversationData: { $mergeObjects: ["$document.conversationData", "$conversationData"] },
              dialogState: { $mergeObjects: ["$document.dialogState", "$dialogState"] },
              userData: { $mergeObjects: ["$document.userData", "$userData"] },
              userAuthData: { $mergeObjects: ["$document.userAuthData", "$userAuthData"] }
            }
          }
        },
        {
          $unset: [ "eTag", "conversationData", "dialogState", "userData", "userAuthData"]
        }
      ]
    );

Rollback aura-groot to the previous schema

If a rollback is needed, documents should be modified to have the previous schema and maintain the users’ context data:

> db["aura-context"].updateMany(
      {},
      [
        {
          $set: {
            document: {
              eTag: "$eTag",
              activeSkill: { $mergeObjects: ["$document.activeSkill", "$activeSkill"] },
              conversationHistory: { $concatArrays: [{ $ifNull: ["$document.conversationHistory", []] }, { $ifNull: ["$conversationHistory", []] }] },
              conversationDataProperty: { $mergeObjects: ["$document.conversationDataProperty", "$conversationDataProperty"] },
              dialogContext: { $concatArrays: [{ $ifNull: ["$document.dialogContext", []] }, { $ifNull: ["$dialogContext", []] }] },
              skillConversationReference: { $mergeObjects: ["$document.skillConversationReference", "$skillConversationReference"] }
            }
          }
        },
        {
          $unset: [ "eTag", "activeSkill", "conversationHistory", "conversationDataProperty", "dialogContext", "skillConversationReference"]
        }
      ]
    );

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Groot operational flows

Description of aura-groot main operational flows

Introduction

The current document includes the main operational flows in the aura-groot component.

Blocked skill

This flow describes the control of the messages received by aura-groot from the skill and the sending of the error message to the user in case the skill does not respond.

sequenceDiagram
title: BLOCKED-SKILL

    actor Channel
    participant AuraGroot #ebdff7
    participant AuraBot #ebdff7

    Channel ->> AuraGroot: request message1
    AuraGroot ->> AuraGroot: create skillConversation1, begin timeout in targetSkill
    AuraGroot ->> AuraBot: request message1
    AuraBot -->> AuraGroot: response message1
    AuraGroot ->> AuraGroot: delete timeout, update targetSkill
    AuraGroot -->> Channel: response message1
    AuraBot -->> AuraGroot: response endOfConversation
    AuraGroot ->> AuraGroot: delete targetSkill

    Channel ->> AuraGroot: request message2
    AuraGroot ->> AuraGroot: create skillConversation2, begin timeout in targetSkill
    AuraGroot ->> AuraBot: request message2
    Channel -->> AuraGroot: request message3
    AuraGroot ->> AuraGroot: control timeout (active targetSkill)

    alt timeout targetSkill < TIMEOUT
        AuraGroot ->> AuraBot: request message3
        AuraBot -->> AuraGroot: response message2
        AuraGroot ->> AuraGroot: delete timeout, update targetSkill
        AuraGroot -->> Channel: response message2

        AuraBot -->> AuraGroot: response message3
        AuraGroot ->> AuraGroot: delete timeout, update targetSkill
        AuraGroot -->> Channel: response message3
        AuraBot -->> AuraGroot: response endOfConversation message2
        AuraGroot ->> AuraGroot: delete targetSkill
        AuraBot -->> AuraGroot: response endOfConversation message3
        AuraGroot ->> AuraGroot: discard endOfConversation in conversation2
    end

    alt timeout targetSkill > TIMEOUT
        AuraGroot -->> Channel: Error
        AuraGroot ->> AuraGroot: begin timeout, in targetSkill
        AuraGroot ->> AuraBot: request message3
        AuraBot -->> AuraGroot: response message2
        AuraGroot ->> AuraGroot: delete timeout, update targetSkill
        AuraGroot -->> Channel: response message2
        AuraBot -->> AuraGroot: response endOfConversation message2
        AuraGroot ->> AuraGroot: delete targetSkill
        AuraBot -->> AuraGroot: response message3
        AuraGroot ->> AuraGroot: discard delete timeout in targetSkill
        AuraGroot -->> Channel: response message3
        AuraBot -->> AuraGroot: response endOfConversation message3
        AuraGroot ->> AuraGroot: discard endOfConversation in targetSkill
    end

Use case with asynchronous calls

This diagram describes the flow of a user’s request that is resolved through a dialog with asynchronous calls.

Further information regarding how asynchronous APIs are handled by Aura can be found in the Guidelines for building a dialog that uses async APIs.

sequenceDiagram
title: MESSAGE_ASYNC_USECASE
    actor User
    User ->> WA: User request
    WA ->> 4P: User request
    4P ->> AuraBridge: User request
    AuraBridge ->> 4P: 200 OK
    AuraBridge ->> AuraGroot: User request
    AuraGroot ->> AuraBridge: 200 OK
    AuraGroot ->> AuraGroot: create skillConversation1, begin timeout in targetSkill
    AuraGroot ->> AuraSkill: User request
    AuraSkill ->> AuraGroot: 200 OK
    AuraSkill ->> AuraSkill: basic whatsapp flow
    AuraSkill ->> LocalDialog: message
    LocalDialog ->> 4P: request async + callback bridge
    4P ->> LocalDialog: 200 OK
    Note right of 4P: when 4p finishes the request sends the response to the indicated callback (AuraBridge)
    4P ->> AuraBridge: response request async
    AuraBridge ->> 4P: 200 OK
    AuraBridge ->> AuraGroot: response request async
    AuraGroot ->> AuraBridge: 200 OK
    Note right of AuraBridge: the response is sent as payload
    AuraGroot ->> AuraGroot: control timeout
    alt targetSkill.timeout < TIMEOUT
        AuraGroot ->> AuraSkill: response request async
        AuraSkill ->> AuraGroot: 200 OK
        AuraSkill ->> LocalDialog: response request async
        LocalDialog ->> LocalDialog: process response
        LocalDialog ->> AuraSkill: response processing
        AuraSkill ->> AuraGroot: async response processed
        AuraSkill ->> AuraGroot: response endOfConversation
        AuraGroot ->> AuraSkill: 200 OK
        AuraGroot ->> AuraGroot: delete targetSkill
        AuraGroot ->> AuraGroot: delete skillConversation1
        AuraGroot ->> AuraBridge: async response processed
        AuraBridge ->> AuraGroot: 200 OK
        AuraBridge ->> 4P: async response processed
        4P ->> AuraBridge: 200 OK
        4P ->> WA: async response processed
        WA ->> User: async response processed
    else
        AuraGroot ->> AuraBridge: error
        AuraBridge ->> AuraGroot: 200 OK
        AuraBridge ->> 4P: async error
        4P ->> AuraBridge: 200 OK
        4P ->> WA: error
        WA ->> User: error
        AuraGroot -> AuraSkill: request async endOfConversation
        AuraGroot -> AuraGroot: delete skillConversation1
        AuraGroot -> AuraGroot: create skillConversation2, begin timeout in targetSkill
        AuraGroot ->> AuraSkill: response request async
        AuraSkill ->> AuraGroot: 200 OK
        AuraSkill ->> AuraSkill: Not recognize
        AuraSkill ->> AuraGroot: none
        AuraSkill ->> AuraGroot: response endOfConversation
        AuraGroot ->> AuraSkill: 200 OK
        AuraGroot ->> AuraGroot: delete targetSkill
        AuraGroot ->> AuraGroot: delete skillConversation2
        AuraGroot ->> AuraBridge: none
        AuraBridge ->> AuraGroot: 200 OK
        AuraBridge ->> 4P: none
        4P ->> AuraBridge: 200 OK
        4P ->> WA: none
        WA ->> User: none
    end

Use case with files sending by the user

This diagram describes the flow of a use case that includes sending a file by the user (WhatsApp channel).

Further information regarding how to receive a file from the user can be found in:

sequenceDiagram
title: MESSAGE_FILE_USECASE
    User ->> WA: Phrase to init UC file dialog
    WA ->> 4P: Phrase to init UC file dialog
    4P ->> AuraBridge:Phrase to init UC file dialog
    AuraBridge ->> 4P: 200 OK
    AuraBridge ->> AuraGroot: Phrase to init UC file dialog
    AuraGroot ->> AuraBridge: 200 OK
    AuraGroot ->> AuraGroot: create skillConversation1, begin timeout in targetSkill
    AuraGroot ->> AuraSkill: Phrase to init UC file dialog
    AuraSkill ->> AuraGroot: 200 OK
    AuraSkill ->> AuraSkill: Basic whatsapp flow for authenticated user
    AuraSkill ->> FileDialog: Start UC file dialog
    FileDialog ->> AuraSkill: Send me a file
    AuraSkill ->> AuraGroot:  Send me a file
    AuraGroot ->> AuraSkill: 200 OK
    AuraGroot ->> AuraGroot: delete timeout in targetSkill
    AuraGroot ->> AuraBridge: Send me a file
    AuraBridge ->> AuraGroot: 200 OK
    AuraBridge ->> 4P: Send me a file
    4P ->> AuraBridge: 200 OK
    4P ->> WA: Send a file
    WA ->> User: Send a file
    User ->> WA: Send the attachment
    WA ->> 4P: Send the attachment
    4P ->> AuraBridge: Send the attachment
    AuraBridge ->> 4P: 200 OK
    AuraBridge ->> AuraGroot: Send the attachment
    AuraGroot ->> AuraBridge: 200 OK
    AuraGroot ->> AuraSkill: Send the attachment
    AuraSkill ->> AuraGroot: 200 OK
    AuraSkill ->> FileApi: Validate / process the attachment
    FileApi ->> AuraBot: 200 OK
    FileApi ->> AuraBridge: Attachment processed, validated and ready (async-callback)
    AuraBridge ->> FileApi: 200 OK
    AuraBridge ->> AuraGroot: Send attachment processed
    AuraGroot ->> AuraBridge: 200 OK
    AuraGroot ->> AuraGroot: control timeout
    Note right of AuraSkill: we could control that it is a system message, but the case would be the same if a message from the user arrives before the asynchronous response
    alt targetSkill.timeout < TIMEOUT
        AuraGroot ->> AuraSkill: Send attachment processed
        AuraSkill ->> AuraGroot: 200 OK
        AuraGroot ->> FileDialog: Send attachment processed
        FileDialog ->> FileDialog: process attachment
        FileDialog ->> AuraSkill: response processed attachment
        AuraSkill ->> AuraGroot: response processed attachment
        AuraSkill ->> AuraGroot: response endOfConversation
        AuraGroot ->> AuraSkill: 200 OK
        AuraGroot ->> AuraGroot: delete targetSkill
        AuraGroot ->> AuraGroot: delete skillConversation1
        AuraGroot ->> AuraBridge: response processed attachment
        AuraBridge ->> AuraGroot: 200 OK
        AuraBridge ->> 4P: response processed attachment
        4P ->> AuraBridge: 200 OK
        4P ->> WA: response processed attachment
        WA ->> User: response processed attachment
    else
        AuraGroot ->> AuraBridge: error
        AuraBridge ->> AuraGroot: 200 OK
        AuraBridge ->> 4P: async error
        4P ->> AuraBridge: 200 OK
        4P ->> WA: error
        WA ->> User: error
        AuraGroot -> AuraSkill: request endOfConversation
        AuraGroot -> AuraGroot: delete skillConversation1
        AuraGroot -> AuraGroot: create skillConversation2, begin timeout in targetSkill
        AuraGroot ->> AuraSkill: response processed attachment
        AuraSkill ->> AuraGroot: 200 OK
        AuraSkill ->> AuraSkill: UnexpectedFile
        AuraSkill ->> AuraGroot: UnexpectedFile
        AuraSkill ->> AuraGroot: response endOfConversation
        AuraGroot ->> AuraSkill: 200 OK
        AuraGroot ->> AuraGroot: delete targetSkill
        AuraGroot ->> AuraGroot: delete skillConversation2
        AuraGroot ->> AuraBridge: UnexpectedFile
        AuraBridge ->> AuraGroot: 200 OK
        AuraBridge ->> 4P: UnexpectedFile
        4P ->> AuraBridge: 200 OK
        4P ->> WA: UnexpectedFile
        WA ->> User: UnexpectedFile
    end

Handover use case

This diagram describes the flow of the handover use case.

Further information regarding about handover use case handover-genesys dialog.

sequenceDiagram

title: handover
actor Channel
participant Bridge #ebdff7
participant AuraGroot #ebdff7
participant AuraBot #ebdff7
participant Genesys #ebdff7

Channel ->> AuraGroot: request message1
AuraGroot ->> AuraGroot: create skillConversation1, begin timeout in targetSkill
AuraGroot ->> AuraBot: request message1
AuraBot -->> AuraGroot: response message1
AuraGroot ->> AuraGroot: delete timeout in targetSkill
AuraGroot -->> Channel: response message1
AuraBot -->> AuraGroot: response endOfConversation
AuraGroot ->> AuraGroot: delete targetSkill
AuraBot ->> Genesys: request chat "init"
Genesys ->> AuraBot: response chat "init"
AuraBot ->> Genesys: message history
Genesys ->> AuraBot: response message history
Genesys -->> Bridge: pushNotification: starter messages with genesys

Bridge ->> AuraGroot: request message2
AuraGroot ->> AuraGroot: create skillConversation2, begin timeout in targetSkill
AuraGroot ->> AuraBot: request message2
AuraBot -->> AuraGroot: response message2
AuraGroot ->> AuraGroot: delete timeout in targetSkill
AuraGroot -->> Bridge: response message2
Bridge -->> Channel: response message2
AuraBot -->> AuraGroot: response endOfConversation
AuraGroot ->> AuraGroot: delete targetSkill

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Migrate Aura Groot context (Dire Straits)

Guidelines for the migration of aura-groot database context schema from releases previous to Dire Straits to Dorian release onwards. Additionally, the rollback procedure is included.

Prerequisites

Mandatory requisite: A valid kubeconfig for the environment must be available.
Recommendation: Create a snapshot/backup of the current DB data.

Process for context DB migration

Access MongoDB console

$ export ATLAS_DEPLOYMENT_NAME=aura-br-pro-70 # Get from kubectl get -n aura-system atlasdeployments.atlas.mongodb.com
$ export ATLAS_DATABASE_NAME=aura-groot-br-pro # Set with the correct DB of your deployment
$ export MONGODB_USER=$(kubectl get -n aura-system atlasdatabaseusers.atlas.mongodb.com mongodb-adminuser-password -o jsonpath='{.spec.username}')
$ export MONGODB_PASSWORD=$(kubectl get -n aura-system secret mongodb-adminuser-password -o go-template='{{ .data.password | base64decode }}')
$ export MONGODB_URI="$(kubectl get -n aura-system atlasdeployments.atlas.mongodb.com $ATLAS_DEPLOYMENT_NAME -o jsonpath='{.status.connectionStrings.standardSrv}'|sed "s|srv://|srv://$MONGODB_USER:$MONGODB_PASSWORD@|g")/${ATLAS_DATABASE_NAME}"
$ kubectl -n aura-system run fix-script -i -q --rm --restart=Never --image=mongo:latest --command mongosh ${MONGODB_URI}

Migrate context collection to new schema

Update documents to the new format:

> db["aura-context"].aggregate([
      {
        $match: {
          _id: {
            $regex: "/conversations/",
          },
          "document.activeSkill.conversationId": {
            $exists: true,
          },
        },
      },
      {
        $lookup:
          {
            from: "aura-context",
            localField:
              "document.activeSkill.conversationId",
            foreignField: "_id",
            as: "document.skillConversationReference",
          },
      },
      {
        $set: {
          "document.skillConversationReference": {
            $arrayElemAt: [
              "$document.skillConversationReference",
              0,
            ],
          },
        },
      },
      {
        $set: {
          "document.skillConversationReference":
            "$document.skillConversationReference.document",
        },
      },
      {
        $merge: "aura-context",
      },
    ])

Check if changes have been applied. The following query should return n documents:

  > db["aura-context"].find({"document.skillConversationReference": { $exists: true }})

Rollback to the previous schema

If a rollback is needed, documents should be modified to have the previous schema and maintain the users’ context data:

> db["aura-context"].aggregate([
    {
      $match:
        {
          "document.activeSkill.conversationId": {
            $exists: true,
          },
          "document.skillConversationReference": {
            $exists: true,
          },
        },
    },
    {
      $project:
        {
          _id: "$document.activeSkill.conversationId",
          document:
            "$document.skillConversationReference",
          expiresAt: "$expiresAt",
        },
    },
    {
      $merge: "aura-context",
    },
  ])

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Migrate Aura Groot context (Dire Straits)

Guidelines for the migration of aura-groot database context schema from releases previous to Dire Straits to Dorian release onwards. Additionally, the rollback procedure is included.

Prerequisites

Mandatory requisite: A valid kubeconfig for the environment must be available.
Recommendation: Create a snapshot/backup of the current DB data.

Process for context DB migration

Access MongoDB console

$ export ATLAS_DEPLOYMENT_NAME=aura-br-pro-70 # Get from kubectl get -n aura-system atlasdeployments.atlas.mongodb.com
$ export ATLAS_DATABASE_NAME=aura-groot-br-pro # Set with the correct DB of your deployment
$ export MONGODB_USER=$(kubectl get -n aura-system atlasdatabaseusers.atlas.mongodb.com mongodb-adminuser-password -o jsonpath='{.spec.username}')
$ export MONGODB_PASSWORD=$(kubectl get -n aura-system secret mongodb-adminuser-password -o go-template='{{ .data.password | base64decode }}')
$ export MONGODB_URI="$(kubectl get -n aura-system atlasdeployments.atlas.mongodb.com $ATLAS_DEPLOYMENT_NAME -o jsonpath='{.status.connectionStrings.standardSrv}'|sed "s|srv://|srv://$MONGODB_USER:$MONGODB_PASSWORD@|g")/${ATLAS_DATABASE_NAME}"
$ kubectl -n aura-system run fix-script -i -q --rm --restart=Never --image=mongo:latest --command mongosh ${MONGODB_URI}

Migrate context collection to new schema

Update documents to the new format:

> db["aura-context"].aggregate([
      {
        $match: {
          _id: {
            $regex: "/conversations/",
          },
          "document.activeSkill.conversationId": {
            $exists: true,
          },
        },
      },
      {
        $lookup:
          {
            from: "aura-context",
            localField:
              "document.activeSkill.conversationId",
            foreignField: "_id",
            as: "document.skillConversationReference",
          },
      },
      {
        $set: {
          "document.skillConversationReference": {
            $arrayElemAt: [
              "$document.skillConversationReference",
              0,
            ],
          },
        },
      },
      {
        $set: {
          "document.skillConversationReference":
            "$document.skillConversationReference.document",
        },
      },
      {
        $merge: "aura-context",
      },
    ])

Check if changes have been applied. The following query should return n documents:

  > db["aura-context"].find({"document.skillConversationReference": { $exists: true }})

Rollback to the previous schema

If a rollback is needed, documents should be modified to have the previous schema and maintain the users’ context data:

> db["aura-context"].aggregate([
    {
      $match:
        {
          "document.activeSkill.conversationId": {
            $exists: true,
          },
          "document.skillConversationReference": {
            $exists: true,
          },
        },
    },
    {
      $project:
        {
          _id: "$document.activeSkill.conversationId",
          document:
            "$document.skillConversationReference",
          expiresAt: "$expiresAt",
        },
    },
    {
      $merge: "aura-context",
    },
  ])

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Groot databases

Description of databases used by Aura Groot

Introduction

Currently, aura-groot uses two databases, described below.

Mongo DB
Redis + MongoDB

MongoDB

In order to reuse the code and clarify the development, all classes related with MongoDB storage are stored in the mongodb-storage.ts file within aura-bot-common repository.

MongoDbStorage

This storage class will be used by Bot Builder v4 to store the conversation and user’s data. An index is created to delete documents older than 24 hours (specified by config var AURA_MONGODB_BOT_COLLECTION_CONTEXT_INDEX_TTL).

The following aura-groot environment variables are used to initialize this class:

Var name	Description	Required/Optional
AURA_MONGODB_BOT_DATABASE	Database name where the documents will be stored	Required
AURA_MONGODB_BOT_COLLECTION_CONTEXT	Collection name where the documents will be stored	Required
AURA_MONGODB_BOT_COLLECTION_CONTEXT_INDEX_TTL	Duration (in seconds) for which the documents will exist without updates before being deleted. By default, `86440`.	Optional

To create and initialize a MongoDbStorage, a code like the following below could be used:

    const storage = await MongoDbStorage.create(
            this.configuration.AURA_MONGODB_BOT_DATABASE,
            this.configuration.AURA_MONGODB_BOT_COLLECTION_CONTEXT,
            () => ConfigurationManager.instance.get('AURA_MONGODB_BOT_COLLECTION_CONTEXT_INDEX_TTL')
        );

Aura Groot cache module

The aura groot cache module allows the generation of different types of cache:

MetricCache: used for remote and local data. Is a TwoLevelsCache.

How to use cache module

This cache stores the timestamp when a request from the channel arrives at the aura-groot and is retrieved after the skill’s response to calculate the duration in the KPI GrootMessageEntity.

Variables for AuraGrootCache

The following aura-groot environment variables are used to initialize this class:

Var name	Description	Required/Optional
AURA_MONGODB_BOT_DATABASE	Database name where the documents will be stored	Required
AURA_MONGODB_CACHE_METRIC_COLLECTION	Collection name where the documents will be stored. By default, `metric-cache`	Optional
AURA_MONGODB_CACHE_METRIC_TTL	Duration (in seconds) for which the documents will exist in local cache before being deleted. By default, `60`.	Optional
AURA_MONGODB_CACHE_METRIC_INDEX	Index used in TwoLevelsCache of metric-cache. By default, `correlator`.	Optional

Example of data storage:

{
    "_id" : "64d0e9608a473f63c8bda87b",
    "correlator" : "6eea808b-30f8-4358-9859-cd42892a962b",
    "data" : [
        2339137,
        419794024
    ],
    "lastAccess" : "2023-08-07T12:53:52.370Z"
}

Redis + MongoDB

If the configuration variable AURA_BOT_CONTEXT_DATABASE is set to REDIS-MONGODB, aura-groot uses a two-level cache to store the context Aura Redis Mongo Sync.

The following environment variables must be set for the two databases.

Variables

Var name	Description	Required/Optional
AURA_MONGODB_BOT_COLLECTION_CONTEXT	Name of the collection where the context is stored. By default: `aura-context`.	Required
AURA_MONGODB_BOT_DATABASE	Name of the database where the bot collections are stored. By default: `aura-bot`	Required
AURA_MONGODB_PASSWORD	Password of the MongoDB. Required	Required
AURA_MONGODB_POOL_SIZE	Pool size of the MongoDB. By default: `60`.	Required
AURA_MONGODB_SSL	Boolean value indicating if MongoDB connection uses SSL or not. By default: `true`.	Required
AURA_MONGODB_URI	URI of the MongoDB	Required
AURA_MONGODB_USERNAME	Username of the MongoDB	Required
AURA_REDIS_CONFIGURATION_PREFIX	Prefix for Redis configuration keys. By default: `aura-config:`	Required
AURA_REDIS_CONTEXT_CACHE_PREFIX	Prefix for Redis context cache keys. By default: `context:`	Required
AURA_REDIS_CONTEXT_CACHE_SHADOW_KEY_PREFIX	Prefix for Redis shadow key cache keys. By default: `shadow-key:`	Required
AURA_REDIS_CONTEXT_CACHE_ACTIVE_CONTEXT_PREFIX	Prefix for Redis active context cache keys. By default: `active-context:`	Required
AURA_REDIS_CONTEXT_CACHE_SHARD_COUNT	Number of shards to generate lists for storing unprocessed context.	Required
AURA_REDIS_CONTEXT_CACHE_TTL	Time in seconds to store a context in cache. By default: `60`.	Required
AURA_REDIS_MODE	Mode of Redis distribution (`Cluster`, `Sentinel` or `Single`). By default: `Single`.	Required
AURA_REDIS_PREFIX	Prefix that will be used by all redis keys when using redis-connector. This allow mixing in a single redis service messages coming from different environments in the same Azure subscription.	Required
AURA_REDIS_SENTINEL_INSTANCE_NAME	Name of the Redis instance. Use in sentinel mode.	Optional
AURA_REDIS_HOSTS	A string with list of nodes (host:port) separate by comma. For example: “localhost:port,localhost2:port2”. Default: ‘127.0.0.0:6379’	Required
AURA_REDIS_DATABASE Database number for `Single` or `Sentinel` mode. By default: `0`.
AURA_REDIS_PASSWORD	A string with the password of Redis.	Required
AURA_REDIS_MAX_RECONNECT_RETRIES	Number of retries to connect to Redis. By default: `25`.	Required
AURA_REDIS_MAX_RECONNECT_INTERVAL	Time in milliseconds to wait before reconnecting to Redis. By default: `5000`.	Required
AURA_REDIS_USE_CONNECTION_POOL	Used connection pool for Redis connections. Default: `true`.	Required
AURA_REDIS_CONNECTION_POOL_MIN	Minimum number of connections in the pool. Default: `2`.	Required
AURA_REDIS_CONNECTION_POOL_MAX	Maximum number of connections in the pool. Default: `100`.	Required

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura microfrontend base

Description of aura-microfrontend-base, foundational component for the development of user interfaces in Aura ecosystem

Aura Virtual Assistant component

Introduction

aura-microfrontend-base is a foundational component that provides a common structure and libraries for the development of user interfaces in Aura ecosystem (Aura Virtual Assistant and ATRIA).

Its microfrontend component is a small, self-contained element that is part of a larger microfrontend architecture. It follows the same principles as microservices, but for the frontend, allowing independent development, deployment and scaling of different parts of a web application.

As defined before, this is a base component over which certain interfaces will be developed, but it will not be directly productized.

Technical foundations

This is a Next.js project bootstrapped with create-next-app.

The project leverages the following technologies and tools:

React: A JavaScript library for building user interfaces, used as the core framework for component development.
@telefonica/mistica: A design system and component library that provides a set of reusable UI components and styles, ensuring consistency and best practices in UI development.
Tailwind CSS: A utility-first CSS framework for styling components and layouts.
TypeScript: A strongly typed programming language that builds on JavaScript, ensuring type safety and better developer experience.
Microfrontend Architecture: Enables independent development, deployment, and scaling of frontend components.
Dynamic Imports: Used to load microfrontend components at runtime, improving modularity and performance.
Environment Variables: Configuration is managed through environment variables like AURA_SITE_CONFIG_PATH to ensure flexibility and adaptability across environments.

These foundations ensure scalability, maintainability, and a seamless developer experience.

Architecture and components

aura-microfrontend-base is composed of two main components:

Core Framework: The foundational Next.js structure that handles routing, rendering, and component lifecycle.
Dynamic Component Loader: The system responsible for loading and rendering components based on the site configuration.

Basic architecture diagram

sequenceDiagram
    participant Project as user request
    participant Config as site-config.json
    participant Pages as Page Manager
    participant Layout as Layout Engine
    participant ComponentLoader as Component Loader
    participant DOM as Browser DOM

    Project->>Config: read site configuration
    Config-->>Project: return site structure
    
    Note over Project: site path or default page
    Project->>Pages: setup page routes from config
    
    Pages->>Layout: process page layout configuration
    
    Layout->>Layout: setup container with containerClass
    Layout->>Layout: create grid with gridClass
    
    Note over Layout: process components by position
    
    loop for each component
        Layout->>ComponentLoader: request component by importType
        
        alt importType is "internal"
            ComponentLoader->>ComponentLoader: load from local components folder
        else importType is "external"
            ComponentLoader->>ComponentLoader: load from library dependency
        else importType is "microfrontend"
            ComponentLoader->>ComponentLoader: load dynamically from remote source
        end
        
        ComponentLoader-->>Layout: Return component
        Layout->>Layout: Apply className and position
        Layout->>Layout: Inject attributes and props
        Layout->>DOM: Render component in grid position
        DOM-->>Layout: Component rendered
    end
    
    Layout-->>Pages: Page layout rendered

    
    Pages-->>Project: page configured and route ready
    Project->>Project: start listening for navigation

Server features

The server component of aura-microfrontend-base provides:

Dynamic Routing: Routes are generated based on the configuration file and mapped to corresponding components.
Server-Side Rendering (SSR): Components can be rendered on the server for improved performance and SEO.
API Integration: Built-in capabilities for connecting to backend services and APIs.
Environment Variable Management: Configuration is handled through environment variables for different deployment environments.
Development Server: Local development environment with hot-reloading for a smooth development experience.

Configuration

The AURA_SITE_CONFIG_PATH environment variable must be set to point to your project’s configuration file, as in the example below:

AURA_SITE_CONFIG_PATH=site-config.json npm run dev

aura-microfrontend-base configuration file must be a valid JSON that defines the structure of the site, including the layout of the pages and the components to render.

The configuration file must follow this structure:

{
    "mainPage": "home",
    "pages": [
        {
            "id": "home",
            "name": "Home",
            "path": "/home",
            "layout": {
                "containerClass": "min-h-screen flex flex-col container mx-auto",
                "gridClass": "grid grid-cols-1 grid-rows-[200px_1fr] flex-1",
                "components": [
                    {
                        "library": "@telefonica/example-library",
                        "componentName": "Header",
                        "position": 1,
                        "importType": "external",
                        "className": "col-span-2 row-span-1"
                    },
                    {
                        "componentName": "TestComponent",
                        "position": 2,
                        "importType": "internal",
                        "className": "h-full col-span-1 row-span-2"
                    },
                    {
                        "componentName": "TestPlugin",
                        "position": 3,
                        "importType": "microfrontend",
                        "className": "h-full col-span-1 row-span-2",
                        "attributes": {
                            "API_ENDPOINT": "${AURA_MICROFRONTENDS_SERVER_HOST}/api/plugin-test",
                            "REFRESH_INTERVAL": "1000"
                        }
                    }
                ]
            }
        }
    ]
}

Configuration fields

mainPage: Default page ID to load when accessing the root path
pages: Array of page configurations
- id: Unique identifier for the page
- name: Display name of the page
- path: URL path for the page
- layout: Container configuration - containerClass: Tailwind classes for the main container - gridClass: Tailwind classes for the grid layout - components: Array of component configurations - library: (Optional) External library name - componentName: Name of the component to render - position: Order in the layout - importType: Type of component (internal, external, or microfrontend) - internal: Components that are part of the current project and are located in the components directory - external: Components from external libraries (like @telefonica packages) that are installed as dependencies - microfrontend: Components that are loaded dynamically from a remote server, allowing for runtime integration of independent applications - className: Tailwind classes for the component - attributes: (Optional) Additional component properties

Technical guidelines

Guides to manage the aura-microfrontend-base component: