Aura –

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura common utilities

Description of aura-common-utilities

Introduction

aura-common-utilities is a multipackage repository with all the libraries provided by Aura team to make development easier.

aura-common-utilities repository is organized into four packages:

aura-bot-utilities, used for handling different tasks over aura-bot.
aura-utilities, used by other Aura components.
aura-develop-utilities, that include utilities used for different development purposes.
aura-clients referred to tools that make easier to call APIs.

Additionally, there are certain utilities that have not been migrated to these packages yet and are documented outside the packages sections.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Develop a plugin and activate it in Aura Bridge

Guidelines for the generation of a new plugin and its activation/deactivation in Aura Bridge

Introduction

The process for the generation of a plugin and its activation or deletion in aura-bridge includes the following steps:

Plugin development

Install aura-development-cli (aucli)

To generate a new plugin, use aucli, a set of tools available in the “Command Line Interface” format to help you with development and testing.

How to use aucli?

Access the “help” command:

$ aucli bridge -h

Usage: aucli bridge [options] [command]

tasks associated with 'bridge' commands

Options:
  -h, --help      display help for command

Commands:
  generate        tasks associated with 'generate' commands
  help [command]  display help for command

Execute the following command in the aura-bridge project path:

$ aucli bridge generate plugin

✔ Whats the origin of the process? (ex.: Directline, Whatsapp, ...)? · channel1
✔ Whats the destination of the process? (ex.: Directline, Whatsapp, ...)? · channel2
✔ The name of processor is correct? (y/N) · true

The following files will be generated in aura-bridge:

added: src/plugins/channel1-channel2-processor/channel1-channel2-consume-services.ts
added: src/plugins/channel1-channel2-processor/channel1-channel2-controller.ts
added: src/plugins/channel1-channel2-processor/channel1-to-channel2-converter.ts
added: src/plugins/channel1-channel2-processor/error-to-channel2-api-converter.ts
added: src/plugins/channel1-channel2-processor/channel1-channel2-flow.ts
added: src/plugins/channel1-channel2-processor/index.ts
added: src/plugins/channel1-channel2-processor/package.json
added: src/plugins/channel1-channel2-processor/channel1-channel2-destination-channel2-response-error.ts
added: src/plugins/channel1-channel2-processor/channel1-channel2-utils.ts

Add a new swagger

Add a swagger inside your plugin in the path:
src/plugins/channel1-channel2-processor/swagger.yaml

openapi: 3.0.0
info:
  title: administration api to aura bridge
  description: Set of endpoints related to bridge administration
  version: 1.0.0
tags:
  - name: admin
    description: Bridge administration plugin
servers:
  - url: 'http://localhost:8045'
    description: Local server
  - url: 'https://svc-ap-current.auracognitive.com'
    description: ap-current
paths:
  /aura-services/v1/testing/channel1: # include the path to the new plugin
      get:
        tags:
          - testing-channe1
        description: channel1 testing.
        operationId: getChannel1Messages # controller function name.
        x-router-controller: plugins # in path src/controllers/plugins.ts
        responses:
          '200':
            description: OK
            headers: { }
          '500':
            description: An internal server error has occurred.

Add a new function controller

Add all the controllers in the path: src/controllers/plugins.ts

import Channel1Channel2Message from '../plugins/channel1-channel2-processor/channel1-channel2-controller';

export function getChannel1Messages(req: express.Request, res: express.Response) {
    Channel1Channel2Message.controller(req, res);
}

Plugin activation in Aura bridge

Every plugin to be loaded in aura-bridge must be defined in the plugin-config.json file.

⚠️ If this plugin has dependence with other plugins, they must also be added to the plugins list. If during the aura-bridge boot a plugin has a dependency with another not installed plugin, the bridge will fail in the boot process and will log an error indicating the lost dependency.

Add the plugin to the configuration file plugin-config.json (located at the root of the aura-bridge project):

[
    ...
    "./lib/plugins/channel1-channel2-processor",
    ...
]

Activate a new plugin during development

To activate a new plugin during development, it is only necessary to add the plugin path in the plugin-config.json file:

[
    "./lib/plugins/directline-service",
    "./lib/plugins/directline-whatsapp-processor",
    "./lib/plugins/whatsapp-incoming-processor",
    "./lib/plugins/whatsapp-service",
    // New plugin location here.
]

⚠️ take into account the dependencies with other plugins, as explained in the previous section.

Activate a new plugin for production

To add a new plugin in production, it is necessary to define the list of plugins using the plugin-config.json key in the aura-bridge ConfigMap for deployment in Kubernetes.

Example of aura-bridge ConfigMap using a configuration file:

apiVersion: v1
kind: ConfigMap
metadata:
  name: aura-bridge
  labels:
    app: aura-bridge
    stack: service
  namespace: {{ kubernetes_core_namespace }}
data: 
  plugin-config.json: |
    {{ lookup('file','<FOLDER_PATH>/plugin-config.json',convert_data=False) | indent(4,indentfirst=False) }}

Plugin removal in Aura bridge

To remove a plugin, it is only necessary to delete the plugin path in the plugin-config.json file.

⚠️ In order to eliminate a plugin correctly, it is necessary to verify that no other plugins from those installed depends on the one we want to eliminate.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Develop a filter for Engine Filter API

Guidelines for the knowledge and generation of filters for Engine Filters API.

Introduction

A filter is a set of configurations and instructions that allow data to be grouped and queried in order to determine whether the flow of a use case should be changed at a given time. For example, if we have a use case that queries an LLM, we can create a filter to limit the use of these LLMs by user, or by channel at a given time. For this documentation we are going to make a filter that limits the use of LLMs by user and use case, we will describe it later.

Engine Filter API

Anatomy of a filter

A filter is composed of a descriptive part, a configuration part, and an execution instruction part. Both parts are stored in JSON format. The service in charge of the storage is the Aura Configuration API. [TODO url]. The execution part is done by MongoDB instructions.

Descriptive Properties

Field name	Type	Required	Description
id	string	true	UUID that uniquely identifies a routing filter in Aura.
name	string	true	Name that uniquely identifies a routing filter in Aura.
description	string	false	Routing Filter description.
type	string	true	Contains the type of filter. Currently, there is only one type ‘userId’.

    {
        "name": "preset-filter-stb-conversational-search",
        "id": "4a879583-5f76-4e6b-87c1-6250e8743dda",
        "description": "Limit the number of messages per user in a month for stb conversational search preset",
        "type": "userId"
        ....
    }

Configuration Properties

Field name	Type	Required	Description
entities	string[]	true	Contains at least one entity necessary to generate the data for the filter.
dataBase	object	true	Contains an object with the collections necessary to store and process data. See database object.
vars	object	true	Contains an object with the custom variables necessary for any phase of the filter. Encrypted field.
fields	object	true	Contains the field mapping for grouping and its relationship with the previously defined entities. Encrypted field.
sourceFilters	object	false	Contains an object to filter source data when loading from entity data. Encrypted field.

    {
        ....
       "entities": [
            "GATEWAYMESSAGE"
        ],
        "vars": {
            "llm_execution_limit": 10
        },
        "dataBase": {
            "dataFilterCollection": {
                "collectionName": "dataFilterPreset",
                "expiration": 5356800,
                "indexes": [
                    {
                        "seqId": 1
                    }
                ]
            },
            "dataSummaryCollection": {
                "collectionName": "dataSummaryStbConvSearch",
                "expiration": 5356800,
                "indexes": [
                    {
                        "month": 1,
                        "total": 1,
                        "year": 1
                    },
                    {
                        "itemId": 1,
                        "month": 1,
                        "year": 1
                    }
                ]
            }
        },
        "fields": {
            "MatchingValue": "^ef3d0603-3fef-4109-a577-0ab92f9060df$",
            "forId": "USER_ID",
            "forMatchingField": "AURA_PRESET_NAME",
            "forTime": "MESSAGE_TM"
        },
        "sourceFilters": [
            {
                "field": "USER_ID",
                "op": "notEqual",
                "val": ""
            }
        ],
        ....
    }

Execution Properties

Field name	Type	Required	Description
match	object	true	Contains an object with the MongoDB aggregation format that must return grouped data. Encrypted field.
summary	object	true	Contains an object with MongoDB command format to insert these obtained aggregates into a summary collection. Encrypted field.
actions	object[]	true	Contains an array of objects with the actions to be performed on the filtered data.
summaryFilter	object	true	Contains an object with MongoDB command format to select the records that meet the filter. Encrypted field.

...
 "match": [
            {
                "$match": {
                    "fieldForMatch": "{{fields.forMatchingField}}",
                    "seqId": {
                        "$gt": "{{ctx.lastSeqId|number}}"
                    },
                    "valueForMatch": {
                        "$options": "i",
                        "$regex": "{{fields.MatchingValue}}"
                    }
                }
            },
            {
                "$group": {
                    "_id": {
                        "itemId": "$itemId",
                        "month": "$month",
                        "year": "$year"
                    },
                    "seqId": {
                        "$max": "$seqId"
                    },
                    "total": {
                        "$sum": 1
                    }
                }
            },
            {
                "$project": {
                    "_id": 0,
                    "itemId": "$_id.itemId",
                    "month": "$_id.month",
                    "seqId": 1,
                    "total": 1,
                    "year": "$_id.year"
                }
            }
        ],
        "summary": {
            "filter": {
                "itemId": "{{doc.itemId}}",
                "month": "{{doc.month|number}}",
                "year": "{{doc.year|number}}"
            },
            "options": {
                "upsert": true
            },
            "update": {
                "$inc": {
                    "total": "{{doc.total|number}}"
                },
                "$set": {
                    "expiresAt": "{{ctx.expiration|expires}}",
                    "month": "{{doc.month|number}}",
                    "updatedAt": "{{__DATE_NOW__}}",
                    "year": "{{doc.year|number}}"
                }
            }
        },
        "summaryFilter": {
            "filter": {
                "month": "{{ctx.month|number}}",
                "total": {
                    "$gt": "{{vars.llm_execution_limit|number}}"
                },
                "year": "{{ctx.year|number}}"
            },
            "options": {
                "projection": {
                    "_id": 0,
                    "itemId": 1,
                    "month": 1,
                    "total": 1,
                    "year": 1
                }
            }
        }

Create, execute and check a filter

To see all the steps necessary to create and use the filter, let’s use the example we discussed in the introduction.

Let’s suppose that we need a filter so that the users have a limit of access to the LLM APIs per use case. We are going to set a limit of 150 interactions per month. And this filter should reset those results at the beginning of each month.

Create a filter. Steps

Filters use data generated for KPIs, so the most important thing before starting to define a filter, is to know which entity has all the necessary fields to be able to extract the necessary conditions for the filter. Note that currently only one Entity can be used per filter, in later versions this may change and support multiple entities.

For our example we will identify what data we need

Date of interaction
User identifier Preset identifier

For our filter the only entity that contains all these data is GATEWAYMESSAGE

Iteration date -> MESSAGE_TM User ID -> USER_ID Preset identifier -> AURA_PRESET_NAME The value of the preset to which we are going to add the filter will be: “ef3d0603-3fef-4109-a577-0ab92f9060df”.

Once the Entity has been identified we proceed to create the descriptive part of the filter.

        "name": "preset-filter-for-users",
        "id": "1a879583-5f76-4e6b-87c1-6250e874bbs",
        "description": "Limit the number of messages per user in a month for LLM User Case",
        "type": "userId"

As we have the entity we are going to use we set it:

       "entities": [
            "GATEWAYMESSAGE"
        ]

We associate the identified fields in the CSV.

        "fields": {
            "MatchingValue": "ef3d0603-3fef-4109-a577-0ab92f9060df",
            "forId": "USER_ID",
            "forMatchingField": "AURA_PRESET_NAME",
            "forTime": "MESSAGE_TM"
        }

We will set the limit to 150 and put it in a variable.

        "vars": {
            "llm_execution_limit": 150
        }

As the USER_ID field is optional, we will filter out those records that do not have USER_ID, i.e. USER_ID notEqual "". This is done in the sourceFilters property. Although it is an array, it currently supports only one element. The available operators are: major, minor, equal and notEqual. SourceFilters also supports other data types for comparison, such as date or number. This is set with the cast: number|date property, otherwise the default is string.

     "sourceFilters": [
            {
                "field": "USER_ID",
                "op": "notEqual",
                "val": ""
            }
        ]

We will need to name the collections and the lifetime of the data in seconds.

      "dataBase": {
            "dataFilterCollection": {
                "collectionName": "dataFilterPreset",
                "expiration": 5356800
            },
            "dataSummaryCollection": {
                "collectionName": "dataSummaryFilterPreset",
                "expiration": 5356800
        }

We have already established all the values for the descriptive part and the configuration part, we must take into account that when we finish defining the execution part we will have to add some more data, such as indexes, if we want the filter execution to be done with the maximum performance.

Create Execution definition

Once a filter has been created, the KPI utilities will already know which entities have to send the Engine Filter API in order to process and group data and then validate the filters with them.

In our example, in order to group those data we need the dataFilterCollection collection that will contain the raw data extracted from the KPIs entity.

  "forId": "USER_ID",                     -> itemId
  "forMatchingField": "AURA_PRESET_NAME", -> fieldForMatch and valueForMatch
  "forTime": "MESSAGE_TM"                 -> year, month, expires and seqId

Example:

{
    "seqId" : 1756883926388.0,                                   -> Timestamp to generate partial summaries
    "itemId" : "IN-ewXp7S8u4mL0RlKJStA",                         -> UserId
    "year" : 2025,                                               -> Year
    "month" : 8,                                                 -> Month 
    "fieldForMatch" : "AURA_PRESET_NAME",                        -> fieldForMatch         
    "valueForMatch" : "a652we235-3fef-4109-a577-0909d8ef234567", ->valueForMatch
    "expiresAt" : "2025-09-03T07:20:15.388Z"                     -> expiration date
}

In our example we have to look for those that have the valueForMatch equal to the one defined in fields MatchingValue and group the data by userId, month, year, counting the number of iterations, i.e. we are looking for something like this:

{
    "month" : 8,
    "itemId" : "bk3iI3GwQWO2-4YI0etc0w",
    "year" : 2025,
    "expiresAt" : "2025-09-04T09:13:48.345Z",
    "total" : 40
}

Before we start defining the execution properties, let’s explain the format in which they are defined.

JSON with PLACEHOLDERS

The JSON that compose the properties with execution parameters have placeholders to be able to inject variables and other values that have other properties.

Prefixes

vars: Refers to the properties defined inside the vars object. Example ````vars.llm_execution_limit``` in our example will have the value 150.
fields: Refers to the properties defined inside the fields object. Example: ````fields.MatchingValue```.
doc: It refers to the document returned by mongodb in the previous execution. It will only appear in the summary property since it is formed with the results of a previous query. Example: ````doc.itemId```. In order to use these properties they must be inside the $project property of the match property.
ctx: Contains variables that can be used to insert in execution properties, these are:
- ctx.day: Today’s day.
- ctx.month: Current month.
- ctx.year: Current year.
- ctx.lastSeqId: Last seqId processed to create summaries.
- ctx.expiration: Expiration time of the data to be generated.

Data Types

number: Numeric, integer or float type. Example: {{ctx.month|number}}.
date: Date type. Example: {{var.EDate|date|date}}.
expires: Current date + placeholder value. Example: {{ctx.expires|expires}}.
boolean: Boolean. Example: {{{var.isTrue|boolean}}.
string: String, if nothing is set it is the default string. Example: {{doc.itemId}} or {{doc.itemId|string}} .
lower: String in lowercase. Example: {{doc.itemId|lower}}.
upper: String in uppercase. Example: {{doc.itemId|upper}}.
trim: Removes whitespace from a string. Example: {{doc.itemId|trim}}.

Variables
- _DATE_NOW_: Contains the current date. Example: "updatedAt":"{{__DATE_NOW__}}".

Create execution properties: match, summary and summaryFilter

Once we have data in the raw data collection what we need is

1.- Get the data that has not been processed yet filtering by a certain condition and grouping them in this case by ItemId, month, year, and finally returning a data model to generate the summaries. This is achieved in the match property:

Step A: $match. Select data to filter by a regex.


"$match": {
    "fieldForMatch": "{{fields.forMatchingField}}",
    "seqId": {
        "$gt": "{{ctx.lastSeqId|number}}"
    },
    "valueForMatch": {
        "$options": "i",
        "$regex": "{{fields.MatchingValue}}"
    }
}

In our case this would be to select those elements that the field “AURA_PRESET_NAME” has the value “ef3d0603-3fef-4109-a577-0ab92f9060df” and whose seqId is greater than the last processed seqId.


"$match": {
    "fieldForMatch": "AURA_PRESET_NAME",
    "seqId": {
        "$gt": "1099288387374"
    },
    "valueForMatch": {
        "$options": "i",
        "$regex": "ef3d0603-3fef-4109-a577-0ab92f9060df"
    }
}

Step B: $group. We group by itemId, month and year. We store the last seqId in $max.


"$group": {
   "_id": {
       "itemId": "$itemId",
       "month": "$month",
       "year": "$year"
   },
   "seqId": {
       "$max": "$seqId"
   },
   "total": {
       "$sum": 1
   }
}

This will be grouping the data, adding the matching elements and obtaining the maximum seqId used. The latter will be stored in ctx.lastSeqId.

Step C: $project. We return the grouped data to update or create the elements in the summary collection.


"$project": {
    "_id": 0,
    "itemId": "$_id.itemId",
    "month": "$_id.month",
    "seqId": 1,
    "total": 1,
    "year": "$_id.year"
}

2.- We update or create the grouped data, this is included in the summary property.


"summary": {
    "filter": {
        "itemId": "{{doc.itemId}}",
        "month": "{{doc.month|number}}",
        "year": "{{doc.year|number}}"
    },
    "options": {
        "upsert": true
    },
    "update": {
        "$inc": {
            "total": "{{doc.total|number}}"
        },
        "$set": {
            "expiresAt": "{{ctx.expiration|expires}}",
            "month": "{{doc.month|number}}",
            "updatedAt": "{{__DATE_NOW__}}",
            "year": "{{doc.year|number}}"
        }
    }
}

The doc prefix refers to the result of the previous execution of match. Once the summary has been executed, we will already have in the dataSummaryCollection data to query if they comply or not with the filter.

3.- Obtain the data that meet the filter. summaryFilter


"summaryFilter": {
    "filter": {
        "month": "{{ctx.month|number}}",
        "total": {
            "$gt": "{{vars.llm_execution_limit|number}}"
        },
        "year": "{{ctx.year|number}}"
    },
    "options": {
        "projection": {
            "_id": 0,
            "itemId": 1,
            "month": 1,
            "total": 1,
            "year": 1
        }
    }
}

In this case it looks for records that comply with the current month and year and that exceed the limit configured in vars.llm_execution_limit. The filter engine API stores the elements that exceed the filter in a cache each time the dataSummaryCollection is updated.

Check filters

Filters can be queried via Configuration API calls: https://ENV.auracognitive.com/aura-services/v2/routing-filters/FILTER_ID/items/ITEM_ID, or within a context filter.

Example:

{
  "name": "tvCustomRecommendation",
  "dialogs": [
    {
      "id": "tv-custom-recommendation",
      "onlyIn": [
        "set-top-box"
      ],
      "allowAnonymous": true,
      "triggerConditions": [
        {
          "intent": "intent.tv.custom_recommendation",
          "contextFilters": [
            {
              "name": "limit-num-messages-LLM-user",
              "type": "RoutingByFilter",                             <-- Set the type
              "conditions": "4a879583-5f76-4e6b-87c1-6250e8743dda",  <-- FilterId
                "name": "send-custom-messsage",
                "breakDialogExecution": true,
                "breakFilterEval": true,
                "redirectToIntent": "intent.send-custom-message",
                "resource": "intent.send-custom-message:default.message",
                "removeBypass": true,                                <-- Remove bypass
                "suggestions": false
              }
            }
          ],
          "settings": {
            "action": "noAction",
            "sound": "positive",
            "type": "common"
          }
        }
      ],
      "bypass": {
        "duration": 4,
        "payloadName": "openai",
        "initialData": {},
        "recognizersEnabled": true,
        "recognizersBreakIntents": {
          "intent.tv.display": [
            "[Display Channel]",
            "[Display Contents]"
          ],
          "intent.navigation.section_show": [
            "[Sections]"
          ]
        }
      }
    },

To include a filter inside a context filter the following properties must be included:

type: Must have the type “RoutingByFilter “.
conditions: It must have the id of the Filter we want to consult.

If the dialog has an associated Bypass it is important to add “removeBypass”: true inside the “true “ property.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Contributing to Aura

Discover how to contribute to Aura Platform global repositories

Introduction

The aim of this document is to establish common minimum rules for developers willing to contribute to Aura Global repositories:

How to request access to Aura repositories in Github
How to write code in aura-bot global repositories
How to migrate a git folder to a new repository
How to manage user stories and tasks in JIRA
Types of branches and guidelines for the generation of a new branch

The target audience of the document are developers of global and local teams contributing to Aura Global repositories:

aura-bot developers
aura-bridge developers
aura-services-api developers
…

⚠️ This document assumes that the developers already have permissions to access Telefónica organization in Github.

Branches used by Aura

Aura uses 2 different types of branches:

Eternal or pseudo eternal branches

These are protected branches, that MUST be always deployable in a given environment.

Protected means:

Only code owners can push code to them
All changes MUST be pushed through a pull request, that MUST pass the continuous integration checks and count on, at least, 2 approval reviews of code owners.

These branches are:

master: it holds next release code. It MUST be deployable in the global development environments.
main/*: it holds the development of a unique User Story or set of User Stories.
release/*: it holds the given release code. It MUST be deployable in global development environments and also in PRE and PRO of the OBs involved in this release developments.
hotfix/*: it holds the resolution of an issue or set of issues happening in a pre or pro environment.

Temporary branches

As any change in eternal branches MUST be done though a Pull Request, it means that they must be done in a temporary or feature branch.

These branches are usually named after the type of change they are going to hold: feat/my-new-feature or fix/fixing-that.

This format is <type>/<description>[#issue_number] where:

type is one of feat, fix, chore and docs
description should be a short description, dash-separated words, of the change. If using Jira as issue manager, add the number of the issue being resolved after a #.

Process for generating a new branch

First of all, clone the repository and choose an eternal branch as base branch for your feature:

$ git clone git@github.com:Telefonica/aura-bot-platform.git
$ cd aura-bot-platform
$ git checkout main/my-feature#12345

Or update the base eternal branch for your feature:

$ cd aura-bot-platform
$ git checkout main/my-feature#12345
$ git pull origin main/my-feature#12345

Then, create a new branch:

$ git checkout -b feat/my-new-feature

Some recommendations should be considered:

Split your changes in small tasks (max 1 day per task)
Use a feature (feat, fix, chore, …) branch per task
Do small commits with spotted functionality
Do small PRs with all the code of a given task
When asking a PR remember to include an explanation of the why, how, what has been done in the PR, in order to help reviewers to understand the change.

Commit messages format

The format followed in Aura is the one described in Conventional Commits.

These messages are used to handle the version of each Aura component so it is very important to follow these rules.

NOTE: Pull Requests including commits without this format will be rejected.

Format

<type>(<optional scope>): <description>

[optional body]

Valid types: feat, fix, chore, docs
Valid scopes:
- global, can be omitted, if affects all the packages or it’s a basic change.
- the affected package in a multipackage repository.
- the affected library, plugin or dialog in library repository.
- if it affects a concrete feature in the repository: logging, kpis, authentication, etc.
- it can also be omitted in global cases, in minor changes, etc.
Description: indicative sentence in lower case.
Body: only if description field is not enough to understand the changes

Folder layout

In general, all the repositories containing typescript code should have an src folder with all the typescript source files there, and those source files will be transpiled into javascript into folder lib.

Unit tests

Currently aura-bot team is moving to Jest as unittest runner, so a couple of versions of unittest can be found:

Projects running with mocha, such as aura-bot-platform.

Test files should be included in the same folder as the files to test, with a trailing .spec.ts in the file name (these files should be .npmignore, to avoid being in the final package).

Test assets required for tests (.json files, images, .env files, etc.) will be placed in a test folder (that should be .npmignore too), so they will be available for typescript/javascript files by getting the path ../test or similar.

Projects running with jest, such as aura-mocks-server.

There will be a specific folder called test, at the same level than src and that will hold all the unit tests following the same structure than src. As with mocha tests, they must be ignored in the production transpilation and the files published in npm.

More about src/ folder layout in aura-bot-platform can be found in this file.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Upgrade Aura bot to node 18 and its dependencies

Guidelines for the upgrade of aura-bot to node 18 and the update of its major dependencies, in order to avoid end-of-life of current versions

Introduction

Current version of node being used is 14 LTS, that finished active support in October 2021 and security support in April 2023. In order to avoid the inconveniences of working with an old version, without security updates, we will update to current latest LTS version 18, with active support until October 2023 and security support until April 2025.

Moreover, some major dependencies are also upgraded in order to keep the latest available versions, with new features and security updates.

:warning:

The upgrade of node and dependencies in aura-bot comes together with the migration to Bot Framework 4.19..

The latter implies that local use cases must be migrated to this new version to be operative. For this purpose, OBs must follow the guidelines Migrate use cases to Bot Framework 4.19.

Guidelines for upgrade

In this section, a list of mandatory and recommended changes will be explained, but consider applying them both, as it will prevent future incompatibilities.

Mandatory changes

Docker base

Docker base image was changed from node:14.17.1-alpine3.13 to node:18.16.0-alpine3.17, in bot Dockerfile and mini-bot Dockerfile. If using a different container on deploys, it must be upgraded to node 18.

Internally, the jenkins slave was set to aura-node18-ubuntu22 on Jenkinsfile (only affects to the global team).

Node types update

Package with node types must be upgraded to version 18 in the package.json file:

{
    "@types/node": "^18.16.0"
}

Upgrade Typescript

Typescript was upgraded in aura-bot to version 5, so it must be also upgraded in all packages and libraries. For this purpose, modify the typescript dependency in the required package.json files:

{
    "typescript": "~5.0.0"
}

Update package-lock.json

After upgrading node version to 18, npm is also upgraded to version 9.5.1, and the package-lock.json is obsolete. Running a one-time npm install will upgrade lockfileVersion to 3, and the required content, such as sha512 integrity hashes instead of sha1.

It is also recommended to delete the whole file package-lock.json before npm install, to upgrade also dependencies to newer versions.

Recommended changes

In this section, other optional upgrades are shown, highly recommended in order to avoid future compatibility problems. We encourage applying them, in addition to the mandatory changes.

mongodb-memory-server

It is recommended to upgrade mongodb-memory-server to the latest version 8.12.2, due to deprecation of current version and, moreover, in order to prepare the code to near future upgrade of MongoDB (from version 4 to version 6). The upgrade requires a minimal code update.

npm-check-updates

It is also recommended to upgrade npm-check-updates to latest version 16.10.9, as current version has a transitive dependency with an obsolete node-gyp that causes incompatibility when using Python >= 3.11, that is the default version in Ubuntu 22.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Release Train Manager

Release Train Manager (RTM) implementation and architecture

Introduction to RTM development methodology

The “release train” software development methodology is a way of planning the delivery of software according to a predetermined and regular schedule, as if it were a “train timetable”. This schedule is public for all the teams that must contribute to the delivery and represents a commitment on their part, as the schedule must be adhered to.

If we use the metaphor: “the train will take the passengers who are ready to travel”, i.e., only the software that is ready to be integrated will be integrated. For this it is necessary to work on the different functionalities in parallel and in isolation without affecting the rest.

We start from a prioritized backlog. The tasks will enter the implementation phase in order of priority.

Github is used for the organization of the process. See here the details of Github as orchestrator.
The file and properties for the configuration of branches is included in Branch configuration.
Learn how locales files and the POEditor project are managed in RTM: Managing locales.

RTM overview

The important thing when we are working on a new feature is to be able to completely isolate the development, environment and testing. To do this, environments will be generated per feature and git branches will be used for the different repositories involved.

RTM Overview

INIT: initial processing of a feature (select repos involved, create branches on those repos and create environment).
TF (Tests Feature): daily smoke tests on feat/new-feature branches.
TFPR (Tests Feature PR): full regression tests on the master merge and the feat/new-feature branch.
TM (Tests Master): complete daily/weekly regression tests on the master branches.
PR Fx (PR Feature): PRs on the feat/new-feature branch.
PR Mx (PR Master): PRs on the master branch.