1 - Dependencies update in Prince 10.0.0 release

Dependencies update in Prince 10.0.0 release

These documents include all the updates in components that have been carried out for the release Prince 10.0.0, along with the corresponding migration or adaptation tasks that should be executed by the OBs Teams.

Index

1.1 - Python repositories migration

Python repositories migration

Migration of Python repositories from Python 3.9 to Python 3.13 version for PyUtils, Aura NLP, Aura Complex Logic Framework plugin packages and ATRIA repositories

Introduction

This document describes the migration process of Python repositories from Python 3.9 to Python 3.13 1 for PyUtils, Aura NLP, Aura Complex Logic Framework plugin packages, ATRIA and other productive Python repositories that has taken place in Prince 10.0.0 release.

This migration is required due to the end of life of Python 3.9, set in October 2025.

PyUtils migration

PyUtils contains multiple repositories that have been migrated to Python 3.13 version by Aura Global NLP Team.

The use of these libraries is the same as before the migration, but OBs constructors must test the use cases to ensure that everything is working correctly.

NLP packages migration

The Aura Global NLP team has migrated the Python repositories to Python 3.13 version and has generated the different NLP packages for the OBs.

Two branches will live together:

  • release/7.0: Compatible branch with Python 3.9 version. If the deployed OB release is previous to Prince 10.0.0.
  • release/8.0: Compatible branch with Python 3.13 version. After Prince 10.0.0 release delivery.

The release/8.0 branch will be launched together with Prince 10.0.0 delivery. However, release/7.0 will remain active until the OB deploys Prince in production environment and even after that, for any issue or hotfix required.

The repositories of NLP training has a JenkinsFile 2 that defines the pipeline in Jenkins for the continuous integration of the repositories. For release/7.0 branch, the JenkinsFile of Aura NLP repositories will be configured for working with Python 3.9 version in the pipelines defines in Jenkins, to allow OBs to continue creating NLP packages, until they deploy this new release.

When deploying Prince release and changing from release/7.0 to release/8.0, both branches must be synchronized to preserve all modifications made in the older one.

Constructors should not do any change, as no trainings with the QnA and LUIS stages should exist (these stages are deprecated and must have already been replaced by the OpenAIEmbeddings and CLU stages respectively).

With the NLP packages provided by Aura Global NLP team, the OB constructors must test and validate that all the use cases are working correctly.

QnA and LUIS recognizers deprecation

The QnA and LUIS recognizers are deprecated and have been replaced by the OpenAIEmbeddings and CLU recognizers respectively in all trainings.

NLP Resources

The repositories resources listed below, which are the ones in production environment, have been migrated:

  • AP
    • resources_pizza
    • resources_ap-demo
    • resources_cr
  • ES
    • resources_tiempo
    • resources_real-academia-historia
    • resources_core-dialogs
    • resources_bingo
    • resources_movistar-shop
    • resources_movistar-gaming
    • resources_es-nov
    • resources_memory
    • resources_leia
    • resources_eset
    • resources_chester
    • resources_mis-tokens
    • resources_movistar-cloud
    • resources_estadio-infinito
    • resources_es
  • BR
    • resources_br-b2c
    • resources_br-b2b
    • resources_br
    • resources_br-rh
    • resources_br-rcs
    • resources_br-stb
    • resources_br-easy
    • resources_br-app
    • resources_br-wa
    • resources_br-nov

Migration process for CLF plugins

The Aura Global NLP team and constructors have carried out the following steps for the migration of CLF plugins:

  • Aura Global NLP team:
    • Updated in requirements.txt the packages to the versions generated for Prince release.
    • Updated tools and scripts to work with the Prince release.
    • These repositories work the same way as the NLP training repositories: a release/7.0 branch has been launched before the Prince release to make changes in previous versions. For Prince release, a release/8.0 branch is available.
    • Updated JenkinsFile:
      • The JenkinsFile is set for versions previous to Prince release (release/7.0).
      • In Prince release, there is no need to update JenkinsFile as it is already configured in master.
  • Constructors:
    • Reviewed the Python changelog for Python 3.13 version.
    • Executed tests with the new version of Python and its libraries.
    • Executed plugins and tested the locally.
    • Adapted code.
    • Generated a new version of the plugin package of the constructors and propagated to the corresponding environment.
    • Tested plugins in the environments.

Currently, there are no constructors outside the global team. For this reason, the Aura Global NLP team will be responsible for the migration of the CLF Plugins.

CLF plugins repository list

Below, the list of migrated CLF plugins repositories is included:

  • aura_clf_video

Migration process for ATRIA

The ATRIA repositories have been migrated to Python 3.13 by Aura Global Team.

The configuration inside ATRIA repositories must be the same as before migration, but constructors must test the use cases to ensure that everything is working correctly.

Information about the migration process for any productive repositories of Python language

Get sure that any productive repository that is not currently being used has been migrated from Python 3.9 to 3.13 version before being put into production.


  1. Python 3.13 is the latest version of Python Programming Language at this moment. Cognitive components are being migrated to this version. ↩︎

  2. JenkinsFile defines the pipeline in Jenkins for the continuous integration of the repositories. ↩︎

1.2 - Nodejs-related dependencies update

Nodejs-related dependencies update in Prince release

Modifications in Aura components associated to the update of Nodejs in release Prince 10.0.0 that must be carried out by the OBs when deploying this version

Introduction

The following dependencies of Aura Nodejs components have been updated in Prince 10.0.0 release.

This affects aura-bot and, consequently, implies that different tasks must be done by the OBs in order to make their dialogs compatible with these modifications.

Changes in the rest of components do not affect neither the constructors nor the deployment itself.

Update nodejs

In Prince release, Nodejs has been updated to Nodejs 22 version.

This upgrade does not need any additional task to be executed by the OB use cases developers.

Update Typescript

In Prince release, Typescript has been updated to Typescript 5.8.2.

Prior to deploying Prince, OBs should update these versions in the package.json:

  • “typescript”: “~5.8.2”

This upgrade could generate certain errors listed below. In these scenarios, OBs constructors can resolve these problems with the indicated solution:

  • Problems with "suppressImplicitAnyIndexErrors": true in tsconfig.json
    • Solution: Remove this property
  • Problems with some dialogs with super.initialDialogId
    • Solution: Replace it by this.initialDialogId
  • Problems with some dialogs with super.addDialog method
    • Solution: Replace it with this.addDialog
  • Problems with certain types
    • Solution: Define them correctly

Update Bot Framework

In Prince release, Bot Framework has been updated to Bot Framework 4.23.2 version.

Prior to deploying Prince, OBs should update these dependencies, if they are needed by the local dialogs:

  • “botbuilder”: “~4.23.2”,
  • “botbuilder-core”: “~4.23.2”,
  • “botbuilder-dialogs”: “~4.23.2”,
  • “botframework-connector”: “~4.23.2”,
  • “@azure/core-auth”: “~1.9.0”,
  • “@azure/data-tables”: “~13.3.0”,
  • “@azure/storage-blob”: “~12.26.0”

Update Superagent

In Prince release, Aura uses the latest Superagent version 10.2.0.

Prior to deploying Prince, OBs constructors should update these versions in the package.json:

  • “superagent”: “^10.2.0”
  • “@types/superagent”: “^8.1.9”

Package.json example

{
    "name": "@telefonica/aura-bot-generic-library",
    "version": "10.0.0",
    "repository": {
        "type": "git",
        "url": "git@github.com:Telefonica/aura-bot-libraries.git",
        "directory": "packages/generic"
    },
    "license": "UNLICENSED",
    "main": "lib/index.js",
    "scripts": {},
    "dependencies": {
        "@telefonica/aura-bot-utilities": "2.0.0",
        "@telefonica/aura-clients": "2.0.0",
        "@telefonica/aura-logging": "6.0.0",
        "@telefonica/aura-utilities": "2.0.0",
        "botbuilder": "~4.23.2",
        "botbuilder-dialogs": "~4.23.2",
        "superagent": "^10.2.0",
        "joi": "^17.6.0"
    },
    "devDependencies": {
        "@telefonica/aura-locale-importer": "9.30.0",
        "@telefonica/eslint-config-aura": "2.14.0",
        "@types/chai": "^4.2.3",
        "@types/jsonwebtoken": "^9.0.1",
        "@types/mocha": "^10.0.1",
        "@types/node": "^18.16.0",
        "@types/sinon": "^10.0.14",
        "@types/superagent": "^8.1.9",
        "@typescript-eslint/eslint-plugin": "^5.57.0",
        "@typescript-eslint/parser": "^5.57.0",
        "chai": "^4.3.10",
        "cross-env": "^5.2.0",
        "eslint": "^8.36.0",
        "eslint-plugin-import": "^2.27.5",
        "eslint-plugin-jsdoc": "^40.1.0",
        "mocha": "^10.2.0",
        "mocha-jenkins-reporter": "^0.4.8",
        "mocha-sonarqube-reporter": "^1.0.2",
        "nyc": "^15.1.0",
        "shx": "^0.3.3",
        "sinon": "^15.0.4",
        "ts-node": "^10.9.1",
        "tslint": "~5.18.0",
        "typescript": "~5.8.2"
    },
    "engines": {
        "node": ">=18.0.0"
    },
    "plugin": {
        "provides": [
            "generic"
        ]
    }
}

2 - Update CLU API version for releases previous to Nodoubt 9.11.0

Update CLU API version for releases previous to Nodoubt 9.11.0

In releases previous to Nodoubt 9.11.0, it is required to update the CLU API used for the clu-storage

Introduction

The current version of the CLU API used for the clu-storage (2023-04-15-preview) will be deprecated on March 31, 2025.

This version should be updated to 2024-11-15-preview for all environments before this date by the GES Team.

The update of the CLU API version enables the copy of CLU projects in cache. This leads to more efficient processes: reuse of CLU trainings with no need for retraining, reduced deployment times, hot swapping processes, etc.

On the other hand, this update does not impact in the use of Aura NLP.

Steps to update CLU API version

To update the version of the CLU API, follow the steps below:

  • Prerequisite: A kubeconfig of the environment must be previously configured.

  • Access to edit configuration:

      kubectl edit configmap nlp-provisioning-bootstrap -n <namespace>
    
  • In the section clu of the nlp-provisioning-bootstrap configmap, the value storage_api_version should be updated to 2024-11-15-preview.

  • Save and close the ConfigMap.

  • Restart nlp-provisioning deployment:

      kubectl rollout restart deployment/nlp-provisioning -n <namespace>
    

3 - Configuration for Nodoubt 9.11.0 release

Configuration for Nodoubt 9.11.0 release

Requisites for the proper operation of Nodoubt 9.11.0 release

aura-databricks-jobs configuration

Manual removal of files from Microsoft Storage account

A pre-requisite in Nodoubt release is required for the component aura-databricks-jobs:

The Operations Teams must eliminate manually the files configured in the common storage of the Microsoft Storage account defined in the aura-databricks-jobs environment variable AURA_MICROSOFT_AZURE_STORAGE_COMMON_ACCOUNT. These files are not needed anymore, but remain when a new release is installed.

The files need to be deleted from two locations:

  • /sizeReportDefault.json
  • avro/sizeReportDefault.json

4 - Workaround for Metallica 9.9.0 release

Workaround for Metallica 9.9.0 release

Description of the workaround required for Metallica release, that implies the restart of ATRIA components

Required ATRIA components restart

In Metallica release, for environments that can be suspended and set up again (DEV and PRE environments), it is required to restart two ATRIA components: atria-model-gateway and atria-rag-server.

For this purpose, follow these guidelines:

  • First, get sure that aura-configuration-api is up and running

  • Now, restart ATRIA components:

  • atria-model-gateway
    kubectl rollout restart deployment atria-model-gw -n <namespace>

  • atria-rag-server
    kubectl rollout restart deployment atria-rag -n <namespace>

In both cases, substitute <namespace> with the corresponding environment.

This requirement does not apply to PRO environments.

5 - Update Kiss 9.7.0 release base configuration

Update Kiss 9.7.0 release configuration

Guidelines to update Kiss base configuration:

  • Update aura-avro-adapter.json configuration for aura-kpis-uploader in the Azure common storage.

Prerequisites

  • A kubeconfig of the Aura environment must be configured.
  • Substitute <YOUR_ENV> with the corresponding environment: es-pre, es-pro, br-pre, de-pre, de-int, etc.
  • The installation output file (output_install/<YOUR_ENV>_info.json) to get:
    • The KPI blob container name:
      • Get <AURA_KPI_CONTAINER> value.
    • The KPI blob container name:
      • Substitute <AURA_KPI_CONTAINER> with the obtained value.
    • The connection of azure common account
      • Substitute <AURA_MICROSOFT_AZURE_STORAGE_COMMON_ACCOUNT> with the obtained value.
      • Substitute <AURA_MICROSOFT_AZURE_STORAGE_COMMON_ACCESS_KEY> with the obtained value.
PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE=output_install/<YOUR_ENV>_info.json
AURA_KPI_CONTAINER=$(cat ${PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE}|jq -r .kpi_blob_container_name)
AURA_MICROSOFT_AZURE_STORAGE_COMMON_ACCOUNT=$(cat ${PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE}|jq -r .common_azure_storage_account_name)
AURA_MICROSOFT_AZURE_STORAGE_COMMON_ACCESS_KEY=$(cat ${PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE}|jq -r .common_azure_storage_access_key)

Update aura-kpis-uploader configuration

Update aura-avro-adapter configuration file

  • Connect to Azure common account with the credentials of the Azure common account and KPI blob container.

  • Remove the file schemas/aura-avro-adapter.json.

  • When aura-kpis-uploader is run, a new json file will be generated, with the updated configuration.

6 - Update Imagine Dragons 9.5.0 base configuration

Update Imagine Dragons 9.5.0 base configuration

Guidelines to update Imagine Dragons base configuration:

  • Update scopes configuration for aura-bot and aura-billing Kernel clients, and use them in Databricks executions
  • Update aura-gateway-api environment variables to control how KPI entities are written

Prerequisites

  • In the corresponding Kernel deployment, all Aura datasets are published and the corresponding scopes are configured to aura-bot and aura-billing clients.
  • A kubeconfig of the Aura environment must be configured.
  • Substitute <YOUR_ENV> with the corresponding environment: es-pre, es-pro, br-pre, de-pre, de-int, etc.
  • The installation output file (output_install/<YOUR_ENV>_info.json) to get:
    • The KPI blob container name:
      • Substitute <AURA_KPI_CONTAINER> with the obtained value.
PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE=output_install/<YOUR_ENV>_info.json
AURA_KPI_CONTAINER=$(cat ${PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE}|jq -r .kpi_blob_container_name)

Update aura-databricks-jobs configuration

Update aura-bot scopes

Add these scopes in the Kernel app of aura-bot:

  • admin:datasets:read
  • data:read
  • data:write

Update aura-billing scopes

Add these scopes in the Kernel app of aura-bot.

  • admin:datasets:read
  • data:read
  • data:write

Update Databricks deployment

It will be necessary to deploy Aura core, indicated in core deployment. This will update the configuration with the following changes:

  • Update the profile to be multi-node instead of single node, configuring workers autoscaled.

  • Update the Databricks cluster node type, modifying databricks.cluster.node_type_id configuration:

    databricks:
      cluster:
        node_type_id: "Standard_DS4_v2"
    
  • Add autoscale config in cluster:

    databricks:
     cluster:  
       config:
          autoscale:
             enabled: true
             min_workers: 2
             max_workers: 4
    
  • Update cluster’s config, including a new config element to add spark config as dynamic allocation enabled or executor memory and cores:

    databricks:
      cluster:  
        config:
          "spark.debug.maxToStringFields": "100"
          "spark.driver.memory": "4g"
          "spark.executor.memory": "4g"
          "spark.dynamicAllocation.enabled": "true"
          "spark.executor.cores": "4"
    

Update aura-gateway-api environment variables to control KPI entities writing

Follow these guidelines to update aura-gateway-api environment variables to control the way KPIs entities are written.

This step can be executed at any time, because it is a performance improvement and will be the default configuration in the following release. But it should be applied as a workaround if the aura-gateway-api starts failing and returns responses with 503 or 502 statuses, meaning that the KPI writing processes are consuming all the server resources.

  • Connect to the environment using your kubeconfig.

  • Read the name of the container where the KPI entities files are being written.

  • Edit the config-map of aura-gateway-api to update the variables.

    kubectl -n <YOUR_ENV> edit cm aura-gateway-api -o 
    ...
    
  • Substitute or add the following environment variables.

If an environment variable is not in the config map, it means that the used value is the default one. Please refer to the aura-gateway-api environment variables documentation for further information.

AURA_KPIS_BLOB_STORE_INTERVAL: 3600000
AURA_KPIS_BLOB_STORE_MAX_BLOCK_BYTES: 2500000
AURA_KPIS_LOG_API_REQUEST_BODY: 'false'
AURA_KPIS_LOG_API_RESPONSE_BODY: 'false'
AURA_HTTP_PATHS_LOG_DISABLED: `v1/.well-known, aura-configuration, metrics, healthz, <AURA_KPI_CONTAINER>'
  • Restart aura-gateway-api pods:

    kubectl -n <YOUR_ENV> rollout restart aura-gateway-api
    

7 - Update Germany configuration for Imagine Dragons 9.5.0

Update Germany configuration for Imagine Dragons 9.5.0

Guidelines to update Germany default configuration for Imagine Dragons:

  • To apply the correct brand in all the channels and applications
  • To point to the correct Kernel deployment
  • To avoid issues of Databricks deployment compatibility with Confidential Computing

Prerequisites

  • In the corresponding Kernel deployment, all Aura datasets are published and the corresponding scopes are configured to aura-bot client.
  • There is an AURA_CONVERSATIONS container in Kernel Azure Storage.
  • A kubeconfig of the Aura environment must be configured.
  • Substitute <YOUR_ENV> with the corresponding pre-production environment: es-pre, es-cert, br-pre, de-pre, de-int, etc.
  • The installation output file (output_install/<YOUR_ENV>_info.json) to get:
    • The APIKey of Aura to access aura-configuration-api.
      • Substitute <AURA_API_KEY> with the obtained value.
PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE=output_install/<YOUR_ENV>_info.json
AURA_API_KEY=$(cat ${PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE}|jq -r .aura_bot_services_api_key)

Include new brand as a valid one

  • Connect to the environment using your kubeconfig.

  • Edit the config-map of aura-configuration-api to include the new brand.

    • To include WhatsAppSim in Germany as new brand, just substitute <NEW_BRAND_ID> with 0111.

    Access the full list of Brands defined in Kernel

kubectl -n <YOUR_ENV> edit cm aura-configuration-api -o 
apiVersion: v1
data:
  AURA_BRANDS: 0101,10000,<NEW_BRAND_ID>
...
  • Restart aura-configuration-api pods:
kubectl -n <YOUR_ENV> rollout restart aura-configuration-api

Change already existing channels and applications brands to point to the new one

Change channels

  • Substitute <UUID> with a valid UUID version 4 generated for every request.
  • Substitute <AURA_API_KEY> with the Aura API key read from the environment installation file.
  • Substitute <CHANNEL_ID> with the channel to be configured.
  • Substitute <NEW_BRAND_ID> with the brand to be configured.
curl --location --request PATCH 'https://<YOUR_ENV>.auracongnitve.com/aura-services/v2/configuration/channels/<CHANNEL_ID>'
\
--header 'correlator: <UUID>' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: APIKEY <AURA_API_KEY>' \
--data '{
    "id": "<CHANNEL_ID>",    
    "brand": "<NEW_BRAND_ID>"
}'
  • Changes in Germany, the previous PATCH should be executed twice, at least, once per existing channel:
    • Sprinklr web chat channel identifier: 33a4dcdc-0ef7-4bf1-886f-48d4fda2a031
    • Testing channel identifier: afec79db-dc0d-4c2e-a098-6c996503a7b4

Change applications

  • Substitute <UUID> with a valid UUID version 4 generated for every request.
  • Substitute <AURA_API_KEY> with the Aura APIKey read from the environment installation file.
  • Substitute <APPLICATION_ID> with the application to be configured.
  • Substitute <NEW_BRAND_ID> with the brand to be configured.
curl --location --request PATCH 'https://<YOUR_ENV>.auracongnitve.com/aura-services/v2/configuration/applications/<APPLICATION_ID>'
\
--header 'correlator: <UUID>' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: APIKEY <AURA_API_KEY>' \
--data '{
    "id": "<APPLICATION_ID>",    
    "brand": "<NEW_BRAND_ID>"
}'
  • Changes in Germany, the previous PATCH should be executed once, at least, to update ATRIA application:
    • ATRIA application: 7586c369-b6a0-4fa5-9416-d8b45920274c

Change Kernel configuration in Aura components

Review the following variables in your environment profile:

fourth_platform:
  client_id: "aura-bot" # Aura 4P app client id
  client_secret: !vault |
          $ANSIBLE_VAULT;1.1;AES256
          <THE_SECRET_FOR_AURA_BOT_IN_KERNEL>
  apigw_url: "https://api.<KERNEL_DEPLOYMENT>.baikalplatform.com" # must be filled with the right value. Usually https://api.{{environment_profile}}-{{environment_type}}.baikalplatform.com
  • For example, in Germany, for WhatsappSim Kernel, the URL should be:
    • de-int: https://api.de-int-whatsappsim.baikalplatform.com
    • de-pre: https://api.de-pre-whatsappsim.baikalplatform.com
    • de-pro: https://api.de-pro-whatsappsim.baikalplatform.com
  • For aura-kpis-uploader, assure that AURA_MICROSOFT_AZURE_STORAGE_CONTAINER_DESTINATION and AURA_MICROSOFT_AZURE_STORAGE_ACCESS_KEY_DESTINATION belong to the corresponding Kernel deployment.
  • Execute deploy_core to assure that the configuration is applied everywhere.

Configure Databricks deployment to be compatible with Confidential Computing

Follow these guidelines:

  • Delete existing workspace

    • If you have already deployed Databricks using deploy_common phase, the first step is to delete the current workspace before redeploying it with the correct configuration for Confidential Computing.
    • Go to the Azure dashboard and navigate to the resource group where your Databricks workspace is located (usually the common resource group used in the initial deployment).
    • Select the Databricks workspace resource and delete it.

  • Set up the environment for Confidential Computing

    • Once the workspace is deleted, you need to adjust the configuration variables to make the new deployment compatible with Confidential Computing. Add the following variables to the environment configuration:
    databricks_enhanced_security_compliance: false
    databricks:
      cluster:
        node_type_id: "Standard_DC8as_v5"
    
  • Explanation of the variables

    • databricks_enhanced_security_compliance: Set to false to avoid additional security compliance configurations that are not needed in this context, and to remove the ability to use confidential type machines.
    • databricks.cluster.node_type_id: This is configured, for example, as “Standard_DC8as_v5”, which is a Confidential Computing node type in Azure.

  • Deploying the workspace with the new configuration

    • To deploy Databricks, we need to re-run the deploy_common phase with the new configuration.
    • Once the deployment is complete, the Databricks workspace will be configured to use Confidential Computing nodes.
    • To configure the Databricks job we must run again the deploy_core.