Update Imagine Dragons 9.5.0 base configuration

Prerequisites

• In the corresponding Kernel deployment, all Aura datasets are published and the corresponding scopes are configured to aura-bot and aura-billing clients.
• A kubeconfig of the Aura environment must be configured.
• Substitute <YOUR_ENV> with the corresponding environment: es-pre, es-pro, br-pre, de-pre, de-int, etc.
• The installation output file (output_install/<YOUR_ENV>_info.json) to get:
  • The KPI blob container name:
    • Substitute <AURA_KPI_CONTAINER> with the obtained value.

PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE=output_install/<YOUR_ENV>_info.json
AURA_KPI_CONTAINER=$(cat ${PATH_TO_YOUR_OUTPUT_INSTALL_ENV_FILE}|jq -r .kpi_blob_container_name)

Update aura-databricks-jobs configuration

Update aura-bot scopes

Add these scopes in the Kernel app of aura-bot:

• admin:datasets:read
• data:read
• data:write

Update aura-billing scopes

Add these scopes in the Kernel app of aura-bot.

• admin:datasets:read
• data:read
• data:write

Update Databricks deployment

It will be necessary to deploy Aura core, indicated in core deployment. This will update the configuration with the following changes:

• Update the profile to be multi-node instead of single node, configuring workers autoscaled.

• Update the Databricks cluster node type, modifying databricks.cluster.node_type_id configuration:

  databricks:
    cluster:
      node_type_id: "Standard_DS4_v2"
  

• Add autoscale config in cluster:

  databricks:
   cluster:  
     config:
        autoscale:
           enabled: true
           min_workers: 2
           max_workers: 4
  

• Update cluster’s config, including a new config element to add spark config as dynamic allocation enabled or executor memory and cores:

  databricks:
    cluster:  
      config:
        "spark.debug.maxToStringFields": "100"
        "spark.driver.memory": "4g"
        "spark.executor.memory": "4g"
        "spark.dynamicAllocation.enabled": "true"
        "spark.executor.cores": "4"
  

Update aura-gateway-api environment variables to control KPI entities writing

Follow these guidelines to update aura-gateway-api environment variables to control the way KPIs entities are written.

This step can be executed at any time, because it is a performance improvement and will be the default configuration in the following release. But it should be applied as a workaround if the aura-gateway-api starts failing and returns responses with 503 or 502 statuses, meaning that the KPI writing processes are consuming all the server resources.

• Connect to the environment using your kubeconfig.

• Read the name of the container where the KPI entities files are being written.

• Edit the config-map of aura-gateway-api to update the variables.

  kubectl -n <YOUR_ENV> edit cm aura-gateway-api -o 
  ...
  

• Substitute or add the following environment variables.

If an environment variable is not in the config map, it means that the used value is the default one. Please refer to the aura-gateway-api environment variables documentation for further information.

AURA_KPIS_BLOB_STORE_INTERVAL: 3600000
AURA_KPIS_BLOB_STORE_MAX_BLOCK_BYTES: 2500000
AURA_KPIS_LOG_API_REQUEST_BODY: 'false'
AURA_KPIS_LOG_API_RESPONSE_BODY: 'false'
AURA_HTTP_PATHS_LOG_DISABLED: `v1/.well-known, aura-configuration, metrics, healthz, <AURA_KPI_CONTAINER>'

• Restart aura-gateway-api pods:

  kubectl -n <YOUR_ENV> rollout restart aura-gateway-api