aura-kpis utility

Introduction

aura-kpis utility is a library used by all Aura apps to generate KPIs entity files equally in all of them.

It is placed in the Github repository:
aura-bot-libraries/packages/aura-kpis/.

It has two working modes:

• file: Entity files will be stored locally on the current server instance. For development purposes.
• blob: Entity files will be stored remotely on an Azure Blob Container with aura-storage-file-manager uploadLogs.

Configuration

These configuration variables MUST be set in the server using the library.

• KPIs environment variables:
  • AURA_VERSION: mandatory, Aura release.
  • AURA_DEFAULT_LOCALE: mandatory, string with the default locale to be used in the server. It uses two letter codes both for country and culture, for example: es-es, en-gb.
  • AURA_DEFAULT_TIME_ZONE: mandatory, timezone where the service is running.
  • AURA_KPI_TO_DSV_CACHE_TTL: optional, number of milliseconds to cache existing requests to calculate their duration. By default, 1800.
  • AURA_KPI_TO_DSV_DELIMITER: optional, string with the delimiter to be used in KPIs entities files. By default, |.
  • AURA_KPI_TO_DSV_EXTENSION: optional, string with the extension to be used in KPIs entities files. By default, txt.
  • AURA_KPI_FILE_PREFIX: mandatory, string with the prefix used in the KPIs entities files of this service. Usually its format is component/prefix. For CSV files it is used as is, so files are stored in Azure Storage in that path, meaning a virtual folder named component and that the files will start by prefix. But for Avro files, the value of the variable is split into 2 by / and only the prefix part is used to name the files, that are store together with files generated by all components in the same virtual folder.
  • AURA_KPI_STORE_MODE: optional, string to indicate what is the destination of the KPIs entities files. By default, blob.
    • If file, they will be stored locally on the instance, in the folder shown in KPI_TO_DSV_LOCAL_FILES_DIRECTORY. For development purposes.
    • If blob, they will be stored remotely on the Azure blob container shown in KPIS_STORE_CONTAINER. Mandatory in environments running on k8s.
  • AURA_KPI_TO_DSV_LOCAL_FILES_DIRECTORY: optional, string with the local directory to store KPIs entities files. By default, ./kpis-dsv. It MUST be the same than the one configured in KPIS_UPLOADER module. Only needed if AURA_KPI_STORE_MODE==file.
  • AURA_MICROSOFT_AZURE_STORAGE_ACCESS_KEY: optional, string with the Azure Storage Access Key to be able to write files in KPIS_STORE_CONTAINER. Only needed if AURA_KPI_STORE_MODE==blob.
  • AURA_MICROSOFT_AZURE_STORAGE_ACCOUNT: optional, string with the Azure Storage Account name to be able to write files in KPIS_STORE_CONTAINER. Only needed if AURA_KPI_STORE_MODE==blob.
  • AURA_KPIS_STORE_CONTAINER: optional, string with the name of the Azure Blob container to store KPIs entities files. By default, aura-kpis. It MUST be the same than the one configured in KPIS_UPLOADER module. Only needed if AURA_KPI_STORE_MODE==blob.
  • AURA_KPIS_BLOB_STORE_INTERVAL: optional, number with the time interval in milliseconds to upload asynchronously logs to the KPIS_STORE_CONTAINER. By default, 60000. Only needed if AURA_KPI_STORE_MODE==blob.
  • AURA_SOURCE_PATH_AVRO_ADAPTERS: optional, string containing the adapters to transform data, '/schemas/aura-csv-adapter.json' for CSV transform and '/schemas/aura-avro-adapter.json' to transform in CSV and AVRO. By default: '/schemas/aura-csv-adapter.json'.

Use aura-kpis utility

aura-kpis utility needs to add a dedicated events emitter that handles KPI events.

• Initialize the specific KPI writer, based on your service configuration:

// Start Kpis Writer
const kpisWriterFactory: KPIWriterFactory = KPIWriterFactory.getInstance();
const kpisWriter: KPIEntityWriter = await kpisWriterFactory.createKpiWriter();

• Use the classes and utilities provided by the library:

import { KPIEventBody, KPIEmittersEvents, KPIActionEmitter } from '@telefonica/aura-kpis';
const kpisLogger = new KPIActionEmitter('MyModule');

• Fill the corresponding event with the data available in every moment. Update this data on each step, before emitting the event.

let kpisEvent: KPIEventBody = {
    request: {
        correlator: correlator,
        requestId: requestId
    },
    user: {
        auraId: auraId,
        channelId: channelId
    },
    channel: {
        channelId: channelId
    }
};

• If the name of the environment variables of your service do not match with the library names, the configuration can be also be passed as an object of type KPIWriterConfiguration.

• Depending on the value of AURA_KPI_STORE_MODE a different set of properties is validated.

• In those meaningful steps, emit the corresponding event. The basic approach is to emit one event (KPIEmittersEvents.REQUEST_STARTED) when the request lands on the server and another when the response is returned (for instance, KPIEmittersEvents.KPIS_USER_REQUEST_FINISHED, in the case of a user request):

kpisLogger.emit(KPIEmittersEvents.KPIS_USER_REQUEST_FINISHED, kpisEvent);

Events

• API entity filling: this case just needs a couple of events to handle incoming and outgoing steps.

  • KPIS_API_REQUEST_STARTED: event that shows that a new request to one external API is started. It adds to a cache a time mark, so the duration of the request can be calculated.
  • KPIS_API_REQUEST_FINISHED: last event of a new request to one external API has finished, emitted when the response is returned.

• MESSAGE entity filling: this case just needs a couple of events to handle incoming and outgoing steps.

  • KPIS_MESSAGE_REQUEST_STARTED: event that shows that a new message goes in aura-bot. It adds a time mark to a cache, so the duration of the request can be calculated.
  • KPIS_MESSAGE_REQUEST_FINISHED: event to write the KPI of the given message being handled by aura-bot. It is emitted when a new message comes in and when its response is returned.

• EXTENDED_MESSAGE entity filling: this case just needs a couple of events to handle incoming and outgoing steps.

  • KPIS_EXTENDED_MESSAGE_REQUEST_STARTED: event that shows that a new message enters aura-bot. It adds a time mark to a cache, so the duration of the request can be calculated. It is an extended logic made from MESSAGE entity.
  • KPIS_EXTENDED_MESSAGE_REQUEST_FINISHED: event to write the KPI of the given message being handled by aura-bot. It is emitted when a new message comes in and when its response is returned. It is an extended logic made from MESSAGE entity.

• NOTIFICATION entity filling: this case just needs a couple of events to handle incoming and outgoing steps.

  • KPIS_NOTIFICATION_REQUEST_STARTED: event that shows that a new notification request has arrived. It adds a time mark to a cache, so the duration of the request can be calculated.
  • KPIS_NOTIFICATION_REQUEST_FINISHED: last event of a notification request, emitted when the response is returned in the corresponding controller.

• RECOGNIZER entity filling: this case just needs a couple of events to handle incoming and outgoing steps.

  • KPIS_RECOGNIZER_REQUEST_STARTED: event that shows that a new recognizer request is started. It adds a time mark to a cache, so the duration of the request can be calculated.
  • KPIS_RECOGNIZER_REQUEST_FINISHED: last event of a recognizer request, emitted when the response is returned.

• SUGGESTION entity filling: this case just needs a couple of events to handle incoming and outgoing steps.

  • KPIS_SUGGESTION_REQUEST_STARTED: event that shows that a suggestion is shown to the user or clicked by her. It adds a time mark to a cache, so the duration of the request can be calculated.
  • KPIS_SUGGESTION_REQUEST_FINISHED: last event of a suggestion shown to the user or clicked by her, emitted when the response is returned.

• USER entity filling: This case needs partial events to assure that no value of the USER entity is missed, both in successful and error cases.

  • KPIS_REQUEST_STARTED: event that shows that a new request has arrived. It adds to a cache a time mark, so the duration of the request can be calculated.
  • KPIS_USER_REQUEST_FINISHED: last event of a user request, emitted when the response is returned in the corresponding controller.
  • KPIS_USER_CREATE_DB: partial user event, emitted when requesting the DB.
  • KPIS_USER_DELETE_DB: partial user event, emitted when the user is deleted from the DB.
  • KPIS_USER_FIND_DB: partial user event, emitted when the user is found in the DB.
  • KPIS_USER_CREATE_SERVICE: partial user event, emitted when returning from the user creation service.
  • KPIS_USER_LOGIN_SERVICE: partial user event, emitted when returning from the user login service.
  • KPIS_USER_DELETE_SERVICE: partial user event, emitted when returning from the user deletion service.

• GATEWAY entity filling: this case just needs a couple of events to handle incoming and outgoing steps.

  • KPIS_GATEWAY_MESSAGE_REQUEST_STARTED event that shows that a new request enters aura-gateway-api. It adds a time mark to a cache, so the duration of the request can be calculated.
  • KPIS_GATEWAY_MESSAGE_REQUEST_FINISHED: event to write the KPI of the given request being handled by aura-gateway-api. It is emitted when its response is returned.

• GROOT entity filling: this case just needs a couple of events to handle incoming and outgoing steps.

  • KPIS_GROOT_MESSAGE_REQUEST_STARTED: event that shows that a new message enters aura-groot. It adds a time mark to a cache, so the duration of the request can be calculated.
  • KPIS_GROOT_MESSAGE_REQUEST_FINISHED: event to write the KPI of the given message being handled by aura-groot. It is emitted when a new message comes in and when its response is returned.