Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura entities definition in Avro format

Aura entities definition version 6.0.0 (entities using Avro format), URM-compliant.

Introduction

The Aura entities definition includes the different entities currently written in Aura in Avro format, URM-compliant.

For further information regarding Kernel Avro datasets access the following links:

Entities generated in Avro URM-compliant format

Entities types	Description	Generation
Aura_Atria_Message	Summary of Aura’s interactions handled by aura-rag-server, aura-groot and aura-bot	Generated by aura-gateway-api
Aura_Audit	Entity used to store information about the interactions handled by Aura components. It will be used to generate Aura’s invoices.	Created by aura-kpis-uploader based on the information received in Groot Message and Gateway Message
Aura_Gateway_Message	Entity used to store information about the messages handled by aura-gateway-api.	Created by aura-gateway-api.
Aura_Message	Entity that contains the summary of Aura’s interactions handled by aura-bridge, aura-groot and aura-bot	Created directly by aura-groot and aura-bot
D_Aura_Dimensionals	They are Aura’s configuration settings that are needed to understand Aura’s entities	Generated directly by aura-kpis-uploader from the content of the aura-configuration-api.

Entities generation

Avro Entities tables nomenclature

Position: Field order in the Avro schema definition.
Field: Name of the field being defined. Usually, capitalized, although more aliases will be available in the schema definition.
PK: Parameter that indicates if the field is a Primary Key or not, that is, a value that uniquely identifies this field.
Referenced Entity: When the field references a value from another dataset, indicating that a join must be defined in the schema, this parameter contains the name of the dataset.
Referenced Entity Field: When the field references a value from another dataset, indicating that a join must be defined in the schema, this parameter contains the name of the field of the other dataset.
Type: Type of the field. It can be one of Avro logical types defined in Kernel.
Doc: Brief description of the field.
Nullable doc: If the field can be null, it should contain the cases when a null value is valid.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura entities definition in CSV format

Aura entities definition in CSV format version 5.0.0.

Introduction

The Aura entities definition includes the different entities currently written in Aura in CSV and required to calculate Aura KPIs or by any other team in their data processes.

Entities types	Description	Generation
Message	Message entity is used to store information about the messages handled by aura-bot and the actions performed on them.	They are created in aura-bot.
Recognizer	Recognizer entity is used to store the request to any of the recognizers during the utterance recognition phase of the messages.	They are created by aura-groot, aura-bot and aura-nlp.
Extended Message	Extended Message entity is used to store extra information of a Message.	They are created in aura-bot.
Groot Message	Groot Message entity is used to store information about the messages handled by aura-groot.	They are created in aura-groot.

Entities generation

Until release 9.3.0 (Gwen Stefani), all entities in Aura were generated in CSV format. They were also uploaded into Kernel storage in CSV format. Although using a script provided by and running in Kernel, some of them are converted to Avro entities (in a deprecated Avro format, not URM compliant).

In particular, the following conversions are provided:

Recognizer:
- Aura Authenticated Users’ Recognizer
- Aura Anonymous Users’ Recognizer

As can be seen, the entities in Avro are duplicated to hold authenticated and anonymous users interactions separately, because the field USER_4P_ID cannot be null if it exists in a dataset definition.

CSV files format

All entities’ files must be created following these rules:

File format: UNIX. UTF8 without BOM
Date format: ISO8601
- Date: 2018-05-02
- Datetime: 2018-05-02T15:18:11Z => Always UTC

Usually, these kinds of files are stored in .txt and zipped, before uploading.

Entities used for the calculation of KPIs are stored in a Kernel bucket. Each entity must be included in a different folder, with files distributed by months, in the path:
[OB]/[ENTITY]/YYYYMM
Files generated in aura-bot can be stored as needed.
Filename: BOT_[HOST_ID]_[OB]_[ENTITY]_YYYYMMDDTHH0000Z.txt
Files generated in aura-services (authentication) can be stored as needed.
Filename: SERVICES_[HOST_ID]_[OB]_[ENTITY]_YYYYMMDDTHH0000Z.txt
Files generated in Aura NLP components can be stored as needed:
Filename: NLP_<HOST_ID>_<OB>_<ENTITY>_YYYYMMDDTHH0000Z.txt
Dimensions entities:
Path: [OB]/DIMENSIONS/YYYYMM
Filename: [OB]_DIM_[DIM_NAME]_YYYYMMDDTHH0000Z.txt
For example: ES_DIM_CHANNEL_20180612T160000Z.txt

All the files are refreshed every day.

CSV Entities tables nomenclature

#: Field ID
FIELD: Specific field of the entity type
PK: Parameter that indicates if the field is a Primary Key or not, that is, a value that uniquely identifies this field.
NULLABLE: Parameter that indicates if a field is allowed to have a null value or not.
TYPE: Type of the field. It can be one of: text, date, number, boolean
DESCRIPTION: Brief description of the field
FORMAT: Field mandatory format, if applicable
ALLOWED VALUES: Prefixed values permitted for this field
EXAMPLE: Example of application

The following considerations must be taken into account:

Numeric values are rounded to two decimal positions
The amount of money must be included in local currency

Aura –

Docs:

Aura entities definition in Avro format

Introduction

Entities generated in Avro URM-compliant format

Entities generation

Avro Entities tables nomenclature

Docs:

Aura entities definition in CSV format

Introduction

Entities generation

CSV files format

CSV Entities tables nomenclature