Aura – data-model

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

auraCommand model

Description of the auraCommand model suitable for sending intents directly to Aura with no recognition stage

Introduction

An auraCommand is an structure to send intents directly to Aura. It includes specific messages consisting on pre-defined options that the user can select from Aura’s interface as part of the response to her request. These messages come from channels using Direct Line format. Currently, they correspond to notifications and suggestions.

The aura-bot recognizer in charge of auraCommand is aura-command-recognizer-middleware. This recognizer detects if the message matches auraCommand format and then processes the message and extract the intent and entities from it.

An auraCommand should be sent within the channelData of the request to aura-bot in an optional property called auraCommand.

AuraCommand property in channelData

It is the channelData property called AuraCommand, which value should fit an AuraCommand Model.

Property	Type	Description
auraCommand	AuraCommand	This property contains an `auraCommand` object.

AuraCommand Model

Property	Type	Description
type	string	`AuraCommand` type: suggestion: used to send a common intent. The prefix of the intent must be `intent.`. Its the type by default. operation: It represents an intent that needs a context in Aura. The prefix of the intent must be `operation`. notification: It represents an intent generated by notification. The prefix must be `intent`. text: The value of the `auraCommand` contains directly a `string` with the text to be recognized by NLP. event: The value of the AuraCommand contains and event that aura-bot recognizes to carry out an action.
value	Intent model or string	Object intent or text.
utm	Utm model	Source that contains the AuraCommand.
actions	Action model	Array of actions.

Intent model

Property	Type	Description
intent	string	Name of the intent. If this name does not contain the prefix `intent.`, Aura will add the it.
entities	Entity model	Array with the entities. This array can be empty.

Entity model

Property	Type	Description
type	string	Name of the intent. If this name does not contain the prefix `intent.`, Aura will add it.
entity	string	Name of the entity. If this name does not contain the prefix `ent.` Aura will add it.
canon	string
score	float	Array with the entities. This array can be empty.

Utm model

Property	Type	Description
source	string	It identifies which site sent the traffic. In Aura, use the channel name.
medium	string	It identifies what type of link was used, such as cost per click or email. In Aura, use banner, button, etc.
campaign	string	It identifies a specific product, promotion or strategic campaign.
term	string	It identifies search terms.
content	string	It identifies what was specifically clicked to bring the user to the site, such as a banner ad or a text link.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

dialogContext model

Description of the dialogContext model, for the management of different existing options in the response

Introduction

A dialogContext is a structure that facilitates the recognition of CardActions and prompts without the need of training.

It has the following characteristics:

It can be autogenerated when a prompt is active.
It can be autogenerated when there is a CardAction.
It can be created manually in a dialog but with certain restrictions.
It is stored in the conversation cache.
It is processed by the dialog-context-recognizer-middleware.
It is auto-generated and stored by the aura-bot adapter.
It can be configured at channel level, prompt level and CardAction level.
It facilitates the recognition of CardActions if a user, instead of pressing the button, types the button text.
It facilitates the recognition of numbered lists either by ordinals or cardinals.
It uses a proximity search algorithm.
It solves the problems of the recognition of numbers and ordinals that the current Bot Framework prompt has.

Existing models

dialogContext model

Property	Type	Description
text	String	Text or texts (separated by ‘\|’) with the options to be matched.
value	Object	Object of `IntentResult` or other with the action to carry out if the option is validated.

Example:

"dialogContext": [
                    {
                        "text": "Hello",
                        "value": {
                            "intent": "intent.factotum-test.dialog-context",
                            "entities": [
                                {
                                    "entity": "greetings",
                                    "type": "ent.dialog-context",
                                    "score": 1
                                }
                            ]
                        }
                    },
                    {
                        "text": "Bye bye mi picolissima dama",
                        "value": {
                            "intent": "intent.factotum-test.dialog-context",
                            "entities": [
                                {
                                    "entity": "goodbyes",
                                    "type": "ent.dialog-context",
                                    "score": 1
                                }
                            ]
                        }
                    }
                ]

Channel model

The dialogContext model at the channel level applies to all dialog components.

This model has the highest priority, i.e., in case of conflict between models, the channel model has the highest weight.

This model is inside the responseOptions.dialogContext property. Check it in Aura channel model: ResponseOptions model.

An example is shown below:


channel: {
...
        responseOptions: {
            dialogContext: {
                normalizeTerms: true,
                disabled: false,
                disabledMerge: false,
                processFromClient: false,
                returnToClient: false,
                defaultListType: 'none',
                promptChoice: {
                    generate: 'never',
                    generateList: false
                },
                cardActions: {
                    generate: 'never',
                    generateList: false
                }
            }
        }
    },
...

Prompt model

The options of dialogContext for a prompt can be included inside the property validations of Prompt Options within its sub-property dialogContext.

When generating the list of options, the system takes into account two properties of the prompt: recognizeNumbers and recognizeOrdinals.

In these cases:

If the prompt has recognizeNumbers=false and we have activated the list of options, only the ordinal values of each option will be generated.
If the option recognizeOrdinals=false, then only the cardinal options will be generated.
If both are set to false, no list items will be generated.

By default, the prompt has both properties set to true.

Property	Type	Description
originalFindChoicesOptions	Object	Original prompt findChoiceOptions. Used if the dialogContext recognizer needs to change it, the prompt would be restored to the original state.
originalStored	Boolean	It indicates whether the dialogContext recognizer had to overwrite the original prompt values. Default: `false`.
enabled	Boolean	It indicates whether the generation of ordinal and cardinal lists for prompt options is active. Default: `Inherited from the channel configuration`.
listType	DefaultListType	Type of list to enumerate the prompt options. Default: `OrdinalCardinal`.

Example of prompt options:


{
  "Choices": [
    {
      "value": "Lunes",
      "synonyms": [
        "El lunes"
      ]
    },
    {
      "value": "martes"
    }
  ],
  "validations": {
    "dialogContext": {
      "enabled": true,
      "listType": "ordinal "
    }
  }
}

CardAction model

The options of dialogContext for a CardAction can be included inside the property channelData of button options, within the sub-property dialogContext.

Property	Type	Description
enabled	Boolean	It indicates whether the generation of ordinal and cardinal lists for CardAction options is active. Default: `Inherited from the channel configuration`.
listType	DefaultListType	Type of the list to enumerate the CardAction buttons. Default: `OrdinalCardinal`.
synonyms	String[]	Array of strings with synonyms of the title.

{
  "buttons": [
    {
      "type": "postBack",
      "title": "Monday",
      "value": {
        "intent": "monday"
      },
      "channelData": {
        "dialogContext": {
          "enabled": true,
          "listType": "ordinalCardinal"
        }
      }
    },
    {
      "type": "postBack",
      "title": "Wednesday",
      "value": {
        "intent": "wednesday"
      },
      "channelData": {
        "dialogContext": {
          "enabled": true,
          "listType": "ordinalCardinal",
          "synonyms": [
            "X"
          ]
        }
      }
    }
  ]
}

Custom dialogContext

It is possible to add in the channelData a dialogContext manually. If the channel configuration allows it, this will be merged with the _dialogContext that is generated automatically.

The following rules must be taken into account when including our own dialogContext:

Do not generate the list values, that is, do not add ordinals and/or cardinals, as this will be done by the system.
The options of a custom dialogContext are included right at the top of the list.

How does dialogContext work?

The dialogContext flowchart includes two differentiate phases: recognition phase and build phase, described below.

Recognition phase

When validating if the user has entered a phrase that matches the dialogContext that we have generated from the previous answer, take into account that the comparison is done by proximity.

The match value will be reported in a score field that goes from 0 to 1. Valid values will be those that are greater or equal to a given threshold. For a dialogContext we have two thresholds.

AURA_DIALOG_CONTEXT_THRESHOLD: Default threshold when comparing two strings. Default value: 0.80.
AURA_DIALOG_CONTEXT_ORDINAL_CARDINAL_THRESHOLD: Threshold used in case dialogContext elements are in list form and also if the input phrase includes a number. By default, 0.95.
This value is so high because if the user writes ‘11’ or ’the 11’, and there are only 5 elements in the list, the comparison could give us a very high value, since comparing 11 with 1 has a threshold higher than 0.80, which means that writing 11 would select the first option. To avoid this kind of cases when there are numbers in the user input and the list is enumerated, then the threshold should be stricter.

Another thing to take into account is that if the dialogContext does not get a valid match, if we have an active prompt choice, the system will override its recognizeNumbers and recognizeOrdinals properties, as long as we are working with enumerated elements.

This is done because when MS Bot Framework detects an ordinal number in a sentence, it ignores the rest of the sentence, which can lead to many errors. These values are restored every time the prompt is regenerated if it has not been resolved.

In the recognition phase, the dialogContext recognizer compares value to value, and in the case that one of them meets the conditions, the intention associated with the value is established. It also assigns to the input text the value expected by the prompt, in case there is one.

Example:

DialogContext

[
  {
    "text": "1|the 1|one|the one|first|the first|Monday",
    "value": {
      "intent": "monday"
    }
  },
  {
    "text": "2|the 2|two|the two|second|the second|Tuesday",
    "value": {
      "intent": "tuesday"
    }
  },
  {
    "text": "3|the 3|three|the three|third|the third|X|Wednesday",
    "value": {
      "intent": "wednesday"
    }
  },
  {
    "text": "4|the 4|four|the four|fourth|the fourth|Thursday",
    "value": "The Thursday"
  },
  {
    "text": "5|the 5|five|the five|fifth|the fifth|Friday",
    "value": "Friday"
  }
]

User Input: the 4

In this case, as the list is numbered and the user has entered a number in his sentence, the selected threshold will be AURA_DIALOG_CONTEXT_ORDINAL_CARDINAL_THRESHOLD.
The elements of each option will be compared with the user’s value:

   1 ≅ the 4  -> Score 0 
   the 1 ≅ the 4  -> Score 0.80
   ......
   the 4 ≅ the 4  -> Score 1.0   -> Option Selected

If the selected option has value, the value is assigned to the activity text. In other case, the assigned value is the last value of the text, in this case ‘Thursday’.
If there is a prompt choice, the validations recognizeNumbers and recognizeOrdinals are set to false.

Build phase

The build phase is responsible for collecting the customized dialogContext (if any) and, in turn, for generating the dialogContext of the CardActions and Prompt in the different activities that make up the output message.

The enumeration of the elements depends on the configuration of the channel and the respective components. Once generated, it is stored in the user’s conversation cache.

The module in charge of this procedure is the Aura Bot Adapter.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Analytics data model

Data model of Aura Analytics 1.1. that can be used as the base for building new elements

Introduction

New elements can be built (or the current elements modified) by making use of the available fields in Kibana through the ingested Elastic Search index.

In this document, we provide a reference of the schema that the index follows, so that it can be used to build such new visualizations, or to better understand the existing ones.

Elements in the Aura-message data model have 3 different types:

Numeric: single numbers, integer or real. Suitable for numerical statistics, such as averages, or for plotting variation across time in graphs.
Keyword: they are opaque strings, i.e., terms that cannot be searched within (it is not possible to look for words inside a keyword field). They can, however, be used to create some term-level queries, such as prefix queries (find all instances that begin with) and they usually work great for aggregations, since most of them are categorical variables (fields that only have a limited number of possible values) and can therefore be bucketed and counted.
Text: these fields are divided into separate terms (words), and some pre-processing is done to them before indexing to improve access though an Elastic Search analyzer. Text fields cannot be used in aggregated visualizations, since they cannot be grouped. They are most useful for queries, because they allow searching for fragments (only a few words) and fuzzy searches.

Fields list

The following table lists all the fields available in the Aura-message-COUNTRY Elastic Search index, together with their type and a brief description.

The most relevant ones include a more detailed description in the section fields explanations.

Note that some fields of text type have a mirror field of type keyword, with the same content. Having the same data indexed in two different ways at the same time (as text and as keyword) enables to perform different types of analysis by choosing the right field.

The “Raw” column indicates if this field is already present in the Aura raw PPD files:

Yes: field contained in raw PPDs.
No: generated field, produced when creating clean PPDs. They can be recognized as lowercase fields.
Partial: It exists in the raw PPDs, but in a somehow different shape.

Field	Type	Raw	Contents
CORR_ID	keyword	yes	Unique identifier for each interaction
VERSION_ID	keyword	yes	Aura Platform version
CHANNEL_CD	keyword	yes	Identifier for the channel this interaction corresponds to
STATUS_CD	keyword	yes	Internal code related to operation status
AURA_ID_GLOBAL	keyword	yes	(Mostly) unique identifier for the user
AURA_ID	keyword	yes	(Mostly) local identifier for the user
INTENT	keyword	yes	Detected user intent, including “system” intents
MESSAGE_USR	text	partial	Text request sent by the user
MESSAGE_USR_NORM	text	no	A normalized version of MESSAGE_USR
MESSAGE_USR_NORM.keyword	keyword	no	A keyword version of MESSAGE_USR_NORM, to enable aggregating on it
MESSAGE_AURA	text	partial	Text message sent by AURA to the user
MESSAGE_AURA.keyword		partial	Keyword version of MESSAGE_AURA, to enable aggregating on it
MODALITY_CD_USR	text	partial	Modality of the user message
MODALITY_CD_AURA	text	partial	Modality of Aura response
ENTITIES	text	yes	Comma-separated list of the entities recognized in the user message
DIALOG_ID	text	yes	Identifier for the dialog that produced Aura response
DIALOG_ID.keyword	keyword	yes	Keyword version of `DIALOG_ID`, to enable aggregating on it
DURATION_NU	number	yes	Elapsed time, in ms, between the reception of the user message and the moment the response is generated to be sent to the channel
userType	keyword	no	A single char identifier that characterizes the user as a test user
usrMsgWc	number	no	Message word count: number of words contained in the user message
usrMsgSig	keyword	no	Message signature: a string that helps clustering user messages
AuraMsgGroup	keyword	no	Cluster the Aura response belongs to
weekday	number	no	Day of the week the interaction happened (0=Monday to 6=Sunday)
hour	number	no	(Integer) hour the interaction happened
country	keyword	partial	Two-letter code for the country
sesId	keyword	no	Session information
sesSize	number	no	Session information
sesDuration	number	no	Session information

Fields explanations

This subsection contains more detailed descriptions of some of the key fields in the schema.

AURA_ID_GLOBAL

This element (mostly) uniquely identifies the user generating the interaction.

Note the concrete value of this field is not the same as the actual identifier used within Aura and uploaded to Kernel: for privacy reasons, the identifier was hashed when generating the PPD and has no resemblance to the original one. The correspondence is however maintained across time, so it is possible to analyse user behavior.

The “mostly” qualifier reflects one quirk of the original Aura identifier: it is generated with a dependence to the authentication method used by the channel, so if two channels follow different authentication methods (e.g., MobileConnect vs. User/Password) then the AURA_ID_GLOBAL identifier for the same user will be different. In summary:

The identifier stays the same for a given user across time.
Different users will not have the same identifier.
But the same user could produce two different identifiers if connected to two channels that use a different authentication method.

AURA_ID

This is a “local” identifier, i.e., one that is generated inside the channel according to specific channel characteristics and it is not tied as much as AURA_ID_GLOBAL to user authentication.

Its main disadvantage is its transient nature: the same user, on the same channel, could generate different AURA_ID strings when connecting different times on a different session. Therefore, for user accounting and tracing, AURA_ID_GLOBAL is usually preferred.

However, there are instances in which AURA_ID works better, namely for anonymous access (when the user is not authenticated). This depends on the channel:  

In the WhatsApp channel, the initial use of the channel will be anonymous from the Aura side (i.e., no authentication is done), hence AURA_ID_GLOBAL will also be empty (at least until the user authenticates, which depends on the use case). But in this channel, AURA_ID has a permanent value, linked to the WhatsApp user, so here it is a good substitute for a persistent id, even for unauthenticated users.

MESSAGE_USR

This field includes the message sent by the user.

It has been partially processed to enhance anonymization by removing some standard identifiers contained in it with <idxxx> strings (e.g., phone numbers appear as <idphone>).

Removal is done mostly through regular expressions, so there might be occasional glitches (such as identifying as phone a number that does not really correspond to a phone, just because it follows the phone number pattern).

MESSAGE_USR is a field of text type. As such, it is searchable: it is possible to search for specific words the user might have said.

Furthermore, it has been processed through an ElasticSearch analyzer adapted to the specific language used. This means that searches are able to match related words (e.g., plural versions of a singular query word, or verb conjugations). Phrase searches are also possible (by using double quotes around the phrase). If a phrase (several words) is used as a query without the quotes, ElasticSearch interprets it as a query for any of the words, so it will return all data elements that contain any of the words in the query.

In Kibana, more sophisticated text searches can be made by switching Lucene query syntax: proximity queries (words close to each other), fuzzy searches (query words allowing typos), wildcards, etc.

MESSAGE_USR_NORM

This is a normalized version of MESSAGE_USR, in which the user text has been streamlined by:

Converting all the sentence to lowercase
Removing all punctuation
Removing any extra spaces

Furthermore, this field is not processed through a language-dependent analyzer as MESSAGE_USR is, so queries on this field must match words exactly. It is still a text type field. However, the same query language can be used.

MESSAGE_AURA

This contains the text message generated by Aura and sent to the user as response to the user query. It is a text type field, so it is possible to search for specific words in it.

In the current version of Aura KPIs logs, this field only contains the text response. Some Aura use cases do not generate a purely textual message, but a more elaborated one (e.g., a card with text and graphics). These complex answers are inserted as attachments into Aura’s response to the channel and since attachments are not logged into the MESSAGE field, this field will appear empty in those cases. So, an empty MESSAGE_AURA field does not necessarily mean that Aura did not provide an answer. As an alternative for those situations, looking at the DIALOG_ID field (or INTENT) may give a hint of the type of answer that Aura delivered.

 MODALITY_CD_USR

This field contains the modality in which the user sent the message.

It is a slightly transformed field because there are some variations across Aura versions and, in order to unify it, the modalities are consolidated into only four different keywords: audio (spoken message), text (written free-text message) o form (commands sent via automatic processing or menus).

 DIALOG_ID

This field contains the identifier for the user case dialog module at the aura-bot Framework that was selected to construct the Aura response.

Dialog identifiers have two components (library  and dialog) separated by a colon e.g., services:service-usage

This field uses a custom analyser that splits the identifier at the colon, generating two terms. This makes possible to construct queries with one of the terms, e.g., “give me all the elements for the domain services”. But being a text field makes it impossible to do aggregations on it, so it cannot be used for statistics like bar charts (use DIALOG_ID.keyword for that).

DURATION_NU

This number reflects the time that took Aura to understand, process and respond to the user message. It is the difference (in milliseconds) between the timestamp of the moment the user message was received and the timestamp in which Aura’s response was finalized and sent to the channel.

Note that it is not a complete end-to-end delay time from the user’s point of view, since it does not include either the time it took the request to arrive to Aura through the channel or the time it took the response to travel back through the channel and get rendered at the client application (those times are outside Aura, and as such not registered by it).

Session Information

Session information includes the fields: sesId, sesSize, sesDuration.

These fields are generated by running a process over the time series formed by interactions from each user at each channel.

A session is automatically identified as a consecutive list of such user’s interactions, each separated from the next by a time interval shorter than 5 minutes. Once each session is identified, it is tabulated and labelled with three fields:

 sesId: string, forming a unique identifier for the session. It should be considered as an opaque identifier and the guarantee is that no other session in the data stream carries the same identifier.
As an aside, interactions that do not correspond to actual user interactions (because no user could be identified or because the datapoint corresponds to an interaction not triggered by the user) are all labelled with a <void> sesId.
 sesSize: number of interactions this session contains. This is labelled only for the first interaction in the session, all other interactions carry a 0 in this field. Non-sessions such as the ones with <void> sesId will be left empty. This facilitates computing averages or other statistics on valid sessions, by just first filtering out all zero and empty values.
 sesDuration: time duration for each session, counted from the instant the first user message was received, to the instant the last Aura message was sent. For single-interaction sessions its value will be the same as DURATION_NU, for multiple interactions it will contain the time interval between all of them.

As with sesSize, only the first interaction in a session is annotated with sesDuration; the remaining interactions will be assigned a 0 value (and interactions that do not correspond to a session will be left empty). Therefore, to compute statistics on sesDuration, remove the 0 and empty values first.

userType

This field may be used, in certain cases, to help identify rows that do not correspond to real users but to test users (internal users that belong to test/QA teams and whose behaviour is, therefore, not representative of actual Aura users).

The field contains a single character, which is s for standard (real) users, and can be Q or T for QA/Test users respectively (there are also lowercased versions q and t, referring to unconfirmed test users).

Note that test user identification is not available on every country, since it depends on having a register of the AURA_GLOBAL_ID identifiers that QA/Test users authenticate and this is not always available.

usrMsgSig

This field is not useful by itself. Instead, it is intended to be used to help grouping together very similar user utterances. It does so by generating a signature of the utterance that is (hopefully) insensitive to small variations in the sentence.

This is an experimental field; it might change if we reach a variant that is better suited for its purpose.

The way to generate this signature is by following these steps with the utterance:

Start with the normalized utterance (i.e., MESSAGE_USR_NORM).
Perform stemming (removal of word suffixes) on all the words. This makes bills and bill the same word.
Substitute words from a fixed list of very common, uninformative tokens (stopwords) by an asterisk. For example, this converts both “get my bill” and “get the bill” to the same phrase “get * bill”.
Group words in sets of 3 elements (trigrams) and sort them alphabetically. This removes the global structure of the sentence, while retaining local structure.

The resulting string is a non-understandable version of the original utterance (hence, it cannot be used by itself), but the fact that several very similar utterances produce the same signature helps cluster those utterances. An example is one of the preinstalled visualizations “Most Frequent User Utterances” which uses this field to group very similar utterances.

Another example is provided in the following figure, which shows message utterances generating the same signature:

As it can be seen, the signature is the same for “how can I upgrade” and “when can I upgrade”, “when does my contract end” and “when is my contract ending”, and “live chat” & “live chats”. So, they would be counted together when aggregating by signature.

The procedure has its limitations and, as explained, it is experimental, so we are trying to improve it, but it can already alleviate a bit the inherent variability in user expressions.

AuraMsgGroup

Messages produced by Aura are as generated by its text resource database. In some cases, the same category of message produces different output texts, maybe because the message includes some user-dependent parameter or because the text database contains several variants of the same text (and Aura picks one at random).

The AuraMsgGroup field is a keyword field that helps categorize Aura answer by abstracting away some of this variation. It classifies the response given by Aura into two types of elements:

Generic group: a name such as <NONE>, <GREETING> or <NOTFOUND>, which corresponds to a response category (see Table 3)
Truncated answer: for answers that do not have a defined generic group, as a fallback the literal answer text is inserted, after substituting all numbers in it with a placeholder and truncating it (i.e., retain only the first characters).

The following table contains the generic groups defined so far. They correspond to the most frequent Aura messages. It is country-dependent, since it also depends on the use cases deployed in each country. As said above, responses not falling into these groups will be assigned a truncated version of the response text.

Note that th emost frequent Aura messages list can be enlarged with time. Also, the correspondence between Aura messages and groups is not static, if the text database is updated with new variants, it will be necessary to also update the translation table in the PPD cleaner process that generates this field.

Group	Meaning
EMPTY	No textual answer from Aura (see note in Section MESSAGE_AURA for the usual meaning of no text answer)
NONE	Aura says it did not understand the user utterance
ERR	There was a processing error of some kind at Aura side, and the request could not be fulfilled
GREETING	Aura is greeting the user
GOODBYE	Aura is acknowledging a conversation end
YOU-ARE-WELCOME	Aura is accepting a compliment
CHURN	Aura recognizes the user intention to terminate a contract
NOTFOUND	Aura tried to search for some bit of data concerning the user query, and could not find it
CANNOT	Aura cannot fulfil the user request because of insufficient information (in the query, or on user data)
BILL-INFO	The user requested information about her bill, and Aura is returning it
DATA-INFO	The user requested information about her data usage, and Aura is returning it

: The list can be enlarged with time. Also, the correspondence between Aura messages and groups is not static, if the text database is updated with new variants, it will be necessary to also update the translation table in the PPD cleaner process that generates this field.

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Context global data model

Description of the global model in which Aura Context data is structured

Introduction

Aura Context data is structured in context fields, each of them defined by a pair name-value.

This model is common for all developers willing to use Aura Context: Aura Global Team defines and manages the model and uploads data from Aura modules, channels or devices into the Aura Context database.

An Aura Context developer:

Can consume this data directly, if the global data model is fully compatible with his own data model, or
Can use a transform or a file schema in order to modify or add new fields to the database

Certain key considerations regarding the Aura Context global data model are included below:

Different Aura components, channels or devices can upload data to certain specific fields of the global database.
It is possible to have fields with no filled data in the database due to several reasons. For example: in a specific lapse of time, information has not been sent from a module, channel or device; a specific channel is not available, thus its associated fields are not of application, etc.
However, there are certain mandatory fields that correspond to the table “Identification fields” of the database that must be always filled. These fields cannot be modified via an Aura Context transform.
The storage of data into the database is always based on:
- A user, identified by the field aura:idGlobal
- An interaction, identified by the field aura:corrId
The name of the field is different from the name of the field that is stored in the database. Both are included in columns 1 and 2 of the following tables.

Identification fields

Name	Name in database	Format	Semantics	Current Aura Context version
`aura:idGlobal`	`aura_id_global`	string (UUID)	`aura_id_global` for the user	✔️
`aura:userId`	`user_id`	string (UUID)	`userId` in the OB	✔️
`aura:corrId`	`corr`	string (UUID)	`corr_id` corresponding to the current interaction	✔️
`aura:timestamp`	`timestamp`	ISO 8601 (UTC, with milliseconds)	First transaction storage timestamp	✔️ (automatically filled)
`aura:timestampLastUpdate`	`timestamp_last_update`	ISO 8601 (UTC, with milliseconds)	Last timestamp update of this interaction	✔️ (automatically filled)
`aura:activeMsisdn`	`active_msisdn`	string (country prefix and MSISDN integer number)	Currently active MSISDN of the interaction. Empty if a MultiMSISDN user is logged with user/pass.	✔️ (phoneNumber)
`aura:userType`	`data.user_type`	enum string: monoMSISDN, multiMSISDN, anonymous, etc.	Type of Aura user	✔️
`aura:subscriptionType`	`subscription_type`	enum string: prepaid, postpaid, control, other	Type of user subscription. Empty, if a multiMSISDN user is logged with user/pass.	✔️
`aura:accountNumber`	`account_number`	string	User AccountNumber in TV use cases. Empty for other use cases.	✔️

Application

Name	Name in database	Format	Semantics	Current Aura Context version
`application:id`	`application_app_id`	string (UUID or similar)	Unique identifier for the application (it changes with each application version)	✔️
`application:channelName`	`application_app_channel_name`	string	Canonic name of the channel where the request is sent by the user	✔️ (prefix format)
`application:contextFilters`	`application_app_context_filters`	Array of strings	`application:contextFilters` semantics. String to filter the final response sent by aura-bot instance back to the Telefónica client. I.e., to determine the most appropriate answer to a generic question. The format of each entry of the array should be `key:value`, where both key and value are application dependant.	✔️
`application:playId`	`application_app_play_id`	string (integer, UUID or similar)	Identifier for the content being played (local or remote)	✔️
`application:playMode`	`application_app_play_mode`	string: `local` or `remote`	Content playing mode set in the application	✔️
`application:playType`	`application_app_play_type`	string: `live`, `catchup`, `vod`, etc.	Content type being played	✔️
`application:playServiceId`	`application_app_play_service_id`	string	Identifier of the service to be played	✔️
`application:playServiceUID`	`application_app_play_service_uid`	string	Unique Identifier of the service being played	✔️
`application:playLiveProgramId`	`application_app_play_live_program_id`	string	Identifier of the show being played	✔️
`application:timezone`	`application_app_timezone`	IANA timezone	Timezone configured at the application/client	✔️
`application:userAction`	`application_app_user_action`	string, app-dependent. Eg: `playContent`, `searchContent`, `viewAccountDetails`, `phoneCall`, etc.	A token describing the action the user was carrying on in the device that generated the request when the Aura request was launched	Field currently not filled
`application:interfaceLanguage`	`application_app_interface_language`	string: ISO 639-1 + ISO 3166 (joined by underscore)	Language in which the application is configured	✔️
`application:channelDataVersion`	`application_app_channel_data_version`	string	Version sent in the channel request	✔️
`device:id`	`application_device_id`	string (UUID or similar)	Unique identifier for the device running the application	✔️ (from device)
`device:type`	`application_device_type`	string: `homebase`, `ios.cell`, `android.cell`, `ios.tablet`, `android.tablet`, `pc`	Device class	✔️ (from device)
`application:screenId`	`application_app_screen_id`	string (application dependent)	Identifier designating the application section/screen where the Aura request came from	✔️
`application:screenElements`	`application_app_screen_elements` Name in the database for internal sub-fields: `ucid`; `type`; `resume_time`; `live_program_id`; `focus`	Array of maps showing content in the device. Each map has these fields: `ucid`: string; `type`: string; `resumeTime`: string; `liveProgramId`: string; `focus`: boolean	List of elements that appears in the application screen. It could be empty.	✔️
`output:showSuggestions`	`application_output_show_suggestions` Name in the database for internal sub-fields: `suggestion_id`; `intent`; `entities`; `type`; `value`; `canon`; `label`; `score`; `text`; `timestamp`; `show_to_user`	Array of maps with suggestions sent to the channel: `suggestion_id`: string; `intent`: string; `entities`: array of maps; `type`: entity-type; `value`: entity-value; `canon`: canon-value; `label`: label-value; `score`: score-value; `text`: string; `timestamp`: ISO 8601; `show_to_user`: boolean	List of suggestions shown to the user. Each has a unique identifier (`suggestion_id`) to correlate the effect on the next user’s action and contains a timestamp of the approximate moment when it was shown to the user. The rest of elements are the suggestion components, including intent, entity types (useful for further semantic analysis on analytical models), entities and the current text shown. The field `show_to_user` tells whether the suggestion has been shown to user by the channel.	Field currently not filled

Environment

Name	Name in database	Format	Semantics	Current Aura Context version
`location:logical`	`environment_location_logical`	string: home, outsidehome	Current logical location for the user	✔️

Conversation

Name	Name in database	Format	Semantics	Current Aura Context version
`input:modality`	`conversation_input_modality`	string: `voice`, `text`, `form`	Modality for the input user interaction	✔️
`input:utterance`	`conversation_input_utterance`	string	Utterance sent by the user. Empty if auraCommand.	✔️
`input:command`	`conversation_input_command`	string	Stringified of the auraCommand object	✔️
`recog:module`	`conversation_recog_module`	string: `nlp`, `value`, `text`, `command`	A string identifying the module that was responsible for the NLP decision	✔️
`recog:domain`	`conversation_recog_domain`	string	Domain recognized by NLP	Field currently not filled
`recog:resultList`	`conversation_recog_result_list` Name for internal sub-fields: `id`; `score`	`id`: string; `score`: number	List of recognized results (up to 3), ordered by they corresponding scores.	✔️
`recog:entities`	`conversation_recog_entities` Name for internal sub-fields: `type`; `value`; `canon`; `label`; `score`	Array of maps: `type`: entity-type; `value`: entity-value; `canon`: canon-value; `label`: label-value; `score`: score-value	List of recognized entities, regarding only to the first intent returned during the main recognition phase	✔️
`recog:negativity`	`conversation_recog_negativity`	double: value in [0, 1]	Level of negative sentiment identified by a trained model	Field currently not filled
`dialog:intents`	`conversation_dialog_intents`	array of strings	Each intent redirection made by the bot will add the name of the intent to the list.	Currently, only redirections with contex-filter are stored
`dialog:ids`	`conversation_dialog_ids`	array of strings	Each dialog redirection made by the bot will add the id of the dialog to the list.	Currently, only first dialog id is stored
`output:text`	`conversation_output_text`	string	Message sent back to the user	✔️
`output:speak`	`conversation_output_speak`	string	Spoken message sent back to the user	✔️
`output:suggestions`	`conversation_output_suggestions` Name for internal sub-fields from ‘suggestions’: `type`; `value`; `canon`; `label`; `score`	Array of suggestion results. A suggestion is a map with type, id and (optionally) an array of entities as a map. The entities array contains: `type`: entity-type; `value`: entity-value; `canon`: canon-value; `label`: label-value; `score`: score-value	Produced list of suggestions	✔️

User

Name	Name in database	Format	Semantics	Current Aura Context version
`userProductList`	`user_user_product_list`	array of strings	List of products/services contracted by the user	Field currently not filled
`videoProfileName`	`user_video_profile_name`	string	The video profile assigned to the user	Field currently not filled

Data model in Kernel

The process of data storage in Kernel is automatic and transparent for developers.

Information is stored in Kernel datasets using URM format. These specific datasets are:

Aura_Agent_Context

{
  "namespace": "com.plainAVRO",
  "name": "Aura_Agent_Context",
  "type": "record",
  "doc": "Contains the data for Aura context stored by agents for each user and correlator",
  "x-fp-version": "5.9.0",
  "fields": [
    {
      "name": "OPERATOR_ID",
      "aliases": [
        "operator_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Global Operator Identifier (Operator acting as owner of the information present in the current entity)"
    },
    {
      "name": "USER_4P_ID",
      "aliases": [
        "user_4p_id"
      ],
      "type": {
        "type": "string",
        "x-fp-user-id": true
      },
      "doc": "Identifier of the user in 4th Platform (as returned by the OB in the 4th Platform APIs)"
    },
    {
      "name": "RECORD_ID",
      "aliases": [
        "record_id"
      ],
      "type": "string",
      "doc": "Id for the record in context"
    },
    {
      "name": "AGENT_DES",
      "aliases": [
        "agent_des"
      ],
      "type": "string",
      "doc": "Agent name using context"
    },
    {
      "name": "FIELD_DES",
      "aliases": [
        "field_des"
      ],
      "type": "string",
      "doc": "Agent stored field name"
    },
    {
      "name": "VALUE_DES",
      "aliases": [
        "value_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Agent stored field value"
    },
    {
      "name": "PARTITION_TM",
      "aliases": [
        "partition_tm"
      ],
      "type": {
        "type": "string",
        "logicalType": "datetime"
      },
      "doc": "Date-time of the record, for data partitioning purposes"
    }
  ]
}

Aura_Context

    {
      "name": "APPLICATION_APP_PLAY_ID",
      "aliases": [
        "application_app_play_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Identifier for the content being played"
    },
    {
      "name": "APPLICATION_APP_PLAY_MODE_DES",
      "aliases": [
        "application_app_play_mode_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Content playing mode set in the application"
    },
    {
      "name": "APPLICATION_APP_PLAY_TYPE_DES",
      "aliases": [
        "application_app_play_type_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Content type being played"
    },
    {
      "name": "APPLICATION_APP_PLAY_SERVICE_ID",
      "aliases": [
        "application_app_play_service_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Identifier of the service being used to played"
    },
    {
      "name": "APPLICATION_APP_PLAY_SERVICE_U_ID",
      "aliases": [
        "application_app_play_service_u_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Unique identifier of the service being played"
    },
    {
      "name": "APPLICATION_APP_PLAY_LIVE_PROGRAM_ID",
      "aliases": [
        "application_app_play_live_program_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Identifier of the show being played"
    },
    {
      "name": "APPLICATION_APP_TIMEZONE_DES",
      "aliases": [
        "application_app_timezone_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Timezone configured at the application/client"
    },
    {
      "name": "APPLICATION_APP_USER_ACTION_DES",
      "aliases": [
        "application_app_user_action_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "A token describing the action the user was carrying on in the device that generated the request when the AURA request was launched"
    },
    {
      "name": "APPLICATION_APP_INTERFACE_LANG_ID",
      "aliases": [
        "application_app_interface_lang_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Language the application is configured in"
    },
    {
      "name": "APPLICATION_APP_CHANNEL_DATA_VERSION_DES",
      "aliases": [
        "application_app_channel_data_version_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Version sent in the request of the channel"
    },
    {
      "name": "APPLICATION_APP_SCREEN_ID",
      "aliases": [
        "application_app_screen_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "An identifier designating the application section/screen where the AURA request came from"
    },
    {
      "name": "APPLICATION_APP_SCREEN_ELEMENTS_ID",
      "aliases": [
        "application_app_screen_elements_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Elements that appear in the screen of the application"
    },
    {
      "name": "APPLICATION_DEVICE_ID",
      "aliases": [
        "application_device_id"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "A unique identifier for the device running the application"
    },
    {
      "name": "APPLICATION_DEVICE_TYPE_DES",
      "aliases": [
        "application_device_type_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "The device class"
    },
    {
      "name": "APPLICATION_OUTPUT_SHOW_SUGGESTIONS_DES",
      "aliases": [
        "application_output_show_suggestions_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Suggestions shown to the user"
    },
    {
      "name": "ENVIRONMENT_LOCATION_LOGICAL_DES",
      "aliases": [
        "environment_location_logical_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Current logical location for the user"
    },
    {
      "name": "CONVERSATION_INPUT_MODALITY_DES",
      "aliases": [
        "conversation_input_modality_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Modality for the input user interaction"
    },
    {
      "name": "CONVERSATION_INPUT_UTTERANCE_DES",
      "aliases": [
        "conversation_input_utterance_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Utterance sent by the user"
    },
    {
      "name": "CONVERSATION_INPUT_COMMAND_DES",
      "aliases": [
        "conversation_input_command_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Stringified of the Aura Command object"
    },
    {
      "name": "CONVERSATION_RECOG_MODULE_DES",
      "aliases": [
        "conversation_recog_module_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "A string identifying the module that was responsible for the NLP decision"
    },
    {
      "name": "CONVERSATION_RECOG_DOMAIN_DES",
      "aliases": [
        "conversation_recog_domain_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Domain recognized by NLP"
    },
    {
      "name": "CONVERSATION_RECOG_RESULT_TYPE_DES",
      "aliases": [
        "conversation_recog_result_type_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Type of result produced by NLP"
    },
    {
      "name": "CONVERSATION_RECOG_RESULT_LIST_DES",
      "aliases": [
        "conversation_recog_result_list_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Recognized results with their scores"
    },
    {
      "name": "CONVERSATION_RECOG_ENTITIES_DES",
      "aliases": [
        "conversation_recog_entities_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Recognized entities"
    },
    {
      "name": "CONVERSATION_RECOG_NEGATIVITY_QT",
      "aliases": [
        "conversation_recog_negativity_qt"
      ],
      "type": [
        "null",
        "float"
      ],
      "doc": "Grade of negative sentiment identified by a trained model"
    },
    {
      "name": "CONVERSATION_DIALOG_INTENTS_DES",
      "aliases": [
        "conversation_dialog_intents_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Each intent redirection made by the bot"
    },
    {
      "name": "CONVERSATION_DIALOG_IDS_DES",
      "aliases": [
        "conversation_dialog_ids_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Each dialog redirection made by the bot"
    },
    {
      "name": "CONVERSATION_OUTPUT_TEXT_DES",
      "aliases": [
        "conversation_output_text_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Message sent back to the user"
    },
    {
      "name": "CONVERSATION_OUTPUT_SPEAK_DES",
      "aliases": [
        "conversation_output_speak_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Message to be spoken sent back to the user"
    },
    {
      "name": "CONVERSATION_OUTPUT_SUGGESTIONS_DES",
      "aliases": [
        "conversation_output_suggestions_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Produced suggestions"
    },
    {
      "name": "USER_USER_PRODUCT_LIST_DES",
      "aliases": [
        "user_user_product_list_des"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "Products/services contracted by the user"
    },
    {
      "name": "USER_VIDEO_PROFILE_NAME",
      "aliases": [
        "user_video_profile_name"
      ],
      "type": [
        "null",
        "string"
      ],
      "doc": "The video profile assigned to the user"
    },
    {
      "name": "PARTITION_TM",
      "aliases": [
        "partition_tm"
      ],
      "type": {
        "type": "string",
        "logicalType": "datetime"
      },
      "doc": "Date-time of the record, for data partitioning purposes"
    }

Docs:

Mon, 01 Jan 0001 00:00:00 +0000

Aura Bot object, data types and models

Description of the different types of data and models supported by Aura Bot

aura-bot supports several data models, compatible with different languages or structures in order to execute the conversational flow.

The current document includes the description of the different models supported by Aura Bot platform, currently:

auraCommand model
The use of auraCommands, that is, pre-defined options that the user can select from Aura’s interface, is managed by the auraCommand model.

dialogContext model
Aura supports the dialogContext model, a structure that facilitates the recognition of CardActions and prompts without the need of training.

Aura API Key
Aura counts on API Keys to connect to available API endpoints. They are based on an encrypted APIKey data model, which is included in aura-validate-apikey utility.