1 - Handling channelData versions
Handling channelData versions
Description of components that make Aura Bot capable of working with different channelData versions.
UML diagram
@startuml
title ChannelData handling diagram
actor User
participant AuraGroot
participant ChannelDataValidatorMiddleware #76bbe7
participant AuraBot #ebdff7
participant Routing #ebdff7
participant MainDialog #1add4d
participant Dialog #1c733b
participant OutgoingChannelDataNormalizerMiddleware #73c1c3
participant Adapter #1bcad5
participant AuraChannelDataHandler #ebba65
User -> AuraGroot: User request
AuraGroot -> ChannelDataValidatorMiddleware: request reaches validation middleware
alt AURA_CHANNELDATA_VALIDATION === true
ChannelDataValidatorMiddleware -> AuraChannelDataHandler: isValidatableVersion checks if channelData request version is validatable
AuraChannelDataHandler -> ChannelDataValidatorMiddleware: isValidatableVersion returns if version is valid
alt isValidatableVersion === true
ChannelDataValidatorMiddleware -> AuraChannelDataHandler: getRequestValidator validates channelData request
AuraChannelDataHandler -> ChannelDataValidatorMiddleware: getRequestValidator returns validation
end
end
ChannelDataValidatorMiddleware -> AuraBot
AuraBot -> MainDialog: Route to main dialog
MainDialog -> Routing: MainDialog routing step
alt channelDataRequest <= v3
Routing -> Routing: Add information to triggerCondition
end
alt channelDataRequest >= v3
Routing -> Routing: Add information to triggerCondition (intent.vX)
end
Routing -> MainDialog: checkLegacyDialogStep check if a conversion to legacy channel data is needed
alt channelDataRequest <= v3
MainDialog -> AuraChannelDataHandler: getNormalized normalizes request to supported aura-bot channelData
end
MainDialog -> Dialog: Reach dialog by intent if exists
alt channelDataRequest === v3 && channelDataVersion === v3
Dialog -> AuraChannelDataHandler: formatChannelDataV3 formats response to be compatible with v3 version
AuraChannelDataHandler -> Dialog: formatChannelDataV3 returns formatted v3 channelData
end
Dialog -> OutgoingChannelDataNormalizerMiddleware: Pass activity formed in dialog to outgoingMiddlewares
alt channelDataRequest <= v3
OutgoingChannelDataNormalizerMiddleware -> AuraChannelDataHandler: getNormalized Normalizes activity (output) and request (input) to be compatible with supported aura-bot version
AuraChannelDataHandler -> OutgoingChannelDataNormalizerMiddleware: getResponseChannelData Gets original response channel data
end
OutgoingChannelDataNormalizerMiddleware -> Adapter: Middleware's pipeline is finished and result is received by the adapter
alt AURA_CHANNELDATA_RESPONSE_VALIDATION === true
Adapter -> AuraChannelDataHandler: isValidatableVersion checks if channelData response version is validatable
AuraChannelDataHandler -> Adapter: isValidatableVersion returns if version is valid
alt isValidatableVersion === true
Adapter -> AuraChannelDataHandler: getResponseValidator gets validation for the response version
end
end
alt channelDataRequest >= v3
Adapter -> Adapter: Add version parameter to response.
end
alt channelDataRequest < v3
Adapter -> Adapter: getResponseChannelData gets transformed channel data from correct version (removes version field)
end
Adapter -> AuraBot: Activity is processed and ready to be sent
AuraBot -> User: Bot response
@enduml
The version format will be a semantic versioning, (MAJOR.MINOR.PATCH). However, aura-bot will work exclusively with the MAJOR version:
If a customer sends the version 3.1.0, for Aura the version is 3.
Aura will keep the original version from the request in the response. That is, if the client sends version 3.0.0, but Aura is currently working with version 3.5.1 as Aura works with the older one, there is no problem and Aura will keep the version that the client sent, 3.0.0, in the response.
For versions greater or equal to 3.0.0, aura-bot accepts either SemVer versioning (3.0.0) or major versioning (3) in channelData.version property.
Request channelData validation
The channeldata-validator-middleware executes request channelData validations. aura-bot receives the request which contains information about the channelData version. With that information, different validations are made.
More information regarding validations being made and utilities that used for this flow can be found in channelData validation.
Aura Bot channelData versions behavior
- aura-bot is able to work with channelData versions
v1, v2 and v3.
- If a channel does not specify the version, it is assumed to be working with version
v1.
- Version
v2 was specific for living-apps, and version v3 is the first version of the normalized channelDatamodel.
- Use cases created for
channelData version v1 will not be able to execute dialogs created with the v3 model and vice versa.
The format for intents in channelData version v3 only changes internally.
Therefore, when defining an intent, it is not necessary to specify its version, the dialog loading system will do it internally.
Dialog configuration
From channelData v3 onwards, it is necessary to specify the version to be used in the channelData dialogs.
If no version is specified, it is understood that they work in v1 mode for classic use cases.
If you want to work with the channelData normalized version (v3), specify it in the use case dialog configuration file.
Example:
{
"name": "packageLibTest",
"dialogs": [
{
"id": "channel-data-V3-dialog-example",
"channelDataVersion": "v3", <- It's mandatory specific version for ChannelData v3
"triggerConditions": [
{
"intent": "intent.factotum-test.channel-data-model"
}
]
}
]
}
When a dialog has specified its channelData version through the channelDataVersion property, it can only be executed if the client has specified that version in the request. Otherwise, the system will return an “I don’t understand you”.
A client can have two dialogs for the same use case with different versions, but it is mandatory that the names of the dialogs are different. The intents must be called exactly the same for v3 and for others, as the format of the intents is set internally and it is not necessary to specify anything.
{
"name": "packageLibTest",
"dialogs": [
{
"id": "channel-data-V1-dialog-example",
"triggerConditions": [
{
"intent": "intent.factotum-test.channel-data-model"
}
]
}
]
}
This configuration contains the same intent as the previous one, but the dialog is for channelData version v1, as its version has not been set. It should be noted that if you set a version channelDataVersion: v1 it will also work in v1 mode.
Routing to version v3
Loading Dialogs configuration
The first step to set the intents to their proper version must be done during the dialogs loading. This is done inside the aura-configuration package in the document channels-configuration-current.ts.
private buildAuraChannel(channelConfiguration: ChannelConfiguration): ChannelConfiguration {
........
// NORMALIZATION. Assign the intent to the channel Data Version
channelConfiguration.dialogLibraries?.forEach((library) => {
library.dialogs.forEach((dialog) => {
const channelDataVersion = this.getChannelDataVersionMajor(dialog.channelDataVersion);
if (channelDataVersion > 2) {
dialog.triggerConditions.forEach((triggerCondition) => {
triggerCondition.intent = this.getIntentByChannelDataVersion(triggerCondition.intent, dialog.channelDataVersion);
});
}
});
});
.......
Routing configuration
Intents whose dialogs have the property channelDataVersion with a value greater than or equal to v3 will name their intents with the corresponding version.
Example:
-
“channelDataVersion”: “v3” -> intent.[intent_name].v3
-
“channelDataVersion”: “v2” -> intent.[intent_name]
-
“channelDataVersion”: “v1” -> intent.[intent_name]
-
“No channelDataVersion” -> intent.[intent_name]
For the previous example, the dialog routing would be as follows:
Establishment of intent versioning in recognizers
When a channel sets a version of the channelData greater or equal to v3, the routing step will set intent format in getIntentByChannelDataVersion(), which can be found in aura-configuration.
public static setIntentResult(context: TurnContext, intentResult: any, utterance: string, promptCheck: boolean,
......
intentResult.intent = AuraChannelsConfiguration.instance.getIntentByChannelDataVersion(intentResult.intent, ChannelDataUtils.getChannelDataVersion(context));
......
}
Routing to v3 dialog
Once the routing and the recognizers steps are executed, based of the result obtained, it will be routed to a v3 dialog if channelDataVersion: "v3" and there is match in the recognized intent.
Dialogs in v3
Currently, only certain dialogs have been migrated to channelData v3 version,as specified in the following sections.
Greetings v3 dialog
Greetings dialog, which is included in common library, has been migrated to v3.
For more information about this implementation, check Create/migrate a dialog to channelData v3.
didntUnderstand special case
There is a particular case: a request needs a V3 dialog and this is not implemented. In this case, None dialog should be triggered.
At the moment, None has not being migrated to v3 so this type of requests are redirected to bot’s fallback case didntUnderstand.
This occurs due to routing appending v3 to None intent and not finding a match. This case returns an activity with the resource core:core.cancel
Prompt case
In this particular case, the last message processed by the dialog must have the channelData correct version.
Firstly, the prompt is displayed to the user going through the mainDialog, but user’s response to that prompt does not follow the same path.
During this stage, channelData is converted into v3 for compatibility with aura-bot core.
If the dialog is in version v1, channelData will not need to be modified, so the channeldata-validator-middleware will check if the dialog is legacy and will convert its response channelData to v1.
Outgoing middlewares
Once aura-bot responds, this response is being processed by certain middlewares that work in v3 version. Hence, there are two possible flows:
-
User is requesting a v1 dialog
In this case, the response from the v1 dialog is converted into v3 immediately after the activity is created.
Once the activity is being processed by the middleware’s pipeline, it is converted from v3 to v1 by the outgoingchannelDataNormalizer middleware, that performs the same actions in reverse order than incomingchannelDataNormalizer middleware.
Once the activity is being processed by the middleware’s pipeline, it is converted from v3 to v1 in the Aura BotFramework adapter.
-
User is requesting a v3 dialog
In this case, the response from the v3 dialog is already in the current supported version, so there is no need to merge to other version.
Adapter
Before sending the response to the user, there are a number of extra steps which are performed inside the Aura BotFramework adapter.
channelData validation
As done in the request stage, there is another validation in the response, before sending the resulting activity to the user, checking if the sent schema is valid for the activity version.
Find more information about which validations are made and utilities used for this flow in channelData validations.
Activity merging
For more information about how channelData merges are processed, check channelData mapping.
2 - channelData mapping
channelData mapping
channelData mapping in Aura response
Introduction
Currently, Aura supports dialogs able to provide responses in v1, v2 and v3 channelData versions. These responses are mapped to be handled by the bot output middlewares as v3, but after that, dialogs are mapped again to the original response version to be returned to the client.
However, incoming middlewares are able to work with different request channelData versions v1, v2 and v3, indistinctly.
Request
The request is currently not transformed and is accessed using channelData utils from the aura-bot-common library.
These utils make transparent the access to all channelData properties needed in v1 or v3.
Response
The response returned by the dialog is normalized in the outgoingchannelDataNormalizer middleware, the first outgoing middleware executed after the dialog.
This normalization is done if the channelData version returned is lower than the current version.
The middleware calls ChannelDataResponseMapper.getResponseChannelData with:
- The normalized version (currently,
v3) as first parameter.
- The channelData in the old version as second parameter.
This mapper function will return a normalized version of the channelData.
After all the middlewares are executed, in the adapter, the response should be transformed to the version returned by dialog again.
To do this, ChannelDataResponseMapper.getResponseChannelData
is called again with:
- The version originally returned by the dialog as first parameter.
- The
channelData as second parameter.
- The original
channelData returned by the dialog as third parameter. This third parameter is needed because in the conversion from old versions to the normalized version certain fields may disappear, so we need to merge the normalized channelData (maybe modified by the outgoing middlewares) with the original returned by the dialog.
Channels using a BotService adapter
Channels such as Facebook that use a BotService Adapter will always use a channelData without version, which internally is translated to v1.
This means that the request will have to be mapped for good. The major implication is that channelData version v3 must take into account the fields needed by these adapters, so the communication and the user’s experience are kept the same.
Currently, aura-bot is only validated to be accessed through the Facebook Messenger adapter. In this case, the protocol allows the exchange of HeroCards, but the way they are sent to the channel is a bit different than in Direct Line channels: they are sent as attachments of the channelData instead of attachments of the activity.
So, in order to allow Aura use cases to send buttons or a complete HeroCard to a user, the attachments field of the channelData must be kept during the version mapping.
3 - channelData validations
channelData validations
channelData validations done both in the request and the response, according to a certain configuration
Validations in request channelData
Request channelData will be validated if its version is greater than or equal to AURA_CHANNELDATA_VALIDATION_MIN_VERSION, that is always v3 (prior versions will never validate and will get all the channelData “as they are”).
That means, that any version from v3 onwards will be validated (even non existing versions, such as 99, will try to be validated).
Versions from v3 onwards will only use the major as a string (e.g., 3), but also semver is understood (e.g., 3.0.0), but note that minor and patch will be ignored (e.g., 3.2.1 === 3.0.0) when looking for a validator.
Logic location
The validation is done in the channeldata-validator-middleware, that must be one of the first middlewares, to ensure that no other logic is fetching data from channelData before validating it.
Errors
There are two types of errors on request validation:
- Validation error: the request
channelData does not comply the schema for that version. The error is stored in requestValidationErrors property within the thrown error.
- Missing validator error: the specified version does not have a matching validator, so
channelData is invalid.
Validations in response channelData
Response channelData will be validated if AURA_CHANNELDATA_RESPONSE_VALIDATION environment variable is set to true.
In that case, all outgoing activities will have their channelData validated, even the error messages, before sending to the user.
The validation must be done after the channelData is transformed into the final version (that could be or not the “internal” version).
Logic location
The response validation is done in Aura Groot, in AuraCloudAdapter class.
Errors
There are two types of errors on response validation:
- Validation error: the response
channelData does not comply the schema for that version and the error is stored in responseValidationErrors property within the thrown error.
- Missing validator error: the specified version does not have a matching validator, so the
channelData is invalid.