This is the multi-page printable view of this section. Click here to print.
Check the performance of Aura Bot components
The current documents contain different tests that can be executed over aura-bot components with several purposes:
How to test an Aura Bot dialog after its development: debug with mocha and Typescript
When developing a use case, and once the use case dialog is built, it is required to test the dialog in order to verify its proper performance in terms of the conversational flow between the user and aura-bot.
The current section includes the guidelines to debug with mocha and Typescript for the execution of the unit tests. This corresponds to the stage Test an Aura Bot dialog within the use case development process over aura-bot.
For this purpose, Bot Framework v4 provides a Test Adapter that allows launching unit tests for the validation of a component performance (mainly a dialog, but it can also be used for other components such as middlewares).
The Test Adapter simulates sending messages from the user to the bot, therefore it requires to mock certain data required by the dialog logic (i.e., Kernel data). With this data, the Test Adapter checks that the conversational flow of the dialog is correct. Tests are executed during each Pull Request and must be passed in order to merge the PR.
Add the following dependencies (or check that they are declared) in the package.json file, within the use case library.
An extract is shown below as an example.
"devDependencies": {
"@types/mocha": "^6.2.0",
"@types/chai": "^4.2.3",
"@types/jest": "^24.0.20",
"ts-node": "^8.4.1",
"mocha": "^6.2.1",
📌 Find a complete example for the common library in the following Github link.
The debug configuration is carried out using Visual Studio Code.
📄 For this purpose, read the Visual Studio Code documentation regarding launch configurations.
Debugging information is kept in the file launch.json, included in the folder .vscode of the developer’s workspace. The launch.json file must be included in the same root repository as the corresponding dialog to be tested.
Within this file, the field configuration must be filled with the following content:
{
"type": "node",
"request": "launch",
"name": "Mocha All",
"program": "${workspaceFolder}/node_modules/mocha/bin/_mocha",
"args": [
"-r",
"ts-node/register",
"--timeout",
"999999",
"--colors",
"${workspaceFolder}/src/**/*.spec.ts",
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"protocol": "inspector"
}
The Test adapter from Bot Framework 4 is used for executing unit tests. This adapter can be used to simulate sending messages from the user to aura-bot.
The following example sets up the test adapter and then executes a simple test:
const { TestAdapter } = require('botbuilder');
const adapter = new TestAdapter(async (context) => {
await context.sendActivity(`Hello World`);
});
adapter.test(`hi`, `Hello World`)
.then(() => done());
At this stage, you can send something to the bot and check the response provided by it.
This is simply a wrapper around calls to send() and assertReply() methods.
adapter.test('hi', 'Hello World')
.then(() => done());
@param userSays: text or activity simulating user’s input.@param expected: expected text or activity sent by the bot as a reply to the user’s input.@param description: (optional) description of the test case. If not provided, one is generated.@param timeout: (optional) number of milliseconds to wait for the bot response. Default value: 3000. public test(
userSays: string | Partial<Activity>,
expected: string | Partial<Activity> | ((activity: Partial<Activity>, description?: string) => void),
description?: string,
timeout?: number
): TestFlow {
return this.send(userSays)
.assertReply(expected, description);
}
An example is shown below:
const { TestAdapter } = require('botbuilder');
const adapter = new TestAdapter(async (context) => {
if (context.text === 'hi') {
await context.sendActivity(`Hello World`);
} else if (context.text === 'bye') {
await context.sendActivity(`Goodbye`);
}
});
adapter.test(`hi`, `Hello World`)
.test(`bye`, `Goodbye`)
.then(() => done())
.catch(e) => done(e));
This method sends something to aura-bot. It returns a new TestFlow instance which can be used to add additional steps for inspecting the bot reply and then sending additional activities.
📄 For further details, please check the corresponding the TestAdapter class documentation.
@param userSays: text or activity simulating user input.
public send(userSays: string | Partial<Activity>): TestFlow
This method generates an assertion if the aura-bot response does not match the expected text/activity.
@param expected: expected text or activity from the bot. It can be a callback to inspect the response using custom logic.@param description: (optional) description of the test case. If not provided, one is generated.@param timeout: (optional) number of milliseconds to wait for a bot response. The default value is 3000.public assertReply(expected: string | Partial<Activity> | estActivityInspector, description?: string, timeout?: number): TestFlow
An example is shown below:
const { TestAdapter } = require('botbuilder');
const adapter = new TestAdapter(async (context) => {
if (context.text === 'hi') {
await context.sendActivity(`Hello World`);
} else if (context.text === 'bye') {
await context.sendActivity(`Goodbye`);
}
});
adapter.send(`hi`,
.assertReply(`Hello World`)
.send(`bye`)
.assertReply(`Goodbye`)
.then(() => done())
.catch(e) => done(e));
Execute the following command:
.assertReply(activity => assert.equal(activity.type, ActivityTypes.Message))
An example is shown below:
await adapter.send('Hello')
.assertReply(activity => {
assert(activity.attachments.length === 1);
assert(activity.attachments[0].contentType === CardFactory.contentTypes.oauthCard);
// send a mock EventActivity back to the bot with the token
adapter.addUserToken(connectionName, activity.channelId, activity.recipient.id, token);
let eventActivity = createReply(activity);
eventActivity.type = ActivityTypes.Event;
let from = eventActivity.from;
eventActivity.from = eventActivity.recipient;
eventActivity.recipient = from;
eventActivity.name = "tokens/response";
eventActivity.value = {
connectionName,
token
};
adapter.send(eventActivity);
})
.assertReply('Logged in.');
Template for testing an Aura Bot dialog
import 'mocha';
import { TestAdapter, ConversationState, MemoryStorage, UserState } from 'botbuilder';
import { DialogTurnStatus, DialogSet } from 'botbuilder-dialogs';
import { AddBillDialog } from '../dialogs/collections/bill/add-bill';
import { AuraDataAccesor } from '../models';
describe('Dialog Test', function () {
this.timeout(5000);
/**
* The dialog template test.
*/
it('Bill dialog Test', (done) => {
/**
* Set the common data of Aura-Bot
*/
const testData = setCommonData();
/**
* Uset TestAdapter to launch the bot framework
*/
const adapter = new TestAdapter(async (turnContext) => {
const dc = await dialogs.createContext(turnContext);
const results = await dc.continueDialog();
if (results.status === DialogTurnStatus.empty) {
await dc.beginDialog('addBill');
}
await testData.conversationState.saveChanges(turnContext);
});
/**
* Create the dialog set and add dialogs to test
*/
const dialogs = new DialogSet(testData.auraDataAccesor.dialogState);
dialogs.add(new AddBillDialog(testData.auraDataAccesor));
/**
* Make a flow with sends and assert responses
*/
adapter.send('Hello')
.assertReply('Dime algo bonito')
.send('Algo bonito')
.assertReply('Gracias por el piropo')
.assertReply('Please enter your name. (1) Yes or (2) No')
.send('claro')
.assertReply('end Yes')
.then(() => done())
.catch((err) => done(err)); // <- send error to done
});
});
/**
* Common Data uses by the Aura Bot.
*/
function setCommonData() {
const memoryStorage = new MemoryStorage();
const conversationState = new ConversationState(memoryStorage);
const auraDataAccesor: AuraDataAccesor = {
conversationData: conversationState.createProperty('conversationData'),
userData: new UserState(memoryStorage).createProperty('userData'),
dialogState: conversationState.createProperty('DialogState')
};
return {
auraDataAccesor,
conversationState
};
}
Template for testing an Aura Bot middleware
/* eslint-disable @typescript-eslint/no-unused-vars */
/* eslint-disable require-await */
/* eslint-disable max-classes-per-file */
import { Activity, ActivityTypes, Middleware, TestAdapter, TurnContext } from 'botbuilder';
describe('Middleare Test', () => {
describe('ChanngeMessage middleware with TurnContext', () => {
class ChangeMessage implements Middleware {
/**
* On turn function of ChangeMessage.
*
* @param {TurnContext} context context
* @param {() => Promise<void>} next next function
*/
public async onTurn(context: TurnContext, next: () => Promise<void>): Promise<void> {
context.activity.text = 'response message';
await next();
}
}
it('Testing for channgeMessage middleware', async () => {
const adapter = new TestAdapter(async (context) => {
await context.sendActivity(context.activity.text);
});
adapter.use(new ChangeMessage());
await adapter.test('request message', 'response message');
});
});
describe('ChanngeTurnState middleware with TurnContext', () => {
class ChanngeTurnState implements Middleware {
/**
* On turn function of ChangeMessage.
*
* @param {TurnContext} context context
* @param {() => Promise<void>} next next function
*/
public async onTurn(context: TurnContext, next: () => Promise<void>): Promise<void> {
context.turnState.set('testInTurnState', context.activity.text);
await next();
}
}
it('Testing for channgeTurnState middleware', async () => {
const messageToTest = 'text in message';
const adapter = new TestAdapter(async (context) => {
const testInTurnState = context.turnState.get('testInTurnState');
await context.sendActivity(testInTurnState);
});
adapter.use(new ChanngeTurnState());
await adapter.test(messageToTest, messageToTest);
});
});
describe('ChanngeActivity middleware with TurnContext', () => {
class ChanngeActivity implements Middleware {
/**
* On turn function of ChangeMessage.
*
* @param {TurnContext} context context
* @param {() => Promise<void>} next next function
*/
public async onTurn(context: TurnContext, next: () => Promise<void>): Promise<void> {
!context.activity.serviceUrl && (context.activity.serviceUrl = 'https://change-service-url.com');
await next();
}
}
it('Testing for ChanngeActivity middleware', async () => {
const sendActivity: Partial<Activity> = {
type: ActivityTypes.Message,
text: 'text in message'
};
const adapter = new TestAdapter(async (context) => {
await context.sendActivity(context.activity);
}, { serviceUrl: undefined });
adapter.use(new ChanngeActivity());
await adapter.send(sendActivity).assertReply((activity) => {
expect(activity.serviceUrl).toBe('https://change-service-url.com');
});
});
});
describe('ErrorMiddleware with TurnContext', () => {
class ErrorMiddleware implements Middleware {
/**
* On turn function of ChangeMessage.
*
* @param {TurnContext} context context
*/
public async onTurn(context: TurnContext): Promise<void> {
throw new Error(context.activity.text);
}
}
it('Testing for ErrorMiddleware', async () => {
const adapter = new TestAdapter(async () => { /* */ });
adapter.use(new ErrorMiddleware());
const messageToTest = 'text in message';
// Method1: with send/catch
await adapter.send(messageToTest)
.catch((err) => {
expect(err.message).toBe(messageToTest);
});
// Method2: with try/catch
try {
await adapter.send(messageToTest);
// Fail test if above expression doesn't throw anything.
expect(true).toBe(false);
} catch (err) {
expect(err.message).toBe(messageToTest);
}
// Method3: with expect/toThrow
await expect(async () => {
await adapter.send(messageToTest);
}).rejects.toThrow(messageToTest);
});
});
});
Execution of tests to compare the performance between the joi library and the ajv library
This document details the execution of the tests to compare the performance between the joi library, currently used in aura-bot development, and the ajv library, which stands out for its performance orientation.
The tests performed are based on a simple channel object. This property is defined in the channelData object to communicate with the bot.
const Ajv = require('ajv');
const Benchmark = require('benchmark');
const Joi = require('joi');
const channelSchema = {
type: 'object',
additionalProperties: false,
properties: {
id: {
type: 'string',
format: 'uuid'
},
interfaceLanguage: {
type: 'string'
},
modality: {
type: 'string',
'enum': [
'audio',
'text',
'form'
]
}
},
required: [
'id',
'modality'
]
};
const ajvValidate = new Ajv().compile(channelSchema);
const joiSchema = Joi.object({
id: Joi.string().guid().required(),
interfaceLanguage: Joi.string(),
modality: Joi.string()
.valid('audio', 'text', 'form')
.required()
}).unknown(false);
const suite = new Benchmark.Suite();
const value = { id: '45494a5b-835a-4fff-a813-b3d2be529dbe', interfaceLanguage: 'es-ES', modality: 'text' };
suite.add('Joi', () => {
joiSchema.validate(value);
}).add('Ajv', () => {
ajvValidate(value);
});
suite.on('cycle', (event) => {
console.log(event.target.toString());
});
suite.on('complete', function () {
console.log('Fastest is ' + this.filter('fastest').map('name'));
});
suite.run({ async: true });
$ node src/index.js
Joi x 284,136 ops/sec ±25.59% (61 runs sampled)
Ajv x 5,087,995 ops/sec ±1.57% (74 runs sampled)
Fastest is Ajv
How to migrate a component unit tests from Jasmine to Jest
Delete the files: src/karma.conf.js and src/test.ts
In package.json remove the karma and jasmine packages from devDependencies.
For example:
"@types/jasmine": "~3.5.0",
"@types/jasminewd2": "~2.0.3",
"jasmine-core": "~3.6.0",
"jasmine-spec-reporter": "~5.0.0",
"karma": "~5.0.0",
"karma-chrome-launcher": "~3.1.0",
"karma-coverage-istanbul-reporter": "~3.0.2",
"karma-jasmine": "~4.0.0",
"karma-jasmine-html-reporter": "^1.5.0",
Uninstall these dependencies:
npm uninstall @types/jasmine @types/jasminewd2 jasmine-core jasmine-spec-reporter karma karma-chrome-launcher karma-coverage-istanbul-reporter karma-jasmine karma-jasmine-html-reporter
Execute the following command:
npm i -D jest @types/jest @angular-builders/jest
Add a file named jest.config.js (in the same folder as package.json)
More information: https://jestjs.io/es-ES/docs/configuration#opciones
module.exports = {
preset: 'jest-preset-angular',
setupFilesAfterEnv: ['<rootDir>/setup-jest.ts'],
collectCoverage: false,
coverageDirectory: 'coverage/testing-in-jest',
testMatch: [
'<rootDir>/src/**/*.spec.ts',
],
coverageReporters: ['lcovonly', 'text-summary', 'cobertura'],
coverageThreshold: {
global: {
statements: 10,
lines: 10,
branches: 10,
functions: 10
}
},
reporters: ['default'],
};
Add file test-config.helpers.ts
import { TestBed } from '@angular/core/testing';
type CompilerOptions = Partial<{
providers: any[];
useJit: boolean;
preserveWhitespaces: boolean;
}>;
export type ConfigureFn = (testBed: typeof TestBed) => void;
export const configureTests = (configure: ConfigureFn, compilerOptions: CompilerOptions = {}) => {
const compilerConfig: CompilerOptions = {
preserveWhitespaces: false,
...compilerOptions,
};
const configuredTestBed = TestBed.configureCompiler(compilerConfig);
configure(configuredTestBed);
return configuredTestBed.compileComponents().then(() => configuredTestBed);
};
Add file setup-jest.ts
More information: https://jestjs.io/es-ES/docs/configuration#setupfiles-array
import 'jest-preset-angular/setup-jest';
/* global mocks for jsdom */
const mock = () => {
let storage: { [key: string]: string } = {};
return {
getItem: (key: string) => (key in storage ? storage[key] : null),
setItem: (key: string, value: string) => (storage[key] = value || ''),
removeItem: (key: string) => delete storage[key],
clear: () => (storage = {}),
};
};
Object.defineProperty(window, 'localStorage', { value: mock() });
Object.defineProperty(window, 'sessionStorage', { value: mock() });
Object.defineProperty(window, 'getComputedStyle', {
value: () => ['-webkit-appearance'],
});
Object.defineProperty(document.body.style, 'transform', {
value: () => {
return {
enumerable: true,
configurable: true,
};
},
});
Replace jasmine with jest in the types array
Add module: commonjs to the compilerOptions
Remove test.ts from the files array
{
"extends": "./tsconfig.json",
"compilerOptions": {
"outDir": "./out-tsc/spec",
"types": [
"jest",
],
"module": "commonjs",
"emitDecoratorMetadata": true,
"allowJs": true
},
"files": [
"src/polyfills.ts"
],
"include": [
"src/**/*.spec.ts",
"src/**/*.d.ts"
]
}
Change the projects[your-project].architect.test section, so it looks like this:
https://www.npmjs.com/package/@angular-builders/jest
"test": {
"builder": "@angular-builders/jest:run",
"options": {}
},
Execute the following command:
npx jest-codemods -f ./src/app/*.spec.ts
? Which parser do you want to use? TypeScript
? Which test library would you like to migrate from? Jasmine: globals
? Are you using the global object for assertions (i.e. without requiring them) No, I use import/require statements for my current assertion libr
ary
? Will you be using Jest on Node.js as your test runner? Yes, use the globals provided by Jest (recommended)
Executing command: jscodeshift -t /Users/moasl/.npm/_npx/13840/lib/node_modules/jest-codemods/dist/transformers/jasmine-globals.js ./src/app/app.component.spec.ts --ignore-pattern node_modules --parser ts --extensions=ts
Executing command: jscodeshift -t .npm/_npx/14583/lib/node_modules/jest-codemods/dist/transformers/jasmine-globals.js ./src/app/app.component.spec.ts --ignore-pattern node_modules --parser ts --extensions=ts
Processing 1 files...
Spawning 1 workers...
Sending 1 files to free worker...
All done.
Results:
0 errors
0 unmodified
0 skipped
1 ok
Time elapsed: 1.286seconds
After all of these changes, it is recommended to delete your node_modules folder and run npm install again.
At this stages, you may have to change some test manually, for example, if jest does not have toBeTrue or toBeFalse.
Interesting library: jest-dom
If you lose some import or injection in your component test, you will get this error type:
(node:71567) UnhandledPromiseRejectionWarning: TypeError: Converting circular structure to JSON
at JSON.stringify (<anonymous>)
at process.target._send (internal/child_process.js:735:23)
at process.target.send (internal/child_process.js:634:19)
at reportSuccess (/Users/moasl/PROYECTOS/AURA/aura-channels-factory/packages/aura-channels-libraries/node_modules/jest-runner/node_modules/jest-worker/build/workers/processChild.js:67:11)
(node:71567) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:71567) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
If you run the test with --detectOpenHandles option, you will get more information to solve the error:
$ ng test --detectOpenHandles aura-channels-views
.....
FAIL projects/aura-channels-views/src/lib/components/alfred/alfred.component.spec.ts
● AlfredComponentComponent › should render title
Found the synthetic property @alfredAnimationState. Please include either "BrowserAnimationsModule" or "NoopAnimationsModule" in your application.
at checkNoSyntheticProp (../packages/platform-browser/src/dom/dom_renderer.ts:269:11)
....
More information: Jest documentation –detectOpenHandles.