Tags:

• aura-platform

Categories:

• monitoring

Aura Analytics 1.1.

Introduction

This document contains a description of a joint dataflow between LCDO OB teams and Aura Global Team for processing Aura log files created in production environment (i.e., coming from actual Aura users) in order to create PPDs (Privacy-Preserving Datasets). All this process is known as Active Listening.

The dataflow produces as a result, among other elements, an analytics component, named as Aura Analytics Dashboard, that can be used to gather statistics on the production system and to analyze user’s behavior. The latest version 1.1 of this dashboard is described in the current document.

The main objectives of the unified dataflow are:

• Consolidate the processing of Aura logs into a framework.
• Provide LCDOs and Aura Global Team with a unified common source for analytics, in a privacy-preserving way.
• Enable extensibility of the dataflow.

In this framework, the current documents provide:  

• The available data analytics versions
• Prerequisites and recommended tools for its use
• Overall description of Aura Analytics architecture
• Description of the OB Analytics subsystem that enables the management by OBs
• Current Aura Analytics data model

The target audience of this document includes the following roles both in LCDO Teams and Aura Global Team:

• Data Scientists and Product teams, that wish to access Aura logs information and perform analytics on them.
• Operation teams, for the architectural description and the requirements on OB environments.

Aura Analytics versions

Release 1.0.

The first release 1.0. sets up the basic paths, deploys the PPD infrastructure and produce:

• Version 1.0. of the OB Analytics system, which includes the OB Dashboard.
• The first version of pre-processed datasets (clean PPDs) for training and analytics at Aura Global.

As mentioned, this version enables OBs to go further by:

• Enhancing the OB Dashboard with new visualizations, as they seem fit (given that panels and dashboards can be exported and imported, it is possible to share new ones across all OBs, as they are developed).

• Processing the PPD files as desired (they are standard CSV files, which can be ingested in alternative platforms if desired). Restrictions on them are softer than on the original logs due to the anonymization process they have been subjected to, although they are still subjected to management precautions (a code of conduct is being prepared for that).

Release 1.1.

Version 1.1. introduces the following changes:

• The table of data has been enlarged with these new fields: AURA_ID, STATUS_CD, sesId, sesSize, sesDuration.
• An expanded list of test users is used, so that the userType column contains more identifications.
• The code for data ingestion into a local Kibana, which previously consisted on a single Python script, has been turned into a full Python package to be installed, due to its increasing complexity.

Prerequisites and recommended tools

The prerequisites for the use of version 1.1. of Aura Analytics Dashboard are set below:

• Aura Platform version:

• Recommended operating system:Ubuntu 18.04 system

• Recommended tool for data visualization: ELK stack