Manage Aura metrics

Introduction

Metrics provide a measurement of certain data that represent a specific aspect of the monitored system at a point in time and offer an aggregated view over the system. They are useful to visualize long-term trends and alerts on log data.

Each Aura component is in charge of publishing its own metrics, which are typically generated at fixed-time intervals from aggregated logs.

Once generated, Aura metrics are pooled by Prometheus, which is in charge of gathering and exposing them.

Grafana is the most suitable tool to represent metrics through different dashboards. Each component counts on a Grafana dashboard to show its current behavior and there is a single dashboard for an Aura overview.

If you think a new metric could useful, please contact the Aura Platform Team, so it can be officially included as part of the platform.

The aim of this section is to explain both how Aura metrics work and all the metrics stored by each component.

⚠️ Saved dashboards, visualizations and queries are not guaranteed to be kept between upgrades because all the stack, including ElasticSearch and Grafana can be upgraded to newer versions.

Prometheus

Aura metrics system is based on Prometheus, a Cloud Native Computing Foundation project that works as systems and services monitoring system. Prometheus collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts when specified conditions are observed.

prom-client is being used to implement prometheus functionality in Node.js.

Prometheus service pools every component to get the metrics generated during the last time period. Every component counts on a private endpoint (not accessible from Internet) called /metrics where Prometheus requests the metrics.

Currently, the metric types used in this component are:

• Summary: similar to histogram metrics, it includes samples observations (such as request durations and response sizes). While it also provides a total count of observations and a sum of all observed values, it calculates configurable quantiles over a sliding time window.

• Counter: cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart. For example, you can use a counter to represent the number of requests served, tasks completed, or errors.

• Gauge: similar to Counter, but it represents a single numerical value that can arbitrarily go up and down.

Prometheus-es-exporter

Working with Prometheus, we can create metrics using queries to ElasticSearch indexes (as well as create alarms, dashboard, etc) using prometheus-es-exporter.

This component is not deployed by default, but it can be enabled changing the variable prometheus_es_exporter_enabled to true in you config.yml file. (In Brazil, it is set to true by default). Access here the guidelines to enable prometheus-es-exporter component.

To config your own metrics from queries, write the new section, as in the following example, in your config.yml.

prometheus_es_exporter:
  query_blocks:
    ob:
      - name: "query_ob_br"
        QueryIntervalSecs: "60"
        QueryJson: '{"size":0,"query":{"bool":{"must":[],"filter":[{"bool":{"filter":[{"bool":{"should":[{"match_phrase":{"msg":"[AzureEventHub] emit"}}],"minimum_should_match":1}},{"bool":{"should":[{"match_phrase":{"kubernetes.labels.app":"aura-bot"}}],"minimum_should_match":1}}]}},{"range":{"@timestamp":{"gte":"now-1m","lte":"now"}}}]}}}'
        QueryIndices: "aurak8s-service-*"

Where:

• name: Mandatory. Name of the query. It must start with query_*
• QueryIntervalSecs: Optional. It indicates how often to run queries in seconds. By default, 60.
• QueryJson: Mandatory. The search query to run.
• QueryIndices: Optional. Indices to run the query on. Any way of specifying indices supported by your ElasticSearch version can be used. By default, _all. Although this field is optional, it is highly recommended to delimit the search query.

Aura components metrics

The main Aura components can generate their own metrics.

Select your intended component in the left menu and access to its details.