An anomaly is any abnormal behavior or deviation from the expected behavior or pattern in a data set. Anomalies can indicate business-critical incidents such as a payment gateway glitch, or new consumer behavior.

Read more

There are three main categories of anomalies:

1. Point anomalies: Here, a single or an independent instance of data is anomalous in an individual-data series.
2. Contextual anomalies: Here, the data instance is anomalous in some predefined context in time-series data.
3. Collective anomalies: Here, a group of data instances exhibits anomalous behavior in comparison to other groups in various data series.

Anomaly detection is a technique to identify unexpected patterns or behavior in data sets. Anomaly detection helps in RCA (Root Cause Analysis) of unexpected events or business incidents.

Read more

There are broadly three techniques for detecting anomalies:

1. Supervised anomaly detection: In this scenario, the model is trained on the labeled data, and the trained model will predict the unseen data. It is used to predict known anomalies such as previously identified fraud.
2. Semi-supervised anomaly detection: The algorithm in this case only has a set of “normal” data points for reference – any data points that are outside this reference range are classified as anomalous.
3. Unsupervised anomaly detection: In this case, there are no labels for the data to train upon. It is a fully automated anomaly detection technique that relies on powerful algorithms to identify anomalies from unlabeled data.

Read more

As the name suggests, time series anomaly detection deals with time-series data, i.e. data that changes with time. E.g. In our personal computer, CPU usage, Network Usage, Memory Usage, etc. changes with time. You can plot it as a line graph with time on the X-axis. It clearly shows the behavior of the metric over time. Time series anomaly detection is the process of applying AI/ML algorithms on the time series data to understand the behavior of the data and then spot any abnormal behavior such as a sudden spike or dip.

Read more

One of the most effective ways to detect anomalies in time series data is to leverage deep learning architecture. The system first learns from historical data to understand and establish normal behavior, and then dynamically monitors real-time data to detect abnormal patterns. The algorithm creates upper and lower limits at a specified confidence level. Whenever an actual perceived value falls beyond the predicted normal range, anomalies are marked – and scored based on their magnitude of deviation.

Read more

Irrespective of the vertical, if you take the example of any business, they will have various metrics or KPIs to monitor the performance of their business, to get a sense of how good or bad the performance is. The data for these metrics will follow a time-series pattern, which can be used for Time Series Anomaly Detection. Since all the metrics follow time, we can use the time as a common feature to tie various similar behaving metrics together by applying correlation which can help the business to focus on the incident with the list of all impacted metrics. It will be very helpful for finding out the root cause in case of an incident.

An anomaly detection system can be deployed either on-premise or in the cloud. While deploying an anomaly detection system, companies must factor in the alert frequency, the requirement for scalability, the need for a supervised or unsupervised solution, and the required system integrations.

Read more

An automated anomaly detection system adds value to your business in the following ways:

• It connects to all the different data sources where the metrics are generated.
• It uses proprietary unsupervised algorithms which are created for understanding and learns all the trend, periodicity, and seasonality in the data.
• In real-time, it can identify all emerging issues autonomously which deviates from normal behavior.
• Various anomalies can be grouped using correlation which will be useful for doing root cause analysis in case of any incident.
• It provides Smart Insights that can be consumed by business users without any dependency on data analysts.
• It reduces significant efforts in configuration and development so that the Network Company can focus on the results and take meaningful decisions.

Read more

CrunchMetrics is an AI and ML-powered software that learns from historical data to understand and establish what is ‘normal’ behavior, and then dynamically monitors data streams to single out ‘abnormal’ patterns or anomalies. The software then contextually analyses these anomalies and correlates them with different data signals in the enterprise to identify if it is a business-critical incident or not. The system flags all the identified incidents in real-time, alerting stakeholders instantly to issues that need action so that corrective measures can be implemented to minimize the impact of the anomaly.

Read more