1. IT-Security >
  2. Programmierung >
  3. Time series analysis in Azure Data Explorer


ArabicEnglishFrenchGermanGreekItalianJapaneseKoreanPersianPolishPortugueseRussianSpanishTurkishVietnamese

Time series analysis in Azure Data Explorer

RSS Kategorie Pfeil Programmierung vom | Quelle: azure.microsoft.com Direktlink öffnen

Azure Data Explorer (ADX) is a lightning fast service optimized for data exploration. It supplies users with instant visibility into very large raw datasets in near real-time to analyze performance, identify trends and anomalies, and diagnose problems.

ADX performs an on-going collection of telemetry data from cloud services or IoT devices. This data can then be analyzed for various insights such as monitoring service health, physical production processes, and usage trends. The analysis can be performed on sets of time series for selected metrics to find a deviation in the pattern of the metrics relative to their typical baseline patterns.

ADX contains native support for creation, manipulation, and analysis of time series. It empowers us to create and analyze thousands of time series in seconds and enable near real-time monitoring solutions and workflows. In this blog post, we are going to describe the basics of time series analysis in Azure Data Explorer.

Time series capabilities

The first step for time series analysis is to partition and transform the original telemetry table to a set of time series using the make-series operator. Using various functions, ADX then offers the following capabilities for time series analysis:

  • Filtering – Used for noise reduction, smoothing, change detection, and pattern matching.
  • Regression analysis – Used for trend change detection in streamed data.
  • Seasonality detection – Used to automatically detect or validate seasonal or periodic patterns in each time series.
  • Element-wise functions – Used to perform arithmetic and logical operations between two time series.

The complete set of functions for time series analysis can be found in the machine learning and time series analysis section of our documentation.

Example of a time series analysis query

The following query uses series_periods_detect and series_fit_line for time series analysis and discovery of periodic patterns and decreasing trends:

let min_t = toscalar(demo_many_series1 | summarize min(TIMESTAMP)); 
let max_t = toscalar(demo_many_series1 | summarize max(TIMESTAMP)); 
demo_many_series1
| make-series reads=avg(DataRead) on TIMESTAMP in range(min_t, max_t, 1h) by Loc, Op, DB
| where series_partial_sf(reads, 0) == false
| extend (p, ps)=series_periods_detect(reads, 0, 24, 1)
| mvexpand p to typeof(double), ps to typeof(double)
| where ps > 0.7
| extend series_fit_line(reads)
| top 2 by series_fit_line_reads_slope asc
| render timechart with(title='Top 2 Periodic Decreasing Web Service Traffic (out of 18,339 instances)')

In this query, Azure Data Explorer analyzes 18,339 time series of web service traffic and extracts those with a periodic pattern. Out of this subset, ADX looks for those instances with a decreasing trend. This entire processing takes only about one minute.

Top 2 periodic decreasing web service traffic chart

Additional information

For a step-by-step walkthrough of time series analysis capabilities read “Time series analysis in Azure Data Explorer.” This topic shows you how to create a large set of time series followed by typical series processing to detect anomalous patterns in specific time series instances.

Our time series functions and capabilities continue to grow. New powerful black box functions for anomaly detection and forecasting are being released. Stay tuned for my next blog to discover additional functions and analysis methods.

To find out more about Azure Data Explorer you can:

...

Webseite öffnen Komplette Webseite öffnen

Newsbewertung

Kommentiere zu Time series analysis in Azure Data Explorer






Ähnliche Beiträge

  • 1. Azure.Source – Volume 61 vom 1611.31 Punkte ic_school_black_18dp
    Microsoft Connect(); 2018 On Tuesday, December 4th, Microsoft Connect(); 2018 provided a full day of developer-focused content—including updates on Azure and Visual Studio, keynotes, demos, and real-time coding with experts. Scott Guthrie’s keyno
  • 2. Azure.Source – Volume 58 vom 1384.91 Punkte ic_school_black_18dp
    Now in preview Update 18.11 for Azure Sphere in public preview This is an update to the Azure Sphere Operating System, Azure Sphere Security Service, and Visual Studio development environment. This release includes substantial investments in our se
  • 3. Azure.Source – Volume 62 vom 1159.66 Punkte ic_school_black_18dp
    KubeCon North America 2018 KubeCon North America 2018: Serverless Kubernetes and community led innovation! Brendan Burns, Distinguished Engineer in Microsoft Azure and co-founder of the Kubernetes project, provides a welcome to KubeCon North America 2018, which took p
  • 4. Azure.Source – Volume 60 vom 1045.45 Punkte ic_school_black_18dp
    Now in preview Simplifying security for serverless and web apps with Azure Functions and App Service New security features for Azure App Service and Azure Functions reduce the amount of code you need to work with identities and secrets under management. Key Vault re
  • 5. Azure.Source – Volume 63 vom 853.86 Punkte ic_school_black_18dp
    Now in preview Transparent Data Encryption (TDE) with customer managed keys for Managed Instance Announces the public preview of Transparent Data Encryption (TDE) with Bring Your Own Key (BYOK) support for Microsoft Azure SQL Database Managed Instance. Azure SQL Database M
  • 6. Azure.Source – Volume 63 vom 853.86 Punkte ic_school_black_18dp
    Now in preview Transparent Data Encryption (TDE) with customer managed keys for Managed Instance Announces the public preview of Transparent Data Encryption (TDE) with Bring Your Own Key (BYOK) support for Microsoft Azure SQL Database Managed Instance. Azure SQL Database M
  • 7. Azure Marketplace new offers – Volume 26 vom 690.55 Punkte ic_school_black_18dp
    We continue to expand the Azure Marketplace ecosystem. During September and October, 149 new consulting offers successfully met the onboarding criteria and went live. See details of the new offers below: Consulting Services   1-Day Big Da
  • 8. Azure Marketplace new offers – Volume 26 vom 690.55 Punkte ic_school_black_18dp
    We continue to expand the Azure Marketplace ecosystem. During September and October, 149 new consulting offers successfully met the onboarding criteria and went live. See details of the new offers below: Consulting Services   1-Day Big Da
  • 9. Microsoft Azure portal May 2019 update vom 678.62 Punkte ic_school_black_18dp
    This month is packed with updates on the Azure portal, including enhancements to the user experience, resource configuration, management tools and more. Sign in to the Azure portal now and see for yourself everything that’s new. Download the Azu
  • 10. Microsoft Azure portal May 2019 update vom 678.62 Punkte ic_school_black_18dp
    This month is packed with updates on the Azure portal, including enhancements to the user experience, resource configuration, management tools and more. Sign in to the Azure portal now and see for yourself everything that’s new. Download the Azu
  • 11. Azure.Source – Volume 59 vom 639.58 Punkte ic_school_black_18dp
    Now in preview Simplifying confidential computing: Azure IoT Edge security with enclaves – Public preview Azure IoT Edge is a fully managed service that delivers cloud intelligence locally by deploying Azure AI, Azure services, and custom logic d
  • 12. Azure Stack IaaS – part ten vom 613.94 Punkte ic_school_black_18dp
    This blog is co-authored by Andrew Westgarth, Senior Program Manager, Azure Stack  Journey to PaaS One of the best things about running your VMs in Azure or Azure Stack is you can begin to modernize around your virtual machines (VMs) by taking adv