All About Acceldata Data Observability Cloud v2.7.0

Acceldata Data Observability Cloud v2.7.0 Introduces Significant Databricks and Snowflake Observability Enhancements and Supports New Data Sources

Recently, we released the latest version of Acceldata Data Observability Cloud (ADOC). Version 2.7.0 is now generally available, and it provides enhancements to compute, data reliability, monitoring, alerting, and Databricks and Snowflake navigation and display capabilities.

The following video offers a snapshot of the enhancements, and more details on new capabilities and features are explained below:

Data Source and Data Plane

Check out Acceldata’s Hive integration and new data sources:

• Enhanced Hive Integration: ADOC has been upgraded to seamlessly integrate with Hive, specifically the Enterprise version.
• Expanded Data Sources: ADOC now offers integration with additional data sources, including Teradata, Azure MSSQL, and Oracle.

Compute Enhancements

In the 2.7.0 release, you’ll see a variety of major enhancements that will provide better experiences for Databricks Query Studio, Snowflake Query Studio, cost center and cloud cost control features, warehouse sizing recommendations, and others.

• Databricks Query Studio: ADOC now offers a Databricks Query Studio experience, akin to Snowflake Query Studio. This exciting addition enables you to validate and monitor SQL queries executed by Spark apps within a notebook.
• Enhanced Databricks Guardrails: In the Databricks Compute tab, a new Guardrail page has been introduced, presenting visualizations for clusters that have been terminated due to inactivity.
• Advanced Cost Center Management with Tag Pairs: ADOC now offers the ability to create chargeback-based cost centers using Tag Pairs. This feature enables you to accurately track charges for the entire account or specific subsets of resources by utilizing multiple key values.
• Streamlined Databricks Navigation: ADOC's Databricks navigation now facilitates efficient correlation of data across jobs and clusters.
• Enhanced Snowflake Query Studio: The Snowflake Query Studio now displays metrics like #Avg Schedule Frequency, warehouse names on hover, easier navigation to the Query Details page, and additional information in the heatmap chart, among other enhancements. 
• Projected Spend Budget Alerts: ADOC introduces the capability to receive alerts when your projected spend, calculated from current utilization, surpasses a predefined budget threshold.
• Snowflake Warehouse Size Selection: You’ll now have the option to choose a Snowflake warehouse size based on intelligent recommendations which streamlines your decision-making process.

Data Reliability

We’ve made significant enhancements to the data reliability capabilities of the Acceldata platform. These include better data cadence support, importing/exporting policies, service monitoring, policy execution, and others.

• UDT Repo Import Export Capability: ADOC introduces a streamlined flow for importing or exporting policies that utilize external User-Defined Types (UDTs). 
• Customizable UDF Metadata for Failed Rows: ADOC now enables you to customize the feedback logic to UDT via metadata for various checks, so you can receive detailed feedback for failed records pertaining to Data Quality Policies.
• Expanded Data Cadence Support: ADOC now offers data cadence functionality for Files and has extended its support to include MySQL, PostgreSQL, BigQuery, S3, ADLS, Azure Data Lake, GCS, and HDFS as data sources.
• Composite Policy Application: You’ll now be able to apply individual rules to specific subsets of data within an asset through the Composite Policy feature.
• Selective Policy Execution: ADOC now grants you the ability to selectively execute policies on specific subsets of rules or data.
• Enhanced Service Monitoring in Dataplanes: ADOC now provides real-time visibility into the ongoing logs of the analysis service executed on the dataplane side. Additionally, you can view the running logs of crawlers on data sources.

Monitors and Alerts

This version has enhanced monitoring functionality, and new recommendation and alerting capabilities.

• Expanded Notification Channels: ADOC now provides support for configuring Microsoft Teams and ServiceNow as notification channels, offering more options for seamless communication.
• Databricks Workspace Cost Monitor: In this release, we’ve provided Databricks Workspace Cost monitor, which tracks the cost at the workspace level for the current month.
• Enhanced Recommendations: ADOC now allows you to enable or disable recommendations directly from the Monitors interface, providing greater control over the recommendations displayed. Additionally, recommendations will automatically expire when they are no longer relevant.
• Alerts for Assignee Action: ADOC now includes a new email notification template that ensures users assigned to an alert receive timely email notifications.

Learn more with a tour of the Acceldata Data Observability Cloud.

‍

‍

Photo by Vincentiu Solomon on Unsplash