Detect Overview

Immuta Detect is a tool that monitors your data environment and provides analytic dashboards in the Immuta UI based on your data use. These dashboards offer visualizations of audit events, including user queries and when classification is enabled, the sensitivity of those queries, data sources, and columns. It can work within your current Immuta integration.

Immuta Detect continually monitors your data environment to help answer questions about your most active data users, the most accessed data, and the events happening within your data environment. Detect can provide even more value with classification enabled to answer questions about the sensitive data accessed by your users and the tables that contain sensitive data. Because of this information, your organization can do the following:

• Meet compliance requirements more effectively
• Quickly decide what data access is allowed for what purposes
• Reduce the effort and time to respond to auditors about data access in your company
• Reduce the effort of classifying data within the scope of security or regulatory compliance frameworks

Detect with classification

Benefits

• Dashboards with data activity patterns for data sources and users
• Dynamic query sensitivity on joined tables calculate sensitivity based on the columns queried and their toxicity when joined
• Dashboards to help users find the most recently accessed data sources and active columns

Architecture

Immuta Detect uses several features of the Immuta platform to create user-friendly dashboards that are always available in the UI and do not need to be generated like Immuta reports. These dashboards are created by combining Snowflake audit events and the sensitivity of your data. Audit information and events are gathered from the Snowflake ACCOUNT_USAGE views into Immuta Detect. Additionally, Immuta Detect calculates the sensitivity of your data using the Immuta Data Security Framework, which finds sensitive data on a column-by-column basis using tags applied by SDD. Once Immuta does this work behind the scenes, users with the AUDIT permission will see dashboards that show the sensitive data within your organization’s data environment and what users are accessing that data.

Sensitivity

With classification enabled, Immuta Detect qualifies both columns and queries as the following sensitivity types in the dashboards:

• Highly sensitive: Includes data that can cause severe harm or loss with inappropriate access or misuse.
• Sensitive: Includes personal data and data that could cause harm or loss with inappropriate access or misuse.
• Non-sensitive: Includes publicly available information or data that would not typically cause harm or loss if disclosed.
• Indeterminate: The sensitivity of the data is unknown. Immuta deems sensitivity indeterminate because of an error in the query or because the sensitive data discovery (SDD) or classification has not completed processing at the time the query was run.

How does Immuta determine column sensitivity?

Column sensitivity is determined by the classification tags applied to the columns by the frameworks. The classification tags contain sensitivity metadata.

How does Immuta determine query sensitivity?

For queries that read from a single table, query sensitivity is determined by the column with the highest sensitivity in the query

For a query that joins tables, Immuta uses the same classification rules applied to tables and applies those rules to columns of the query. Immuta applies a new set of classification tags to the query columns and calculates sensitivity for the query event in the audit record. These query classification tags are not included on the tables' data dictionary.

Detect without classification

Benefits

• Quicker and easier onboarding experience
• Dashboards with data activity patterns for data sources and users
• Dashboards to help users find the most recently accessed data sources and active columns

Architecture

Immuta Detect uses several features of the Immuta platform to create user-friendly dashboards that are always available in the UI and do not need to be generated like Immuta reports. These dashboards are created from audit information and events gathered from Snowflake, Databricks Spark, and Databricks Unity Catalog into Immuta Detect. Users with the AUDIT permission will see dashboards that show the data events within your organization’s data environment and what users are accessing that data.

Limitations

While you can migrate between classification on and off, your audit data will always have the classification from when it is ingested. Once you enable classification, all your data from that day forward will have the classification tags, but any audit data from before will remain unclassified. Conversely, if you decide to turn off classification, the classification tags will stay on the past audit data, but moving forward the data will be unclassified.

Detect features

User, data, and audit dashboards

Immuta Detect provides at-a-glance dashboards to monitor change in user activity, data access, and security posture.

Historical audit ingestion

The Immuta Detect historical audit ingestion is a feature that pulls in past query audit information from Snowflake. You can ingest up to a year's worth of historical audit information, but the time is configurable. Once you confirm classification tags are working well against your organization’s data, security, and compliance needs, you and your Immuta representative may enable the historical audit collection of Snowflake query events. The tags applied at the time of the audit will be applied to all of the historical queries that are synced. This historical audit data cannot be changed after, so ensure that the tags applied at the time are accurate.

If you are not interested in the sensitivity of your data, you can ingest the historical audit information without any classification tags. However, this action will make the historical information permanently unclassified. If you decide to enable classification and start viewing sensitivity in the future, it will only be available from that point forward; the historical information cannot be updated.

Limitations

• Organizations with multiple integrations should configure and run the historical audit for them at the same time. If you run historical audit for just one of your integrations, then try to run it for another separately, there will be a gap of historical data for the months you pulled in for your first integration.

Unified audit model

Immuta’s unified audit model (UAM) and export features allow you to export your audit logs to S3 to monitor services and data pipelines and process structured audit logs in your log data processors and tools. This capability fosters convenient integrations with log monitoring services and data pipelines.

With UAM, you can specify an S3 bucket destination where Immuta will periodically export audit logs. The events captured are only events relevant to user and system actions that affect Immuta or the integrated data platforms, such as creating data sources and running queries.

Limitations

• As part of the preview, Immuta would like to assess and validate the SDD, Frameworks, and Immuta Detect features with you as each organization has different data, security, and compliance needs. Your Immuta representative will work with you during the preview to review tags and classification of data sources.

• The Immuta Detect dashboard shows near real-time events for Immuta events, such as login, policy changes, and data platform policy changes. Query events are ingested from Snowflake and Databricks once a day, but you can manually trigger an immediate query retrieval by using the ↻Native Query Audit button on the Audit page. You can work with your Immuta representative to configure a query history retrieval more frequent than 24 hours.

  The most recent query history that is available to Immuta Detect depends on the underlying data platform latency. For example, there is 15 minutes of latency between an executed query and recording the event on the Snowflake data platform side.

• Immuta will only pull audit information from Snowflake and Databricks Spark for data sources and users registered in Immuta; Databricks Unity Catalog will pull in all users and tables.

• The historical audit data cannot be changed after it is synced to Immuta, so ensure that the tags applied at the time are accurate.

• Detect with Databricks Spark:

  • This integration does not support classification to determine query sensitivity at this time.
  • This integration does not support historical audit ingestion. However, UAM has been enabled for most SaaS accounts since January 2023, so this UAM information will populate into the user and data source activity dashboards when Detect is enabled in Immuta.

• Detect with Databricks Unity Catalog:

  • This integration does not support classification to determine query sensitivity at this time.
  • This integration does not support historical audit ingestion.