AWS PrivateLink for Databricks

AWS PrivateLink provides private connectivity from the Immuta SaaS platform to customer-managed Databricks accounts hosted on AWS. It ensures that all traffic to the configured endpoints only traverses private networks.

This front-end PrivateLink connection allows users to connect to the Databricks web application, REST API, and Databricks Connect API over a VPC interface endpoint. For details about AWS PrivateLink in Databricks and the network flow in a typical implementation, explore the Databricks documentation.

This feature is supported in most regions across Immuta's Global Segments (NA, EU, and AP); please contact your Immuta account manager if you have questions about availability.

Requirements

Databricks

Ensure that your accounts meet the following requirements:

• Your Databricks account is on the E2 version of the platform.
• Your Databricks account is on the Enterprise pricing tier.
• You have your Databricks account ID from the account console.
• You have an Immuta SaaS tenant.
• AWS PrivateLink for Databricks has been enabled.

Databricks workspace

Ensure that your workspace meets the following requirements:

• Your workspace must be in an AWS region that supports the E2 version of the platform.

  Info

  In the us-west-1 region, Databricks does not support PrivateLink even for workspaces on the E2 version of the platform.

• Your Databricks workspace must use Customer-managed VPC to add any PrivateLink connection.

• Your workspaces must be configured with private_access_settings objects.

Enablement

Contact your Databricks representative to enable AWS PrivateLink on your account.

Configure Databricks with AWS PrivateLink

1. Contact your Immuta representative, who will provide the PrivateLink endpoint IDs to register with your accounts. You will need to provide the following information:

   • AWS region
   • Databricks hostname
   • Private access level (either ACCOUNT or ENDPOINT)

2. Register the PrivateLink endpoint IDs.

   • If the private_access_level on your private_access_settings object is set to ACCOUNT, you should not need to do anything else beyond the registration.

   • If the private_access_level on your private_access_settings object is set to ENDPOINT, you will need to add it to the allowed_vpc_endpoint_ids list inside your private_access_settings object in Databricks. For example,

     "private_access_settings_name": "immuta-access",
     "region": "us-east-1",
     "public_access_enabled": false,
     "private_access_level": "ENDPOINT",
     "allowed_vpc_endpoint_ids": [
          "vpce-0fe5b17a0707d6fa5"
     ]
     

3. Configure the Databricks integration using your Databricks workspace URL.

4. Register your tables as Immuta data sources.

   Info

   Note that the privatelink-account-url from the JSON object in step one will be the Server when registering data sources.