Skip to main content
Databricks is a powerful data analytics and lakehouse platform that enables organizations to unify data engineering, data science, and business analytics at scale. Integrate Databricks with Superblocks to build internal tools and workflows that query, analyze, and update your Databricks data, empowering your team to automate processes, gain insights, and accelerate decision-making.

​
Setting up Databricks

If you’re using Superblocks Cloud, you may need to allowlist Superblocks IP addresses to connect to your database. Learn how to configure this in Databrick’s guide on Configuring IP access lists. Alternatively, use the Superblocks Hybrid or Cloud-Prem architectures to keep your data within your VPC.

​
Configure authentication

To access your Databricks warehouse, you’ll need to authenticate using a Databricks account. Superblocks provides several different ways to authenticate. See the Databricks documentation below for how to configure your preferred authentication method.
MethodDescription
Personal access token (PAT)Use a short or long-lived access tokens for a user or service principal.
Machine-to-machine OAuthConfigure oauth client credentials for a service principal. Superblocks will exchange the client credentials with Databricks to retrieve a short-lived OAuth token.
OAuth token federationUse OAuth tokens issued by your identity provider when users log in to Superblocks to authenticate with Databricks using the authenticated user’s permissions

​
Add integration

  1. In the web app, navigate to the Integrations page
  2. Click Add integration
  3. Search for Databricks and select it from the list of available integrations
  4. Name the integration
  5. Fill out the integration configuration as follows:
FieldRequiredDescription
Host✓Databricks instance host name
Port✓Port to use when connecting to your warehouse
HTTP Path✓HTTP path either to a DBSQL endpoint (e.g. /sql/1.0/endpoints/1234567890abcdef) or to a DBR interactive cluster (e.g. /sql/protocolv1/o/1234567890123456/1234-123456-slid123)
Default catalog✗An optional initial catalog to use
Default schema✗An optional initial schema to use
Access token✓Databricks personal access token
  1. Optionally, add more configurations to set credentials for different environments
  2. Click Test Connection to check that Superblocks can connect to the data source
  3. Click Create
Databricks connected!
Now you can use Databricks in any Application, Workflow, or Scheduled Job.

​
Using Databricks in APIs

Once your Databricks integration is created, you can start using Databricks by writing SQL in Superblocks APIs.

​
Troubleshooting

If you run into issues, first see our guide on Troubleshooting Database Integrations. There are also several common errors you may see when using Databricks. The table below includes error messages, why they happen, and how to address them.
Error messageWhy it’s happening & Resolution
IntegrationTimeoutError: Failed to connect to warehouse. Connection timed out after <N>msReason
Databricks warehouses can be configured to automatically hibernate after a period of inactivity. This timeout usually occurs when the warehouse is hibernating and did not restart within the timeout threshold of the agent.

Resolution
Retry the API/test connection after the warehouse has restarted, or increase the automatic hibernation time for your Databricks warehouse.
IntegrationOAuthError: OAuth2 - "On-Behalf-Of Token Exchange" could not find identity provider tokenReason
You’ve selected Login identity provider as the subject token source when using OAuth token federation, but you are not currently logged in to Superblocks using an OIDC-based Identity Provider.

Resolution
Reach out to [email protected] for assistance configuring SSO or migrating your SSO configuration to OIDC.
Failed to process token: TOKEN_EXPIREDReason
The access token issued to Superblocks when you logged in, or the static token you’ve provided has expired.

Resolution
If using Logged in identity provider, log out of and back into Superblocks. If using a static token, obtain a new federated JWT from your identity provider.
Failed to process token: TOKEN_INVALID (Ensure a valid federation policy has been configured)Reason
Your Databricks account either does not have a federation policy configured, or the subject_token being sent to Databricks by Superblocks does not satisfy the policy. This can happen if the token is not a valid JWT, or has a different aud or iss than configured in the Databricks federation policy.

Resolution
Make sure the aud and iss configured in Databricks are the same aud and iss your IdP uses when issuing tokens to Superblocks. Make sure the Databricks federation policy points to a valid JWKS URI. By default, Databricks uses the URI provided at <issuer-url>/.well-known/openid-configuration. You may need to change this if your IdP uses a non-default authorization server or does not support a /.well-known discovery URL.
If you are encountering an error that’s not listed, or the provided steps are insufficient to resolve the error, please contact us at [email protected]