--- title: "Connect to Databricks" slug: "connect-to-databricks" description: "Connect Validatar to a Databricks SQL warehouse with the Databricks Driver connection type and a service principal. One data source per Unity Catalog catalog." status: "new" updated: 2026-11-20T21:49:49Z published: 2026-11-20T21:49:49Z canonical: "docs.validatar.com/connect-to-databricks" --- > ## Documentation Index > Fetch the complete documentation index at: https://docs.validatar.com/llms.txt > Use this file to discover all available pages before exploring further. # Connect to Databricks Connect Validatar to Databricks with the **Databricks Driver** connection type and the built-in **Databricks** data source template. Validatar signs in as a Databricks service principal with OAuth, runs its queries on a SQL warehouse, and catalogs one Unity Catalog catalog per data source. > **Applies to:** Validatar Cloud · Validatar Server · 2026.1 and later The Databricks Driver connection type arrived in Validatar 2026.1. It talks to the Databricks SQL Statement Execution API over HTTPS, so there is no ODBC driver to install on Validatar Server or a Data Agent. When you finish this page, the catalog's schemas, tables and columns are in the Validatar Catalog, ready to profile and test. ## A data source or a processing engine? Databricks shows up in two places in Validatar: | | Databricks data source | Databricks data processing engine | | --- | --- | --- | | What it is | A connection to one Unity Catalog catalog that Validatar catalogs, profiles and tests | Where Validatar runs comparisons and stores detailed results, inside your Databricks workspace | | Where you set it up | **Settings** > **Data Sources** | **Settings** > **Data Processing Engines** | | Connection type | **Databricks Driver**, or **Databricks Engine** when it runs through an engine | Not a data source | | Needed for | Every Databricks catalog you test | Optional, and licensed. See [Creating a Databricks Data Processing Engine](/docs/creating-a-databricks-data-processing-engine). | Both sign in the same way, with a service principal's OAuth client ID and secret, so one service principal can serve both. ## Before you start - Validatar 2026.1 or later, and the **Data Source Admin** role. - A Databricks workspace with Unity Catalog and a SQL warehouse. The template reads each catalog's `information_schema`, which Unity Catalog provides. - A service principal with an OAuth secret. You need its client ID and the secret. [Creating a Databricks Data Processing Engine](/docs/creating-a-databricks-data-processing-engine) walks through creating one, granting it workspace access and **Can use** on the SQL warehouse. - Read access for the service principal on the catalog you want to test. - The warehouse ID. Find it on the warehouse's **Connection details** tab, or as the last part of the warehouse URL. Validatar Cloud reaches the workspace over the internet. On Validatar Server, or through a Data Agent, that host needs outbound HTTPS to the workspace. A catalog-level grant covers every schema and table in it, including ones created later: ``` -- Read access to one catalog for the Validatar service principal. -- Replace the catalog name and the service principal application ID. GRANT USE CATALOG ON CATALOG `` TO ``; GRANT USE SCHEMA ON CATALOG `` TO ``; GRANT SELECT ON CATALOG `` TO ``; ``` ## Create the Databricks data source 1. Go to **Settings** > **Data Sources** and click **New Data Source**. 2. Enter a **Name**, choose the **Environment** and add a **Description**. 3. In the **Connection** panel, click **Add**. 4. Leave **Processing Engine** on **Built-In Engine: Save results in Local Storage**. 5. Choose **Databricks Driver** as the **Connection Type**. Validatar selects the **Databricks** template. 6. Leave **Data Agent Group** on **No Assigned Group**, unless your workspace accepts connections only from your own network. 7. Click **Build**. The **Build Connection String** dialog opens. 8. Fill in the five fields (table below) and click **Update Connection String**. 9. Enter the service principal's OAuth secret in **Password**. 10. Click **Test Connection** and wait for **Connection succeeded**. 11. Click **Create Connection**, then **Save**. | Field in **Build** | Enter | Example | | --- | --- | --- | | **Databricks Host Domain** | The workspace host name. `https://` is optional. | `adb-1234567890123456.7.azuredatabricks.net` | | **Database Name** | The Unity Catalog catalog to catalog and test | `<catalog>` | | **Warehouse Name** | The SQL warehouse **ID**, not its display name | `<warehouse-id>` | | **Client ID** | The service principal's application (client) ID | `<application-id>` | | **OAuth Scope** | Leave the default, `all-apis` | `all-apis` | The finished connection string keeps the secret out of the text with the `[[PASSWORD]]` placeholder: ``` host=adb-1234567890123456.7.azuredatabricks.net;database=;warehouse=;client_id=;scope=all-apis;client_secret=[[PASSWORD]]; ``` ### Authentication The Databricks Driver connection type signs in with OAuth machine-to-machine credentials only: a service principal's client ID and secret. Personal access tokens are not accepted. When the secret expires, the data source stops connecting until you generate a new secret in Databricks and enter it in **Password**, so note the expiry date. ## Add it to a project and ingest 1. On the data source, open **Projects**, check the projects that will test it, and click **Save**. 2. Open **Schema Metadata** and click **Refresh Now**. 3. Wait for the run in **Ingestion History** to finish. The Databricks template reads the catalog's `information_schema`: every schema except `information_schema`, tables and views with their definitions, and columns with their full data type, nullability, identity, primary key constraints and comments. To test several catalogs, create one data source per catalog. [Databricks Data Source Template](/docs/databricks-data-source-template) lists what the template maps and profiles. ## Check that it worked - **Metadata Statistics** on **Schema Metadata** shows the schema, table and column counts. - In **Catalog**, select the data source. The **Schemas** tab lists the catalog's schemas. - Open a table and its **Preview** tab. Rows come back from the SQL warehouse. A stopped warehouse starts when the first query arrives, so the first test or ingestion after a quiet spell can take longer than the rest. ## Connect through a Databricks processing engine If your team has set up a [Databricks data processing engine](/docs/creating-a-databricks-data-processing-engine), a data source can use the engine's connection instead of its own: 1. In the connection dialog, choose the engine in **Processing Engine**. The **Connection Type** changes to **Databricks Engine**. 2. Choose the **Database** (the catalog). The list shows the catalogs the engine's service principal can read. 3. Leave **Schema** on **Use default schema**, or choose one. 4. Click **Test Connection**, **Create Connection** and **Save**. There are no credentials to enter: the source signs in as the engine's service principal. Tests on this source run inside Databricks and keep their detailed results in the engine's results catalog. ## Do it with Validatar AI > **Do it with Validatar AI** (Validatar Cloud, beta). Open the AI panel and ask: "Create a Databricks Driver data source named Lakehouse for the catalog on host , warehouse ID and client ID , using the Databricks template. Leave the client secret blank." Review what it creates before you run it. You can undo the change from the chat history. Enter the secret yourself: open the data source's **Connection** page, click **Edit**, type it in **Password**, test and save. Never paste secrets into the chat. The AI cannot ingest metadata, so click **Refresh Now** afterwards. ## Troubleshooting | Symptom | Cause | Fix | | --- | --- | --- | | `invalid_client` or another OAuth error on **Test Connection** | Wrong client ID, wrong or expired secret, or a secret from another workspace account | Generate a new secret for the service principal and enter it in **Password**. Check **Client ID**. | | "Missing ClientSecret" | **Password** is empty | Enter the OAuth secret in **Password**. | | An error about the warehouse | **Warehouse Name** holds the warehouse display name or HTTP path, or the service principal lacks **Can use** | Enter the warehouse ID. Grant **Can use** on the warehouse. | | Host or name resolution errors | **Databricks Host Domain** is wrong, or includes a path | Enter only the host name, such as `adb-1234567890123456.7.azuredatabricks.net`. | | Ingestion finds no schemas or tables | **Database Name** is not the catalog name, or the service principal lacks `USE CATALOG`, `USE SCHEMA` or `SELECT` | Fix the catalog name or run the grants above, then click **Refresh Now**. | | Tables in `hive_metastore` are missing | The template reads Unity Catalog `information_schema` | Test Unity Catalog tables, or migrate the tables to Unity Catalog. | | The data source stopped working after months | The OAuth secret expired | Generate a new secret and update **Password**. | ## Related - [Create a Data Source](/docs/create-a-data-source) - [Databricks Data Source Template](/docs/databricks-data-source-template) - [Creating a Databricks Data Processing Engine](/docs/creating-a-databricks-data-processing-engine) - [Ingesting Schema Metadata](/docs/ingesting-schema-metadata) - [Create and Run a Profile Set](/docs/create-and-run-a-profile-set) Engineers who script their setup can create Databricks data sources through the [Validatar MCP server](/docs/connecting-to-the-validatar-mcp-server) and the [Validatar Agent Kits](/docs/validatar-agent-kits). ## Related - [Create a Data Source](/create-a-data-source.md) - [Databricks Data Source Template](/databricks-data-source-template.md) - [Creating a Databricks Data Processing Engine](/creating-a-databricks-data-processing-engine.md) - [Ingesting Schema Metadata](/ingesting-schema-metadata.md) - [Create and Run a Profile Set](/create-and-run-a-profile-set.md)