To run a test, open it, click Run Test, choose Test Repository under Send Results To, and click Run. Validatar opens the result page, which fills in when the run finishes and stays in the test's history.
This page covers running one test, several tests at once and a template test's child tests, then how to read what comes back. To run tests on a timetable, put them in a job and schedule the job.
Before you start
- You need the Run Tests permission on the data sources the test reads. Without it, Run Test is greyed out.
- To see saved results, you also need View Test Results on those data sources.
- A new test runs only after you save it once.
Run one test
The example runs Wholesale Orders Reach the Warehouse by Month, a standard test in the Warehouse Reconciliation project. It counts orders per calendar month in the Wholesale Distribution System (SQL Server) and in the warehouse copy in the Enterprise Data Warehouse (Snowflake), and compares the two month by month.
- Go to Projects and open Warehouse Reconciliation.
- Click Tests, then open the Wholesale System to Warehouse folder.
- Open Wholesale Orders Reach the Warehouse by Month.
- Click Run Test in the toolbar. The Run Test dialog opens.
- Under Send Results To, select Test Repository.
- Click Run.
The result page opens and shows the run as Running until it finishes.
Screen or Test Repository
Send Results To decides whether the run counts.
| Option | What happens | Use it when |
|---|---|---|
| Screen | The result opens in a Test Execution Progress window with Events and Result tabs. Nothing is saved: the run is not in the test's history or the project's results, does not change the test's overall status and does not count toward any trust score. | You are building or changing the test and want to see what it returns |
| Test Repository | The result is saved and the result page opens. The run appears on the test's History tab and in the project's results, updates the test's overall status, and counts toward the trust scores of the tables and columns the test is linked to. | Any run you want to keep, show someone or have count |
Screen is selected when the dialog opens, so check it before you click Run.
- If you lack View Test Results on one of the test's data sources, Screen is unavailable and the dialog says "You do not have View Results access to the selected data source(s)". The run goes to the repository.
- If the test has unsaved changes and you choose Test Repository, the dialog says "Changes to this test will be saved automatically".
- Some older tests also show Advanced Configuration, where a script data set can run against a different Test Data Source or Control Data Source.
Run several tests at once
From a test folder:
- Open the folder and select two or more standard tests. Child tests count as standard tests.
- Click Run. The Let's run these tests! dialog lists the tests with their last run.
- Set How many tests would you like to run in parallel? The default is 10.
- Click Run All.
A group of tests always saves to the repository; the dialog says "Running multiple tests as a group requires sending the tests to the repository." A results window opens and fills in as the tests finish.
Run works on tests, not folders. With a folder selected, or a template test among several items, it stays greyed out; a template test selected on its own opens the dialog described in the next section. To run everything in a folder, subfolders and template tests included, add the folder to a job as a step and run the job.
From the Catalog:
- Go to Catalog and open the table, for example Wholesale Distribution System > Sales > Orders.
- Open the Tests tab. It lists the tests linked to the table, which can come from several projects.
- Select one test or several and click Run. One test opens the Run Test dialog; several open the Let's run these tests! dialog.
Tests from different projects cannot run together: Validatar says "Only tests within the same project can be run together."
Run a template test's child tests
A template test does not run itself; its child tests do. Each child test is a standard test, so you can also open one and run it like any other test.
- In the Wholesale System to Warehouse folder, open the template Wholesale Row Counts Match the Warehouse.
- Click Run Child Tests.
- Under What would you like to run?, choose Single Child Test and pick a child, or All 49 Child Tests.
- Under Send Results To, choose Screen or Test Repository. Running all child tests always saves to the repository ("Results from multiple tests cannot be sent to the screen").
- Under Advanced Configuration, set How many tests would you like to run in parallel?
- Click Run.
If the template changed since its child tests were last materialized, the dialog says so and offers Materialize Child Tests. Materialize first: until you do, the child tests still hold the old definition. See Materialize Child Tests.
After you change a template, run one child to the screen (for example the Sales.Orders child) to check it, then run all 49 to the repository.
Read the result page
The page header reads Test Result: with the test name, the run time and the result. Go to Test opens the test, and Export to Excel exports the rows of a row-level result. The page has three tabs.
| Tab | What it shows |
|---|---|
| Results | The result, the values compared and the success conditions |
| Execution Detail | The test and its folder, each side's data source, data set type and script duration, who ran it, start and completion times, duration and storage location |
| Test Detail | The test definition as it was for this run, read-only. The tab title carries the version, for example Test Detail (v3) |
Passed, Failed and Error
| Result | What it means | What to do |
|---|---|---|
| Passed | The data met the test's success conditions | Nothing |
| Failed | The test ran, and the data did not meet the conditions. This is a finding about your data | Read the values and rows on the Results tab |
| Error | The test could not finish: a script failed, a data source could not be reached, or some rows could not be evaluated. It says nothing about the data yet | Read the Errors box on the Results tab, then see Why Is My Test Failing or Erroring? |
The Errors box has two columns. Location says where the error happened (Test Script, Control Script, Test or Batch), and Description carries the message, often the database's own. When some rows error and the test does not otherwise pass, the result is Error, not Failed.
One value on each side
A test that returns one value per side shows a Metric and Value grid: Test Value, Control Value (or Control Range) and Difference (or % Difference). The Result Success Condition next to it is the rule the values had to meet.
The child test Wholesale Row Counts Match the Warehouse - Sales.Orders counts the rows of Sales.Orders in the wholesale system (the test side) and in its warehouse copy (the control side). On 17 November 2026 the system had 478,737 orders and the warehouse 431,507, a difference of 47,230, so the child failed.
A value shows as missing when its side returned no rows, and as null when the query returned a null. An asterisk marks a missing value that the test's settings replaced; the page then says "One or more missing result values were overridden."
Rows compared by key
A test that returns many rows, each with a key, compares the two sides row by row. Overall Results counts Total Rows and how many Passed, Failed or hit an Error, then splits them by match status.
| Match status | What it means |
|---|---|
| Matched | The key is on both sides, so the values were compared |
| Test Only | The key is only in the test data set |
| Control Only | The key is only in the control data set |
Whether a key on one side only passes or fails is the test's Unmatched Keys setting, which is Fail unless someone changed it. Click any number in Overall Results to filter the grid to those rows.
The grid shows Row #, the key columns, Match Status and Record Result, then for each value column Test Value, Control Value, Difference and Result. The Columns panel lists each column by role (key or value) with its name on each side and a bar of passed, failed and errored rows; clear a column's box to hide it. Conditions/Quality Score shows the Value Success Condition, the Overall Success Condition and, when the test scores partial passes, the Quality Score.
In Wholesale Orders Reach the Warehouse by Month, the test side is the warehouse, the control side is the wholesale system, and the key is the month (2015-02, 2024-04). The 17 November 2026 run compared 167 months: 46 passed and 121 failed.
- Every month from May 2024 on is Control Only. The warehouse copy stops at an order taken on 23 April 2024, so those months have orders in the system and none in the warehouse.
- Matched months as far back as February 2015 fail by a few orders each. A backorder keeps the date of the order it came from, so the system kept adding orders to old months after the copy stopped.
Other lines you may see above the results:
| Message | What it means |
|---|---|
| Showing failed results only | The test keeps only failing rows |
| Showing top N results | Only part of the rows are shown |
| Aborted test after N failures | The test stopped reading after N failing rows, as its settings ask. Use the count tests for the full size of a problem |
| One or more result keys were longer than N characters and were truncated to that length | Long keys were cut short |
| The results of this test execution were purged | The row detail was deleted under the test's retention setting. The result itself stays |
| Click to view record detail from Snowflake (or Databricks) | The row detail is stored in a data processing engine. Click to load it |
History, versions and overall status
On the test's own page:
- The History tab lists every saved run under Execution History (Run Date, Result, Runtime Context, Runtime Configuration, Duration). Select a run and click View Results to open it. The right side shows who changed the test and when.
- The version list in the toolbar (it shows v and the number) opens an earlier version of the test; Revert brings it back. A result's Test Detail tab shows the version that ran.
- The card at the top right shows the test's overall status, worked out from saved runs only.
| Overall status | Card shows | What it means |
|---|---|---|
| Never Run | Never Run | No saved run yet |
| Consistent Passing, Consistent Failing, Consistent Erroring | Passing, Failing or Erroring | Every saved run in the last 90 days had the same result |
| Now Passing, Now Failing, Now Erroring | The status, with "Since" and a date | The result changed after a steady stretch |
| Intermittent Passes, Intermittent Failures, Intermittent Errors | Intermittent, with "Passed on", "Failed on" or "Errored on" and a date | Mixed results within the last 14 days |
| Stale Passed, Stale Failed, Stale Errored | The status | No saved run within the trust score look-back period (14 days unless your admin changed it). The word after Stale is the last result |
The test grids in folders and in the Catalog show the same values in the Overall Status column.
See every run in a project
Every saved run in a project lands on its results page, whoever or whatever started it: a person, a job, a schedule, the API or Validatar AI.
- Go to Projects and open the project.
- Click View Test/Job Results.
- Set Start Date, or From and To, and click Refresh.
The grid shows Run Date, Status, Result, Context, Name, Type, Run By and Duration. Double-click a row to open the result, or click Download to save the list. Status says whether the run is still going or how it ended (for example Running, Completed or Abandoned); Result summarizes the tests in it.
When Validatar AI runs several tests at once from the chat, the Name column shows the conversation's subject and links back to that conversation, and Context names the AI agent.
Do it with Validatar AI (Validatar Cloud, beta). Open the AI panel and ask: "Find the tests in Warehouse Reconciliation > Wholesale System to Warehouse that are failing, run them again, and tell me which still fail." The chat runs standard tests and child tests, not a template as a whole, and runs several at once only within one project. Check which tests it ran in the tool calls the chat shows. A run changes no test, so there is nothing to undo; its results land in the project's results with a link back to the conversation.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| Run Test is greyed out | You lack Run Tests on one of the test's data sources, you are looking at an earlier version of the test, or the test is not saved | Ask a data source admin for the permission, switch to the latest version, or save the test |
| Screen cannot be selected | You lack View Test Results on one of the test's data sources | Run to the repository, or ask for the permission |
| Run stays greyed out in a folder | The selection includes a folder, or a template test among several items | Select standard tests only, select the template on its own, or run the folder through a job |
| A run is missing from the test's history | It ran to Screen | Run it again with Test Repository |
| Child tests ignore a change you made to the template | The children were not materialized after the change | Materialize, then run |
| The result is Error or fails unexpectedly | A query, connection or data difference | See Why Is My Test Failing or Erroring? |
Related
- Create a Standard Test
- Create a Job and Schedule a Job
- Configuring Result Criteria
- Understanding Trust Scores
- Why Is My Test Failing or Erroring?
Engineers who run tests from their own tools can use the MCP server or the Validatar Agent Kits.