Overview
When a Validatar test finds bad data, someone has to fix it, and most data teams already track that work in Jira. The Jira integration opens a Jira issue when a test fails, comments on it while the test keeps failing, and resolves it when the test passes again. Each issue links back to the test and to the failing run, and carries the test's description and the query that failed, so the person who picks it up can start without asking where the problem is.
The integration also works in the other direction. Open issues appear on the warehouse tables and columns they affect in the Validatar catalog, and Jira's own configuration and the issues Validatar raised are cataloged on the Jira data source, so you can write Validatar tests about them.
It runs inside Validatar as a data source template written in plain Python, so there is no Jira app to install and no separate server or scheduler to maintain.
Get the integration
Six items on the Validatar marketplace make up the integration. Two of them, the Issue Tracking field sets, are shared with the other issue-management integrations as they arrive.
| Marketplace item | What it is | Reference |
|---|---|---|
| Jira Cloud - Data Source Template | The Jira data source: credentials for Jira and for the Validatar API, the actions that open and resolve issues, the reconciliation sweep, read macros, and a catalog of the Jira site | Jira Cloud Data Source Template |
| Issue Tracking - Project Custom Fields | An Issue Tracking section on every test: where its issues go, and the issue it currently has open | Issue Tracking Custom Fields |
| Issue Tracking - Catalog Custom Fields | An Issue Tracking section on warehouse tables and columns, listing their open issues | Issue Tracking Custom Fields |
| Jira - Catalog Custom Fields | A Jira section for the Jira data source's own catalog: projects, issue types, fields and issues | Jira Catalog Custom Fields |
| Jira Issue Automation | An example test with the Jira action already attached, the reconciliation sweep, and a job that runs them in order | Jira Issue Automation |
| Jira Health Checks | 17 tests that check the integration's configuration, whether Jira and Validatar agree, and the hygiene of the issues Validatar raised | Jira Health Checks |
What happens in Jira
A test gets the integration through a test action: a Test Ran trigger with one Run a Macro step that calls the template's Sync Jira Issue with Test Result (Automatic) macro. The step needs no settings: the macro reads the finished run from Validatar, including the scripts that ran on the test and control sides. After every run, the macro looks in Jira for the test's open issue and decides:
| The test | Open issue in Jira | What the action does |
|---|---|---|
| Failed or errored | None | Opens an issue |
| Failed or errored | One | Adds a comment with this run's counts and a link to it, and adds one to the issue's occurrence count |
| Passed | One | Adds a closing comment and moves the issue into a done status |
| Passed | None | Nothing |
A closed issue is never reopened. If the test fails again after its issue was closed, a new issue is opened, so the history of each incident stays in its own issue.
A new issue holds:
- a summary of
[DQ], the test name and its folder path - the project, folder, result, failed and errored counts, and when the run completed
- the test's description
- a link to the failing run in Validatar
- the error messages, when the test errored
- the query the test ran, so the issue shows the data that failed
- a link in Jira's Web links back to the test
The issue goes to the data source's default project and issue type, with any field values the data source sets for every issue, such as a severity field your Jira project requires. A test can send its issues somewhere else through its Issue Tracking fields: another project or issue type, an assignee, a priority, extra labels, or extra field values. See Issue Tracking Custom Fields.
Before writing, the action reads the project's create screen from Jira and checks every value against it. A required field with no value, or an option Jira no longer offers, stops the action with a message naming the field and the values Jira accepts, rather than a bare error from Jira.
Noise controls
The defaults open an issue on the first failure and resolve it on the first pass. Each of these can change that, on the data source or for a single action:
| Setting | Default | Effect |
|---|---|---|
| Issue Grain | incident | incident keeps one open issue per test. failure opens a new issue for every failing run |
| Open After Failures | 1 | Open an issue only after this many failures in a row |
| Cooldown Hours | 0 (off) | A failure this soon after a close comments on the closed issue instead of opening a new one |
| Resolve After Passes | 1 | Resolve only after this many passes in a row. 0 leaves closing to a person |
| Error Policy | failure | failure treats an Error result like a failure. ignore takes no action on errors |
| Escalate After | 0 (off) | Raise the issue's priority one step after this many occurrences |
While a count is running, the test's Issue Status field shows it, for example Pending: 2 of 3 failures.
Dry Run
New data sources start with Dry Run on. Every action still looks up the test's issue and decides what it would do, and nothing is written to Jira or to Validatar. Each action macro has a Plan twin that returns the decision and the exact fields of the issue it would open, without writing, whatever Dry Run says. Turn Dry Run off on the data source when the setup checks out; the last steps of Jira Setup for Validatar cover this.
Keeping Jira and Validatar in step
An action runs after its test finishes, and a failed action step does not fail its test. If Jira is unreachable for a minute, the test's history shows nothing wrong and the issue simply is not opened. The Reconcile Jira Issues with Validatar macro covers that gap. Placed as the last test in a scheduled job, it checks every test linked to your warehouse data sources and:
- opens issues for failing tests that have none
- resolves open issues whose test now passes
- closes duplicate open issues for one test, keeping the oldest
- refreshes the Issue Key on each test
- writes the open issues onto the tables and columns each test is linked to
Because the sweep is itself a Validatar test, it has a pass or fail history and can alert like any other test. It waits 30 seconds before changing anything, so the actions from the same job land first and are not duplicated. The Jira Issue Automation pack includes the sweep and a job that runs it last.
What appears in Validatar
On each test. The Issue Tracking section shows the test's open issue key, a link to it, its status and when it was last checked. These are a cache. Jira is the record, and every decision is made from Jira, so a stale value on a test cannot cause a wrong action.
On warehouse tables and columns. When the sweep runs, each table linked to a test with an open issue shows the open issue keys, how many are open, the oldest and newest, and the highest priority among them. Columns show their own keys and count. A data consumer browsing the catalog sees that a table has a known problem before using it.
On the Jira data source. Metadata ingestion catalogs the Jira site itself:
| In the catalog | Holds |
|---|---|
| A schema per Jira project | A table per issue type, with a column per field on its create screen. Each field records whether it is required, whether Jira fills a default, its type and its allowed values |
A _config schema |
Statuses, status categories, priorities, resolutions and fields across the site |
A <KEY> Issues schema per project |
A table per issue Validatar raised: open ones, plus those closed in the last 30 days, up to 1,000. Each holds its status, priority, assignee, age, occurrences and a link to its test |
The create contract is the useful part. "What must I send to create a Bug in DQ" becomes the columns of the DQ.Bug table marked required with no default, and a test can fail when someone adds a required field the integration cannot fill.
The template also has read macros for issue data, JQL queries, issue counts, aging, time in status, field fill rates and change history, so Jira data can be tested like any other source. See Jira Cloud Data Source Template.
How an issue stays linked to its test
Each issue carries two labels: validatar-<instance>, which marks every issue this Validatar instance raised, and validatar-<instance>-t<test id>, which marks the issues of one test. <instance> is a short name for the Validatar instance, so two Validatar instances writing to the same Jira site never mix their issues. An issue property named validatar holds the test's id, URL, when it first and last failed, and how many times.
To find a test's open issue, the action searches for its label, adds the issue key cached on the test, and then reads each candidate directly. The direct read matters: Jira's search index trails its writes by a few seconds, and an issue created or resolved a moment ago can still be missing from, or wrongly present in, search results. If two runs of the same test open issues at the same moment, the action closes the newer one as a duplicate of the older.
Do not remove the labels by hand. An issue without its test label can no longer be found by its test, and the Jira Health Checks pack reports it.
Requirements
Jira
- Jira Cloud. Jira Data Center is not supported yet.
- An Atlassian account for the integration and an API token for it, classic or scoped.
- In each project the integration writes to, that account needs: Browse Projects, Create Issues, Edit Issues, Add Comments, Transition Issues, Resolve Issues and Link Issues, plus Assign Issues if the integration assigns issues. Create Issues is needed even to read a project's fields: Jira returns an empty list, with no error, to an account without it.
Validatar
- A Validatar API token for the integration, with the Authoring, Data Steward and Explorer scopes. The actions use it to read each run and to keep the Issue Tracking fields current. Its user needs access to every project whose tests open issues.
- The Standard Test Actions optional feature. Without it, action steps are not saved.
- Govern Edition, to add the Issue Tracking and Jira custom fields.
The integration uses Jira's REST API version 3 over HTTPS and needs no extra Python packages.
Rollout
- Prepare Jira. Choose the project, check its required fields, and create the integration's account and token. See Jira Setup for Validatar.
- Create the data source. Import the template and the custom fields, create the Jira data source, and run metadata ingestion to check that Validatar sees the project and its fields.
- Try it in Dry Run. Import the Jira Issue Automation pack, run its example test, and read the decision in the action log.
- Turn Dry Run off and attach the action to your critical tests. Open and Resolve Jira Issues from Test Results walks through it.
- Add the sweep as the last step of each scheduled job, and import the Jira Health Checks to watch the integration itself.
What has been tested
The integration was built against a Jira Cloud site with a company-managed project that has two required custom fields with no default, a select list, a number field and a date field.
- An offline suite of 216 checks runs the template's own code against a simulated Jira and Validatar, including Jira's search lag, its paging, its type checks on writes and its silent empty answer to an account without Create Issues.
- A live suite of 65 checks runs the same code against the Jira Cloud site and reads every write back from Jira.
- Inside Validatar, real test actions opened, commented on and resolved issues, the sweep opened an issue an action had missed, and open issues appeared on the linked warehouse tables and column and cleared when resolved. The 17 health checks gave their designed result against a seeded Jira project, and the starter pack's example opened and resolved its issue.
Not yet tested: scoped API tokens (classic tokens were used throughout), a project with a custom permission scheme (the test site's plan has none), and workflows with a required screen on their done transition.