What this pack contains
Seventeen standard tests that check the Jira integration itself: whether it can still write to Jira, whether Jira and Validatar agree about which tests have open issues, and whether the issues it raised are being worked. They run on a data source built from the Jira Cloud Data Source Template, and look only at issues raised by your Validatar instance.
Each test counts the problems it finds and passes at zero. A failing test prints the problems in its result and attaches them as a CSV file, so you see the issue key, the test and what is wrong, not only a number. Thresholds and scope sit in a PARAMETERS block at the top of each script.
The tests are imported into three folders.
Configuration
These check every place the integration writes to: the data source's default project and issue type, plus any project, issue type or field value a test's Issue Tracking fields route to. A routing mistake on one test shows here before that test fails and its action cannot open an issue.
| Test | Dimension | Severity | Fails when |
|---|---|---|---|
| Account can create issues in every target project | Security | Critical | A target project is not visible to the integration's account, or the account lacks Browse Projects, Create Issues, Edit Issues, Add Comments, Transition Issues or Resolve Issues there |
| Issue types used by the integration exist | Validity | High | A target issue type does not exist in its project. The result lists the types that do |
| Field defaults and routing values are valid | Validity | High | A value in Field Defaults (JSON), a test's Issue Field Values, Issue Priority or Issue Assignee names a field that is not on the create screen, an option Jira no longer offers, or a user Jira cannot find; or a test's Issue Field Values is not valid JSON |
| Required fields have a value on every create | Completeness | Critical | A required field with no Jira default gets no value from the integration. Usually a field made required after setup |
| Workflows can resolve Validatar issues | Processing | High | A target issue type has no status in the done category, or an open Validatar issue has no transition into one from where it stands |
Integration consistency
| Test | Dimension | Severity | Fails when |
|---|---|---|---|
| One open issue per test | Uniqueness | High | A test has more than one open issue. Passes when Issue Grain is failure, where several are expected |
| Validatar issues keep their markers | Consistency | Medium | An open issue has the instance label but lost its test label, so no test can find it; or it has no Validatar state, so its occurrence count starts over |
| Open issues belong to existing tests | Relationships | Medium | An open issue's test was deleted. Nothing will resolve it |
| No open issue for a passing test | Consistency | High | A test passed on its last run and its issue is still open. Skipped while a Resolve After Passes count is running, and when closing is left to people |
| No closed issue for a failing test | Consistency | High | A test is failing, its issue was closed in the last 30 days, and no new one was opened. A close inside the Cooldown Hours window is skipped |
| Validatar issues are not reopened by hand | Consistency | Low | An open issue was reopened after a close. The integration never reopens, so a person did, and the next passing run will close it again |
| Issue Key on each test points at an open issue | Consistency | Low | A test's Issue Key field names an issue that is closed or gone |
Issue hygiene
| Test | Dimension | Severity | Fails when |
|---|---|---|---|
| Data quality issues are resolved within 14 days | Timeliness | Medium | An open Validatar issue is older than 14 days |
| Data quality issues are assigned within 24 hours | Timeliness | Medium | An open Validatar issue still has no assignee 24 hours after it was opened |
| Open issues have a priority and severity | Completeness | Low | An open Validatar issue has no priority, or no value in the severity field you name |
| No test opened more than 3 issues in 30 days | Consistency | Medium | A test opened more than 3 issues in 30 days. Usually a flapping test or a threshold set too tight |
| Closed issues carry a resolution | Completeness | Low | A Validatar issue closed in the last 30 days has no resolution, so Jira reports that count resolved issues miss it |
Every test name in Validatar starts with Jira -, for example Jira - One open issue per test.
Where to get it
Marketplace: Jira Health Checks
Prerequisites
- A Jira data source built from the Jira Cloud Data Source Template, with Validatar Base URL and Validatar API Token set.
- Reconcile Data Source Ids set on the data source to the warehouse data sources whose tests open issues. The configuration tests read the routing fields, and three consistency tests read the last result or Issue Key, of tests linked to tables or columns of those data sources. With it blank, the configuration tests check the data source's own defaults only, and the three consistency tests pass with a note that no tests were checked.
- The Issue Tracking - Project Custom Fields, for the checks that read a test's routing fields and Issue Key.
Import
- Create a folder for the pack in the target project.
- Open Marketplace in Validatar, find Jira Health Checks, choose Get, then Import Directly.
- Choose the target project when prompted.
- Map the pack's data source, Jira Cloud (Dev), to your Jira data source, and choose the folder from step 1.
- Choose Review Changes. The review lists the 17 tests in a Jira Health Checks folder with Configuration, Integration Consistency and Issue Hygiene subfolders. Choose Import.
- Add the three subfolders to a daily job, or one folder step that includes subfolders.
Adjusting a test
Open the test and edit the PARAMETERS block at the top of its script:
# Jira Health Check: Data quality issues are resolved within 14 days
# Runs on the Jira Cloud data source. VALUE = problems found; passes at 0.
# PARAMETERS --------------------------------------------------------
MAX_AGE_DAYS = 14 # oldest acceptable open issue
# -------------------------------------------------------------------
dataframe = health_frame(health_open_too_long(MAX_AGE_DAYS), "open_too_long")
| Parameter | In | Default | Change it to |
|---|---|---|---|
DATA_SOURCE_IDS |
The configuration checks, No open issue for a passing test, No closed issue for a failing test, Issue Key on each test | blank: Reconcile Data Source Ids | Check other warehouse data sources, comma-separated |
PROJECT_ID, FOLDER_ID |
The same tests | blank: everything | Limit the check to one Validatar project or folder |
SAMPLE |
Workflows can resolve Validatar issues | 20 | Check the transitions of more or fewer open issues |
DAYS |
No closed issue for a failing test, No test opened more than 3 issues, Closed issues carry a resolution | 30 | A longer or shorter window |
LIMIT |
Validatar issues are not reopened by hand | 200 | Read the history of more open issues |
MAX_AGE_DAYS |
Data quality issues are resolved within 14 days | 14 | Your resolution target |
MAX_HOURS |
Data quality issues are assigned within 24 hours | 24 | Your triage target |
SEVERITY_FIELD |
Open issues have a priority and severity | blank: priority only | Your severity field's name or id, such as DQ Severity |
MAX_ISSUES |
No test opened more than 3 issues in 30 days | 3 | The most issues one test may open in the window |
Rename a hygiene test when you change its threshold, so its name still states what it checks.
What a failing result looks like
A failing test lists one row per problem in its result, and the same rows in the attached CSV. For Jira - Validatar issues are not reopened by hand, a result looks like this:
1 problem(s) found:
issue_key status test_id test_name reopened reopened_by change times
DQ-68 Backlog 2839 Every order has a customer 2026-10-15T14:02:11 Alex Kim Done -> Backlog 1
On the Jira site the integration was built against, the pack ran against a seeded project with one example of each problem that site could hold. Twelve tests failed with exactly one problem each, as designed. The other five passed: the seeded project has no unfilled required field, its workflow resolves every issue, every issue has a priority and a resolution, and the seeded issues were too new for the 14-day and 24-hour limits.
Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| Tests print "No warehouse data sources in scope" | Reconcile Data Source Ids is blank on the Jira data source | Set it, or set DATA_SOURCE_IDS in those tests |
| Every test errors with HTTP 401 | The Jira API token expired or was revoked | Create a new token and update the data source |
Open issues have a priority and severity errors with No Jira field named |
SEVERITY_FIELD does not match a Jira field |
Use the field's exact name, or its id |
| A check finds issues from another Validatar instance | Both instances use the same Validatar Instance Key | Give each instance its own key on its Jira data source |