What this report shows
Job Execution Timings gives you one row per job run, with the batch duration alongside the number of tests that passed, failed, and errored in that run.
That combination is the point. Duration on its own tells you a job got slower; duration next to test counts tells you whether it got slower because it did more work or because the server had less capacity to do it with. That distinction is what makes this report useful as an infrastructure baseline.
Use it when you are:
- Establishing a performance baseline before changing server resources
- Comparing job performance before and after a resize, an upgrade, or a configuration change
- Responding to a cloud cost-optimization review and need evidence rather than opinion
- Tracking whether execution times are drifting as your test volume grows
It is the companion artifact to Validatar Server Instance Sizing Best Practices, which explains how to interpret what you find here.
Where to get it
This report is published to the Validatar marketplace: Job Execution Timings
Prerequisites
- Validatar Cloud or Validatar Server
- Permission to create or import reports
- At least one job with execution history. The report defaults to the last 30 days — widen the Run Date filter if your history is older.
Import
Marketplace content imports directly. You do not need to download the file and then work out which screen to import it on — Validatar stages the content for you and routes it by content type, so a report lands on the report import review automatically.
- In Validatar, open Marketplace from the main navigation.
- Find Job Execution Timings and open it.
- Choose Import.
- Validatar takes you straight to report management with the import already loaded. Review it — the report appears under Job Execution History as Job Execution Timings.
- Commit the import.
> Note: Every marketplace item behaves this way, but the destination depends on what it is. A test pack asks which project to import into and then opens that project's import screen; a data source template opens Settings > Data Source Templates; catalog and project custom fields open their matching custom fields page. Items that are reference material rather than importable content offer a download only.
Importing from a downloaded file
If you would rather keep a copy of the artifact, or import it into an environment without marketplace access, download job-execution-timings-report.xml from the marketplace listing, then go to Reports, choose Import, and select the file. The review and commit steps are identical.
What you get
Columns
| Column | Why it is here |
|---|---|
| Project | Which project the job belongs to |
| Job Name | Your benchmark job |
| Job Folder Name | Where the job lives |
| Run Date | Sorted newest first — this is your time series |
| Duration | The primary measurement. Total batch wall-clock time, including steps that ran in parallel |
| Status | Batch status — confirms the run actually completed |
| Result | Overall pass/fail outcome of the run |
| # Passed Tests | Volume of work completed |
| # Failed Tests | Should not change when you change server resources — if it does, investigate before drawing conclusions |
| # Errored Tests | Errors appearing after a resize are a strong signal of resource pressure |
| Triggered By | Scheduled runs and ad-hoc runs are not comparable; use this to separate them |
| Runtime Parent | Identifies runs invoked as part of a parent job |
Filters
| Filter | Default | Purpose |
|---|---|---|
| Run Date | Last 30 Days | Keeps the report to a recent, relevant window |
| Job Name | Is Not Blank | A placeholder so the report returns all jobs out of the box |
Customization
Isolate one benchmark job. The Job Name filter ships as Is Not Blank so the report works immediately across every job. To focus on a single job, change the operator to Equals and enter the job name — or Contains if you want a family of related jobs.
Tip: Pick a job that runs on a regular schedule and contains a realistic mix of standard tests, template tests, and any Python steps. A job that only runs occasionally, or that varies a lot in what it does, will not give you a stable baseline.
Adjust the window. Change Run Date to a longer period if you need to cover a full business cycle. A baseline should include at least one month-end close, and any seasonal peak in your source systems.
Separate scheduled from manual runs. Add a Triggered By filter if ad-hoc executions are distorting your averages.
Export for comparison. Export the results to Excel and record them against the before/after table in Validatar Server Instance Sizing Best Practices. Capture the same job across both periods so the comparison holds.
Reading the results
Compare the same job across two periods and look at duration alongside test counts:
- Duration steady, test counts steady — no capacity change. This is the outcome you want after a downsize.
- Duration up, test counts up — the job is doing more work. Not a capacity problem.
- Duration up, test counts steady — the server has less capacity than it did. This is the signal that matters.
- Errored tests appearing where there were none — resource pressure. Investigate before accepting the change.
If the whole job takes considerably longer but individual tests complete at roughly their previous speed, you have lost parallel capacity rather than raw speed — which points at core count rather than memory. The sizing guide covers this distinction in detail.
Troubleshooting
The report is empty. Most often the Run Date window does not cover any execution history. Widen it, and confirm the job has actually run.
Import reports an unknown field. The report definition targets the Job Execution History report type. If your Validatar version predates one of the columns, remove that column from the report after import.
Durations vary widely between runs of the same job. Check Triggered By — you may be mixing scheduled and manual runs. Also confirm the job is genuinely doing the same work each time; a job whose test count moves around will not give you a usable baseline.