Validatar Server Instance Sizing Best Practices

Overview

This guide explains how to size the infrastructure that runs Validatar Server — how much memory and CPU the application server needs, how to size the repository database, what to measure before and after a change, and how to make a sizing decision defensible with data rather than guesswork.

It is written to be independent of any particular hosting platform. The guidance applies whether Validatar runs on AWS, Azure, another cloud, a virtual machine in your own data center, or physical hardware. Where a specific monitoring tool is useful, equivalents are given for each platform.

This guide is for administrators who are provisioning a new Validatar Server, responding to a cloud cost-optimization review, planning for growth, or investigating whether a performance problem is a capacity problem.

The core principle: Validatar Server is memory-bound

Almost every sizing mistake with Validatar traces back to a single misunderstanding, so it is worth stating plainly.

Validatar Server is constrained by memory far more often than by CPU.

When Validatar executes a test using the built-in processing engine, the server streams the result set back into the application, holds it in memory to sort and compare it row by row, evaluates pass or fail, and writes the outcome to the repository database. Test steps written in Python do the same thing through dataframes. Sorting and comparison are the expensive operations in this workload, and they are expensive in RAM rather than in clock cycles.

Two consequences follow:

• CPU utilization is a poor proxy for how close the server is to its limit. A Validatar instance can sit at low average CPU and still be one large test away from exhausting memory. Sizing decisions driven mainly by CPU will consistently under-size this workload.
• Peak matters more than average. Validatar workloads are typically batch-shaped: quiet for most of the day, then concentrated demand during a scheduled job window. An average taken across 24 hours describes the quiet period, not the period that determines whether the server is big enough.

Baseline requirements

These are the published minimums from Before You Begin. Treat them as the floor, not as a target.

The 4 GB figure at the bottom of the web server range is a functional minimum — enough to install the application and run the web front end. It is not a throughput number. For any server that executes scheduled jobs against production data volumes, design to the upper end of the range.

Validatar Server itself imposes no ceiling. As described in Validatar Server Architecture, there are no product limits on cores or memory; you size to your workload. Customers running very large parallel test volumes have scaled well beyond the documented range.

Sizing the application server

Start with memory

The question that determines memory is not "how many tests do we have?" It is "what is the largest result set a single test has to hold and sort in memory, and how many such tests run at the same time?"

Use these as planning starting points. They are intended to get you to a sensible first configuration; validate them against your own measurements using the method later in this article.

Two rules of thumb worth applying on top of the table:

• Leave headroom at peak. Aim to keep at least 20–25% of memory free during your busiest job window. A server that peaks at 95% is not correctly sized; it is a server that has not failed yet.
• Size for your growth horizon, not just today. If you plan to onboard additional applications or data sources in the next two quarters, size for that and revisit, rather than resizing repeatedly.

Then size CPU for parallelism

CPU governs concurrency — how many tests can execute at once — more than it governs the speed of any individual test.

• If your jobs run a modest number of tests in parallel, a smaller core count is usually adequate.
• If you run large jobs with high test concurrency, core count becomes the limiting factor. The symptom is distinctive: individual tests complete in about the same time as before, but the job as a whole takes considerably longer.

That symptom is the most useful diagnostic in this whole article. Slower individual tests point at memory or the data source. A slower job with unchanged individual test times points at core count.

Processor generation matters as much as core count. On cloud platforms, a current-generation instance with fewer cores frequently outperforms an older-generation instance with more, at lower cost. When a cost review recommends a reduction, moving to a newer instance family is often a better response than reducing memory — you keep the headroom the application depends on and still capture most of the saving.

Storage

• 50 GB for the application server covers the installation and its logs comfortably; the application files themselves are roughly 150 MB.
• Prioritize the type of storage over the amount. On cloud platforms, choosing a modern general-purpose SSD tier over an older one usually improves throughput and reduces cost at the same time, and lets you provision IOPS independently of volume size.
• Monitor free space as a first-class metric. Log growth on a busy server is easy to overlook.

Sizing the repository database

Size the repository database independently of the application server. Databases degrade differently from application servers under pressure — they turn memory shortfalls into I/O rather than failing outright, which makes under-sizing easy to miss until it is severe.

• Start at the documented 16 GB of RAM and monitor from there.

• Choose the storage type before increasing storage size. Moving from an older to a current general-purpose SSD tier typically reduces cost while improving IOPS and throughput.

• Control growth at the source. The repository grows primarily from retained test results. Several settings reduce both storage growth and write load, and are worth enabling regardless of the sizing outcome:

  • Result purging and retention policies
  • Only Keep Failures, which stores failing rows rather than complete result sets
  • Abort Processing After X Failures, which stops a badly failing test from writing an enormous result set

  These are covered in more detail in Performance Issues.

• Use database-native monitoring. Managed database services provide better instrumentation than infrastructure metrics alone — for example, Performance Insights and Enhanced Monitoring on Amazon RDS, or Intelligent Insights and Query Performance Insight on Azure SQL. If the database runs on a virtual machine you manage, the same guest-memory caveat described earlier applies.

Reduce the requirement before you increase the size

Before adding capacity — and before defending capacity you already have — consider whether the demand itself can be reduced. Several of these carry no performance risk at all.

Move processing off the server. This is the structural answer, and the one that changes the sizing conversation permanently.

• Push processing down to the data platform. Validatar can execute test processing inside Snowflake or another supported data processing engine rather than in the Validatar Server process. The comparison work then happens where the data already lives.
• Offload execution to a Data Agent. A Validatar Data Agent executes queries and Python scripts on separate hardware and streams results back. Agents can be grouped, and additional agents can be added to scale the processing layer horizontally without touching the application server.

When processing no longer happens inside the Validatar Server process, the server becomes a genuinely lightweight web and orchestration tier — and at that point a small instance is not merely acceptable, it is correct.

Improve what the tests themselves ask for.

• Include ORDER BY in test queries, or enable the Scripts Include Order By setting, so sorting happens in the source database rather than in server memory.
• In Python steps, prefer set-based logic over row-by-row loops, use streaming rather than reading entire files into memory, and be deliberate about when a full dataframe is genuinely required.
• Apply retention and purge settings, as described above.

Reduce cost without reducing capacity. If the driver is cost rather than performance, commitment-based pricing usually delivers a larger reduction than a downsize, with no performance impact at all. On AWS this is Savings Plans or Reserved Instances; on Azure it is Azure Reservations or the Azure savings plan for compute. For a server that runs continuously, this is normally the highest-value and lowest-risk lever available, and it is worth comparing directly against any proposed downsize.

What to measure

A sizing decision should rest on two sets of measurements. Infrastructure metrics tell you how hard the machine is working; Validatar metrics tell you what it was asked to do. Neither is sufficient alone.

Collect over a minimum of 14 days, and make sure the window covers a full business cycle — month-end close, and any seasonal peak in your source systems. Record peak and 95th percentile values, not just averages.

Infrastructure metrics

On Windows, the corresponding performance counters are Processor(_Total)\% Processor Time, Memory\% Committed Bytes In Use, Memory\Available MBytes, Memory\Pages/sec, Process(...)\Working Set - Private, and LogicalDisk\% Free Space. On Linux, the equivalents come from /proc/meminfo and standard system metrics.

Where to collect them:

Validatar metrics

Validatar already collects the workload profile you need. The full list is documented in Aggregated Server Statistics; these are the entries that bear directly on sizing.

Instance information — Average CPU Usage, Max CPU Usage, Average RAM Usage, Max RAM Usage, ProcessorCount

Execution metrics — the demand side

• Number of Jobs Executed, Standard Tests Executed, Template Tests Executed
• Average and Maximum Test Query Duration
• Average and Maximum Test Query Row Count — the single best predictor of memory demand. The maximum tells you the largest result set the server has had to hold and sort in memory
• Average and Maximum Data Source Ingestion Duration and Row Count
• Daily Python Count compared with Daily SQL Count — Python steps are the memory-hungry ones
• Number of Profile Sets Executed, Number of Errors

Scale metrics — Data Sources, Tables, Columns, Tests, Jobs, Users

Job-level baseline

This is the measurement that settles most sizing arguments. Choose one job that runs on a regular schedule and contains a realistic mix of standard tests, template tests, and any Python steps. From the Job Execution History, record the following for each run:

Capture both the whole-job wall-clock time and the individual test durations, for the reason described earlier: the difference between the two distinguishes a memory problem from a core-count problem.

Setting decision thresholds

Agree the criteria before making a change, so the outcome is a measurement rather than a matter of opinion.

As an illustration: a job of 100 tests that averages seven minutes before a change and eight minutes afterwards is a good outcome. The same job taking twenty minutes is not.

Changing the size safely

1. Instrument first. Make sure memory is actually being collected. Do this even if you decide to change nothing — without it you have no visibility into the metric that governs this workload.
2. Baseline for at least 14 days on the current configuration, capturing all three measurement sets above, across a full business cycle.
3. Schedule a maintenance window when no jobs are running, and confirm no scheduled job is due to start during the restart.
4. Make the change. On a virtual machine or cloud instance this is normally a stop, a resize, and a start — typically a short interruption, with no other configuration change required. Check your platform's documentation for any constraints on changing between instance families, and note that a cloud instance without a static or reserved IP address may receive a new public address on restart.
5. Observe for at least 14 days, capturing the same measurements.
6. Compare against your thresholds and keep, hold, or revert. Reverting is the same operation in the opposite direction.
7. Re-run your platform's right-sizing analysis afterwards, now with memory data included, and treat it as an ongoing input rather than a one-time verdict.

Related articles

• Job Execution Timings Report — the importable report behind the job-level baseline in this article
• Before You Begin — full system requirements and prerequisites
• Validatar Server Architecture — how the components fit together
• Performance Issues — diagnosing and resolving specific performance problems
• Validatar Data Agent — offloading query and script execution
• Aggregated Server Statistics — the full list of collected metrics
• Upgrading to a New Release Version — upgrade and rollback procedure