Overview
Validatar tests Power BI through the Power BI Service Data Source Template. Most of what goes wrong in Power BI can be tested on any license. Rendering reports needs capacity, and that includes rendering a report as a specific user to test row-level security. This page shows which checks need what, and lists what to ask your Power BI administrator for.
Checks by tier
| Check | Any license | Contributor role | Capacity |
|---|---|---|---|
| Workspaces, semantic models, reports, pages and dashboards in the catalog | Yes | ||
| Model tables, columns and measures in the catalog | Yes | Models with row-level security | |
| Refresh succeeded, ran recently, ran on schedule | Yes | ||
| DirectQuery source still answers | Yes | ||
| Model parameters point at production | Yes | ||
| Totals and breakdowns match the warehouse (DAX against SQL) | Yes | ||
| Orphan keys, duplicate keys, broken measures, blank attributes | Yes | ||
| Column profiling and measure totals in profile history | Yes | ||
| Columns, measures and relationships unchanged since the last run | Yes | ||
| Measure expressions unchanged | Yes | ||
| Visuals point at fields that still exist | Yes | ||
| Row-level security role filters as approved, every fact table covered | Yes | ||
| Page renders, not blank, no Power BI error text | Yes | ||
| A card or chart shows the same number as the model or warehouse (OCR) | Yes | ||
| Page or visual unchanged since recent runs, brand colours present | Yes | ||
| Paginated report data | Yes | ||
| Report rendered as a user shows only that user's rows | Yes, plus Contributor |
- Any license means Pro (shared capacity), Premium Per User, Fabric, Premium or Embedded.
- Contributor role means the service principal is Contributor, Member or Admin on the workspace. It is the role we recommend in every case.
- Capacity means the workspace sits on Fabric (F), Premium (P) or Embedded (A, EM) capacity.
Two things that surprise people
Premium Per User cannot export. Microsoft's Export To File API, which every rendering check uses, does not support Premium Per User workspaces. A workspace on Premium Per User behaves like Pro for these tests. Rendering needs a Fabric, Premium or Embedded capacity, and every semantic model the report uses must be on capacity too. A Fabric capacity can be paused when it is not in use.
A service principal cannot query a model that has row-level security. Power BI refuses DAX from a service principal against any model with row-level security or single sign-on, whichever user it names. For those models Validatar reads the role definitions (Contributor) and renders reports as a user (capacity).
What to ask your Power BI administrator for
- An app registration with a client secret, and its service principal in a security group. The app needs no Power BI API permissions. Remove
Tenant.Read.AllorTenant.ReadWrite.Allif it has them. - Tenant settings for that security group: Service principals can call Fabric public APIs (Developer settings) and Semantic Model Execute Queries REST API (Integration settings). For rendering, also Export reports as PowerPoint presentations or PDF documents, and optionally Export reports as image files.
- The service principal as Contributor on each workspace to be tested.
- For rendering: the workspaces on Fabric, Premium or Embedded capacity, and reports without a sensitivity label, because a service principal cannot export a labelled report to PDF.
The Prerequisites section of the Power BI Service Data Source Template article has the detail behind each item.