- 30 Sep 2022
- 2 Minutes to read
Creating a Data Source
- Updated on 30 Sep 2022
- 2 Minutes to read
Creating a data source is one of the first steps you'll take in Validatar as a data source provides the connection to the data you want to learn more about and execute tests against.
Before creating a data source, you'll want to modify your data environments list. A data environment is a way to distinguish similar data sources. Most often, users have Development, QA or Test, and Production environments. To add environments:
- Navigate to Settings > Configuration > Data Environments.
- Click + Add Environment and enter each name.
- Click Save.
Creating a New Data Source
After you have your environments added, you can start creating data sources. A user must be a Global or Data Source Admin to create a data source.
- Navigate to Settings > Data Sources.
- Click + New Data Source.
- Enter a name for your data source. Use something that's easily identifiable by your entire team.
- Select the Environment from the dropdown.
- Enter a data source description (optional).
Creating the Connection
You will have to enter a connection string or Python script and parameters to connect to data. This is referred to as your Primary Connection. To set up the connection, click Edit.
SQL Based Connections
- Choose the appropriate connection type.
- Choose the associated Data Source Template or choose No Assigned Template.
- Choose the Data Agent Group or choose No Assigned Group.
- Enter the connection string. Use the Connection String Builder for assistance.
- Use the secure text box field to enter the Password.
- Test the connection.
Validatar has four connection types: SQL Server, ODBC, OLE DB, and a Python Script. Note: Make sure the appropriate drivers are installed on the web server before creating the connection. Also, if you'd rather use a DSN for the ODBC connection, be sure to create the DSN using the ODBC Data Source (64-bit).
Data Source Templates help Validatar pull metadata from the Data Source. You may have to create or import a Data Source Template before using the data source in Validatar.
Data Agents allow Validatar to connect to data that is not stored locally and isn't directly accessible. View the articles on Connecting to Remote Data to learn more about Data Agent installation and the prerequisites required before creating a connection to a remote data source. Only select a Data Agent if you need to connect to data remotely.
The Connection String Builder uses the selected Connection Type to help you create the connection string. If your connection string requires any additional parameters, exit the Connection String Builder, and add them to the configuration box.
Python Based Connections
To connect to a flat file using Python, select Python Script as the Connection Type. You can find a list of Python-based Data Source Templates in the Marketplace or create your own in Settings. The example above uses the AWS S3 Data Source Template found in the Marketplace.
Script Parameter names are populated based on the default settings in the associated Data Source Template therefore they can change depending on which template is selected. Complete the respective parameter values based on the data agent fields.
Note: Python execution requires the use of a data agent because you can not directly execute scripts on Validatar Server or Cloud.
Pre- and Post-Execution Scripts are exactly as they sound, script elements that are run before and after a script that is executed against this data source. This applies to tests, metadata ingestion, and custom field ingestion.