Overview
The SFTP data source template enables Validatar to discover and profile data files on remote servers accessible via SFTP (SSH File Transfer Protocol). It uses Python scripts with the paramiko library to connect, list files, and read data for metadata discovery and profiling.
Platform: SFTP servers
Connection Category: Script
Template Category: Marketplace
What's Included
Default Parameters
| Parameter | Type | Description |
|---|---|---|
hostname |
String | SFTP server hostname or IP |
port |
Integer | SFTP port (default: 22) |
username |
String | SFTP username |
password |
Secret | SFTP password |
private_key_path |
String | Path to SSH private key (alternative to password) |
remote_directory |
String | Root directory to scan |
file_pattern |
String | Glob pattern for file matching |
Data Type Mappings
Maps inferred types from file headers (CSV, delimited files).
Metadata Ingestion
The ingestion script:
- Connects to the SFTP server
- Lists files matching the pattern in the remote directory
- Downloads file headers to discover column names
- Remote directories become schemas, files become tables
Profiling
The profiling script downloads sample data from remote files and calculates standard metrics.
Installation
Customization
- Key-based authentication — Configure SSH key authentication instead of passwords
- Jump hosts — Modify the script for bastion/jump host configurations
- Compressed files — Extend to handle gzip or zip compressed files