Source intake
Discover source files, calculate file hashes, infer schema signatures, and assign registry/run identifiers.
Source filesManifestSchema version grouping
Pipeline model
Responsibilities
Discover source files, calculate file hashes, infer schema signatures, and assign registry/run identifiers.
Load raw values into schema-version grouped bronze tables. Unknown or unmapped structures are quarantined, not silently dropped.
Map source fields to canonical columns using approved dictionary aliases, match rules, type policy, and transform rules.
Select reviewed silver fields into a flat analytics contract with business rules, filters, and coverage checks.
Validate generated contracts and promote approved metadata or runtime changes through pull requests.