GitHub - tjirab/tff: Guided evolution for data transformation projects
Fast, zero-warehouse-cost architectural linter & health dashboard for dbt, SQLMesh & Dataform.
[](https://github.com/tjirab/tff#fast-zero-warehouse-cost-architectural-linter--health-dashboard-for-dbt-sqlmesh--dataform) **Catch illegal upstream joins, layer violations, and duplicate transformation logic in CI.**
SQL linters check syntax and formatting in individual files, but they do not analyze the transformation graph. **tff** evaluates transformation models holistically—enforcing layer boundaries, domain contracts, and logic deduplication across the entire data warehouse DAG.
``` $ tff check
TFF ARCHITECTURE AUDIT 6 models · 4 errors · 4 warnings · 0.05s
STATUS LOCATION RULE COUPLING ────────────────────────────────────────────────────────────────────────────── WRN models/core/layer_violation.sql duplicate_ctes algorithm
- CTE 'cleaned_users' duplicates transformation logic with 3 other models.
ERR models/core/users.sql banselectstar name ! SELECT * is prohibited. Explicitly name your columns to reduce coupling.
ERR models/core/users.sql nomissingowner metadata ! Model owner should always be specified.
WRN models/marts/finance/finance_stats.sql duplicate_ctes algorithm
- CTE 'cleaned_users' duplicates transformation logic with 3 other models.
ERR models/marts/marketing/marketing_all_users.sql layer_integrity dynamic ! marts/marketing depends on sqlmesh_example.finance_stats (marts/finance)
WRN models/marts/marketing/marketing_all_users.sql duplicate_ctes algorithm
- CTE 'marketing_cleaned_users' duplicates transformation logic.
ERR models/marts/marketing/marketing_type_violation.sql join_type_parity type ! Join condition 'o.user_id = u.user_id' compares integer with text (CoT). ────────────────────────────────────────────────────────────────────────────── FAIL — 4 errors block merge. Run `tff --fix` for auto-correctable rules. ```
- * *
Quickstart (Zero Configuration)
[](https://github.com/tjirab/tff#quickstart-zero-configuration) Run `tff` inside any existing dbt, SQLMesh, or Dataform repository. **No configuration file required**—`tff` automatically infers standard layer conventions (`staging` → `intermediate` → `core` → `marts`) and immediately audits your DAG:
Instant zero-install invocation via uvx:
uvx --from "tff-core[dbt]" tff check # for dbt
uvx --from "tff-core[sqlmesh]" tff check # for SQLMesh
uvx --from "tff-core[dataform]" tff check # for Dataform
Or install for your pipeline framework:
pip install "tff-core[dbt]" # for dbt (or: uv add "tff-core[dbt]")
pip install "tff-core[sqlmesh]" # for SQLMesh
pip install "tff-core[dataform]" # for Dataform
Catch layer violations, duplicate CTEs, and circular dependencies
tff check
Calculate your repository architecture health score (0–100)
tff health
- * *
Demo
[](https://github.com/tjirab/tff#demo)
- Terminal audit demo recording (GIF) — live capture of layer integrity enforcement and duplicate CTE detection.
- Terminal demo recording suite — automated VHS session scripts across dbt, SQLMesh, and health index reporting.
- * *
Architecture & Capabilities
[](https://github.com/tjirab/tff#architecture--capabilities) | Capability | Specification & Function | | --- | --- | | **Layer Boundary Integrity** | Enforces directional DAG constraints (`staging` → `intermediate` → `core` → `marts`) and blocks unauthorized cross-mart coupling. | | **Duplicate CTE Detection** | Identifies identical transformation sub-queries across independent models (Connascence of Algorithm) to drive upstream consolidation. | | **Model Health Scoring** | Computes an objective repository health metric (0–100) and gates CI builds via `tff health --fail-under 80`. | | **CI/CD Pipeline Governance** | Official GitHub Action (`tjirab/tff@v1`) diffs PR changes against base branches, gates merges, and generates inline annotations. | | **Automated Remediation** | Safely synchronizes model name declarations to file stems via `tff lint --fix`. | | **Multi-Engine Support** | Native semantic analysis across dbt, SQLMesh, and Google Cloud Dataform projects without warehouse connection overhead. |
- * *
Documentation
[](https://github.com/tjirab/tff#documentation) Complete specifications, configuration guides, and architectural documentation are available at **tff.readthedocs.io**:
- **Getting Started & Integrations** — Guides for dbt, SQLMesh, and Dataform.
- **Rules & Checks Reference** — Catalog of architectural checks and linter rules.
- **CLI Reference Guide** — Command-line interface reference, options, and output formats (SARIF, JSON, HTML).
- **CI/CD & GitHub Actions** — Automated PR governance with `tjirab/tff@v1` and pre-commit hooks.
- **Extending tff** — Authoring custom rules, checks, and plugins.
- **Architecture Decision Records (ADRs)** — Technical decision log and architecture decision records.
- **GitLab Case Study** — Architectural audit of 2,200+ models in enterprise dbt pipelines.
- **Style Guide & Design System** — Functional design principles, typography, and documentation standards.
- * *
License
[](https://github.com/tjirab/tff#license) Distributed under the MIT License.