Syntrace - trace the break, ship the fix

Schema drift breaks pipelines.
Syntrace heals them.

Syntrace detects breaking schema changes in a live DataHub catalog, traces the column-level blast radius, ships the fix as a pull request - and writes the healed lineage back to the catalog.

Launch App View on GitHub
No mocks - live DataHub Apache-2.0 stdlib-only core
How it works

Six steps from drift to a healed catalog

One run of python3 -m src.main run - every read and every write goes to the live DataHub instance.

  1. 1
    Detect drift

    Diff the two latest versioned schemaMetadata aspects over OpenAPI v3: renames, drops, adds, type changes.

  2. 2
    Trace lineage

    Ask the DataHub MCP Server for each drifted column’s downstream edges - column-level, with automatic GraphQL fallback.

  3. 3
    Analyze impact

    Walk the graph hop by hop to the exact columns of each affected dbt model and Airflow DAG, with severity per asset.

  4. 4
    Generate fixes

    Deterministic rules rewrite the dbt SQL and DAG code: renames propagate at any depth, dropped columns are stubbed.

  5. 5
    Prepare the PR

    One branch, one fix per file, a readable impact report in the body. Dry-run by default; live only with --create-pr.

  6. 6
    Write back

    Healed lineage, renamed schema fields, syntrace-remediated tags, and a remediation note - four surfaces, one consistent catalog.

Features

The metadata does the heavy lifting

Everything below is what the code in the repo actually does - verified against DataHub Core v1.7.0.

DataHub MCP Server, first-class

Agent context is read over the official mcp-server-datahub (stdio): get_lineage with column filtering plus get_entities - and the remediation tag is written back through the MCP add_tags mutation tool.

Column-level blast radius

Fine-grained lineage maps each drifted column to the exact downstream columns it feeds - severity and hop count per asset, not just “something downstream broke.”

Deterministic fixes

Rules R1–R5 rewrite dbt SQL and Airflow DAGs: renames propagated at any depth, dropped columns stubbed to preserve the downstream contract. An LLM pass is optional (--llm), never required.

Self-healing PRs

One branch, one fix per file, and a human-readable impact report in the PR body. Dry-run by default - a real PR opens only with --create-pr and GitHub credentials.

4-surface catalog write-back

Remediated fine-grained lineage, renamed schema fields, syntrace-remediated tags, and an idempotent remediation note in each asset’s documentation - the catalog stays consistent with the fix.

Zero-dependency core

Detection, lineage analysis, codegen, the web dashboard, and the CLI run on the Python standard library alone - nothing to install. The DataHub SDK is used only by the seed script.

Audit log

The write-back, as it runs

An excerpt from the audit log of a real run against the live DataHub quickstart.

Lineage and schema upserts go over OpenAPI v3; the tag attach goes over the MCP add_tags mutation tool.

Quickstart

Three steps to a healed catalog

You need Docker with ~8 GB of RAM and any Python 3.10+ for the seed venv - the Syntrace core itself runs on the standard library alone.

1

Clone the repo and start DataHub

The DataHub quickstart brings up GMS on :8080 and the UI on :9002.

git clone https://github.com/mrnetwork0001/Syntrace.git && cd Syntrace
python3.12 -m venv .venv  # any Python 3.10+
.venv/bin/pip install acryl-datahub mcp-server-datahub
.venv/bin/datahub docker quickstart
2

Seed the demo scenario

Emits the real raw.orders dataset, dbt models, and Airflow DAG into the live catalog, then applies the v2 schema drift.

.venv/bin/python scripts/seed_datahub.py --stage baseline
.venv/bin/python scripts/seed_datahub.py --stage drift
3

Run Syntrace

CLI or web dashboard - both drive the identical stdlib-only pipeline.

python3 -m src.main run  # CLI pipeline
python3 -m src.main ui   # dashboard at http://127.0.0.1:8642/app