feat: chart decisions Phase 4 — SQLGlot column lineage
Problem
Chart decisions currently lack visibility into column-level dependencies — when a SQL query transforms, renames, or derives columns, the system has no way to trace which source columns feed into each output column. This means AI agents and chart rendering logic cannot reason about upstream data provenance, leading to incorrect column selections, broken references when upstream queries change, and missed opportunities to surface lineage-based context (e.g., "this metric is derived from columns X and Y in table Z").
Context
- Column-lineage extraction runs reliably on target SQL patterns and warehouses.
- Chart decision logic can consume lineage metadata for ranking/guardrails.
- Lineage quality metrics and fallback behavior are visible for debugging.
Possible Solutions
- A - Keep chart decisions dependent only on runtime result metadata: simplest, but misses source-column semantics.
- B - Recommended: add SQLGlot-derived column lineage as an optional chart-decision input: propagate source-column context into decision logic where it materially helps.
- C - Build a broader lineage subsystem first: potentially useful later, but too much prerequisite work for this phase.
Plan
- Wire SQLGlot lineage extraction into context/catalog processing pipeline.
- Define lineage payload contract consumed by chart decision components.
- Implement graceful fallback for unsupported SQL constructs.
- Add tests with representative transformation patterns and edge cases.
- Document lineage-powered decision behaviors and known coverage boundaries.
Implementation Progress
- GitHub issue: https://github.com/fivetran/dataface/issues/377
Review Feedback
- [ ] Review cleared