Ensure dbt generate_schema_name-safe BigQuery ref source resolution
Problem
Dataface currently resolves dbt ref() and source() calls via a lightweight manifest parser that reconstructs identifiers as schema.alias (and for sources schema.table) instead of using dbt's full relation identity (relation_name or adapter relation object). This is risky for BigQuery because project/dataset/table identity and quoting semantics can be lost.
For ~/Fivetran/analytics/dbt_ft_prod, where generate_schema_name overrides are used, we need to guarantee Dataface query execution respects the same dbt relation naming outcome that dbt would use for BigQuery.
Context
dataface/core/execute/adapters/dbt_utils.pycurrently resolvesref/sourceusingschema+alias/name, notrelation_name.dataface/core/serve/server.pyregisters adapters in order:DbtAdapterthenSqlAdapter.DbtAdapter._can_execute()declines SQL queries with inline source dicts or profile refs; those are routed toSqlAdapter.- In executor flow (
dataface/core/execute/executor.py), named sources fromdataface.ymlare resolved to full source dicts before adapter execution, which means analytics-styletype: bigquerysource usage often bypassesDbtAdapter. ~/Fivetran/analytics/dbt_ft_prod/macros/dbt_overrides/generate_schema_name.sqlexists and is active in that repo.- Current consequence: Dataface may not always use the same adapter-level relation resolution semantics as dbt for BigQuery queries in analytics/dbt installs.
Possible Solutions
- Recommended: Make manifest resolution relation-first (
relation_name) and BigQuery-safe - Updateresolve_dbt_refs()to preferrelation_namefrom manifest nodes/sources, with robust fallback only when absent. - Ensure quoting/identifier shape is preserved for BigQuery (project.dataset.table). - Pros: minimal architectural change, aligns with dbt artifact truth, works for both adapters. - Cons: still artifact-driven (depends on manifest freshness). - Force all dbt SQL through dbt adapter execution path - Pros: closest runtime behavior to dbt internals. - Cons: larger change, not compatible with all current source-config override flows.
- Keep current behavior and document limitations - Pros: no implementation effort. - Cons: known correctness risk for BigQuery naming semantics and analytics parity.
Plan
- Replace ref/source identifier construction in
dataface/core/execute/adapters/dbt_utils.py: - Prefer manifestrelation_name. - Use db/schema/alias fallback only whenrelation_nameis unavailable. - Add tests in adapter unit coverage:
- BigQuery manifest fixture with project+dataset+table relation names.
- Verify
ref/sourceexpansion preserves full relation identity. - Verify fallback path behavior remains stable. - Add an analytics compatibility check:
- Document and test expected behavior for dbt repos with
generate_schema_nameoverrides. - Validate both execution paths:
-
DbtAdapterpath. -SqlAdapterpath after source/profile normalization. - Update docs/troubleshooting with manifest freshness and relation resolution notes.
Implementation Progress
- 2026-03-23: Investigation complete.
- Confirmed current resolution code uses
schema.aliasand can drop full relation identity details. - Confirmed analytics repo has
generate_schema_nameoverrides indbt_ft_prod. - Confirmed source-config queries can bypass
DbtAdapterand run inSqlAdapter. - 2026-03-23: Implementation complete (Solution 1 — relation-first resolution).
- Updated
resolve_dbt_refs()indbt_utils.py: bothresolve_refandresolve_sourcenow preferrelation_namefrom the manifest when present, withschema.alias/schema.tablefallback. - Added
TestResolveDbtRefsclass (6 tests) intests/core/test_adapters.pycovering: relation_name preference for ref and source, fallback paths, BigQuery three-part names, andgenerate_schema_nameoverride scenarios. - All 54 adapter tests pass, zero regressions.
QA Exploration
- [x] QA exploration completed (or N/A for non-UI tasks)
- N/A: non-UI task (execution adapter correctness + tests).
Review Feedback
- [ ] Review cleared