Adoption hardening for internal teams

Problem

The profiling pipeline was built and validated against a single team's BigQuery warehouse during the M1 pilot. Expanding to multiple internal teams and first design partners exposes assumptions that don't generalize: different warehouse dialects (Snowflake, Redshift) may produce incomplete statistics, schemas with hundreds of tables exceed current memory and timeout limits, permission models vary across organizations, and the inspect.json cache format may not handle multi-user concurrent writes. Without hardening the pipeline for diverse warehouse environments, varied schema scales, and multi-team usage patterns, adoption beyond the initial pilot team will produce confusing failures and erode trust in the tool.

Context

• Pilot work proved the baseline flow for warehouse profiling, semantic inference, and analyst-facing inspect/context artifacts, but repeated use by multiple internal teams and first design partners will surface reliability and UX edge cases that the pilot could work around manually.
• This milestone is about turning a promising path into something teams can use predictably without bespoke engineering help, support heroics, or undocumented workarounds.
• Expected touchpoints include dataface/core/inspect/, schema-context consumers, inspect docs, and core tests, plus any onboarding, validation, and issue-triage surfaces that repeatedly fail during broader adoption.

Possible Solutions

• A - Fix only the currently visible blockers: fast, but it tends to preserve hidden fragility and leaves the next round of adopters to rediscover the same classes of issues.
• B - Recommended: define an adoption-hardening checklist and close the highest-impact gaps: combine real usage review, targeted fixes, docs/runbook updates, and validation so repeated use becomes predictable.
• C - Pause broader adoption until a larger redesign is finished: reduces short-term support load, but delays the real feedback this milestone is meant to capture.

Plan

1. Review prototype/pilot outputs, open issues, and support friction to identify the top blockers to repeated team adoption of warehouse profiling, semantic inference, and analyst-facing inspect/context artifacts.
2. Turn those findings into a prioritized hardening scope that covers product behavior, docs/runbooks, and validation for the highest-risk flows.
3. Land or queue the highest-impact fixes and document explicit known limits for anything intentionally left out of scope.
4. Re-run representative multi-team or design-partner scenarios and update follow-up tasks based on what still fails or feels too brittle.

Implementation Progress

Review Feedback

• [ ] Review cleared