IDE extension usability hardening for internal analyst pilot

Problem

The extension's YAML editing, validation, and preview workflows have usability gaps that block non-engineer analysts from completing routine tasks independently. Error messages are developer-oriented (raw stack traces or generic "invalid YAML"), the preview pane doesn't indicate what went wrong when rendering fails, and recovering from errors often requires reloading the extension window. Startup performance on large dbt projects is also slow enough to frustrate impatient users. These issues compound: an analyst who hits two or three confusing failures in a session will stop using the extension entirely. The UX needs to reach a threshold where pilot analysts can complete a weekly dashboard workflow and report it as usable.

Context

• Top pilot friction points are removed across authoring, preview, and diagnostics flows.
• Extension startup/performance and failure messaging meet pilot readiness thresholds.
• At least three pilot analysts complete weekly workflows and give explicit thumbs-up.
• M1 pilot scope (from cancelled M1-IDE-001): Ship IDE + Suite; VS Code and Cursor are the same extension surface; runbook covers workflow through save to disk (git is out of band).

Possible Solutions

• A — Full UX polish pass: Interviews + fix-all; too broad for M1 unless IDE is flagship.
• B — Top-N friction list (recommended): Pick 3–5 failure modes (e.g. YAML error opacity, preview desync, slow startup, missing “what do I do next”) and fix those with tests.
• C — Docs-only mitigations: Runbooks and GIFs; only if paired with B for true blockers.

Plan

• Run a 60–90 min structured session with 1–2 internal analysts (not engineers); capture friction log with severity (P0/P1/P2).
• Triage into: (1) must-fix for M1 pilot, (2) defer, (3) document workaround.
• Implement highest-impact fixes with regression tests where behavior is stable (e.g. diagnostic messages, preview refresh on save).
• Add in-product guidance for top 3 compiler/render errors (link to doc or one-line fix hint).
• Publish pilot runbook + known issues with owners/dates.
• M1 IDE acceptance (absorbed from cancelled M1-IDE-001): Run apps/ide/vscode-extension/PILOT_RUNBOOK.md in VS Code and Cursor on a real dbt repo; record pass/fail and gaps in this task’s Implementation Progress.

Likely P0 candidates (validate in session)

1. Preview does not refresh after save / shows stale chart.
2. Validation errors do not point to line + fix (raw trace or “invalid”).
3. Extension window reload required to recover from recoverable errors.
4. Cold start on large monorepo > N s with no progress UI.

Dependencies / ordering

• Proceed in parallel with task-m1-ide-profiler-surface-pilot-ready where UI does not overlap. M1-IDE-001 (wrapper) was cancelled; this task owns pilot runbook + acceptance smoke.

Implementation Progress

• cloud suite pilot onboarding flows
• 2026-03-23: Added friction-based plan, P0 candidates, and ordering vs other IDE tasks.
• 2026-03-23: in_progress — absorbs cancelled M1-IDE-001 acceptance; PILOT_RUNBOOK.md is the checklist. Running in parallel with M1-IDE-002.
• 2026-03-23: Implemented P0 fixes (Option B — top-N friction list): 1. Stale preview (P0-1): Fixed per-document debounce — refreshTimeout was a single global timer; changes to one document could cancel another document's pending refresh. Now uses a per-URI Map<string, Timeout>. 2. Actionable error hints (P0-2): formatPreviewError() now returns a fixHint field with one-line guidance for undefined variables, invalid chart types, YAML syntax errors, and missing query references. Displayed as "How to fix:" in the preview error page. 3. Error recovery (P0-3): Error fallback rendering was using a fake { cspSource: 'self' } webview, which broke the CSP policy for toolbar buttons (refresh, open in browser). Now passes the real panel.webview so the refresh button works after errors — no extension window reload needed. 4. Loading indicator (P0-4): Preview panel now shows an animated loading indicator with the file name while the CLI renders (up to 30s). Only shown on first render or when panel was already in loading state.
• All fixes covered by 5 new regression tests (89 total, all passing). TypeScript compiles cleanly.

QA Exploration

N/A — This is a code-only change to preview-manager.ts (TypeScript extension logic). The fixes target runtime webview behavior that cannot be exercised by the Playwright MCP inspector (it requires a live VS Code extension host). Covered by unit tests; manual pilot validation via PILOT_RUNBOOK.md.

Review Feedback

• [ ] Review cleared