Dataface Tasks

Profiler surface in IDE extension is pilot-ready

IDM1-IDE-002
Statuscompleted
Priorityp0
Milestonem1-ft-analytics-analyst-pilot
Ownerui-design-frontend-dev
Completed bydave
Completed2026-03-24

Problem

The inspect workstream produces table profiles, column distributions, and schema metadata, but these outputs are not yet reliably rendered inside the VS Code/Cursor extension. The profiler panel either fails to load for certain datasets, shows stale data after re-profiling, or renders loading spinners indefinitely with no error message. Analysts cannot profile tables or inspect column distributions without leaving the editor to use a separate CLI or web interface, which breaks the integrated authoring flow. Error and empty states (permission failures, missing tables, timeout on large datasets) are also unhandled, leading to silent failures that erode trust.

Context

  • The inspect workstream's core backend is mature: profiler contract v1.1 with distributions (#281), spark bar charts (#282), and column comments (#386) all completed and tested.
  • The extension already has an inspector panel (inspector-panel.ts) with a 4-state machine (idle/loading/error/ready) and server-rendered HTML from the local dft serve server.
  • Data source: extension spawns a local dft serve process, reads profiles.yml for connection/dialect detection, and fetches server-rendered HTML via HTTP (/inspect/model/). Cached profiles live in target/inspect.json and are read by the server at render time.
  • Server lifecycle, retry logic, and CLI fallback exist in inspector-runtime.ts.
  • Error handling covered lock errors and missing tables but not permission/auth failures, connection errors, or navigation timeouts — leading to silent spinners.
  • Stale data after re-profiling could occur because HTTP fetches used identical URLs without cache-busting.

Acceptance mapping to inspect deliverables

Issue Feature Extension-exposed? Status
#281 Histogram bins + date distributions Yes — rendered by server in model/column templates Done (backend)
#282 Spark bar chart for profiler cards Yes — rendered in categorical/string column templates Done (backend)
#386 Database column comments Yes — fetched by inspector, rendered in templates Done (backend)

Open questions (resolved)

  1. Extension runs only against local workspace repos for M1 (no Cloud-connected projects).
  2. 60s fetch timeout per request; 120s for profiling POST. No soft cap beyond that for M1.
  3. All errors recover in-panel with actionable messages and retry/runDbt buttons — no reload required.

Possible Solutions

  • A — Full profiler parity with web/CLI for pilot tables: Highest effort; use only if IDE is M1-critical.
  • B — Read-only profile viewer with clear errors: Load inspect.json / API payload; show distributions and comments when present; hard-fail with human-readable message when not.
  • Recommended for M1 if IDE stays in scope: B first, then iterate toward A if analysts hit walls.
  • Selected: B — the extension renders server-produced HTML (all #281/#282/#386 features come free from the backend). Focus implementation on error/loading/stale resilience.

Plan

  • [x] Confirm data source: extension uses local dft serve with profiles.yml connection detection.
  • [x] Add permission/auth error detection (isPermissionError, isConnectionError) to inspector-errors.ts.
  • [x] Add connection error detection and user-friendly messages for BigQuery 403, Snowflake auth, connection refused.
  • [x] Add 60s fetch timeout to navigateInspectFromQuery (was missing — could spin forever).
  • [x] Add 120s timeout to submitInspectForm POST (profiling can be slow but must not hang).
  • [x] Add _ts cache-bust param to inspect GET requests to prevent stale data after re-profiling.
  • [x] Fix unreachable fallthrough in inspectViaServer retry loop — now shows error instead of silent return false.
  • [x] Write 12 new tests covering permission errors, connection errors, auth failures, and false-positive guards.
  • [x] Run full extension test suite (96/96 pass).
  • [x] Map acceptance to inspect deliverables: #281, #282, #386 — all extension-exposed via server templates (see Context table).
  • [ ] Publish short analyst guide for profiler inside extension (deferred to PILOT_RUNBOOK updates).

Implementation Progress

  • inspect-profiler issues #281, #282, #386
  • M1 IDE pilot acceptance checklist: task-m1-ide-extension-usability-hardening + apps/ide/vscode-extension/PILOT_RUNBOOK.md (M1-IDE-001 cancelled as redundant wrapper).
  • 2026-03-23: Added solutions, payload/source question, and acceptance mapping.
  • 2026-03-23: in_progress — execution started alongside M1-IDE-003.
  • 2026-03-23: Implementation complete — error resilience hardening:
  • inspector-errors.ts: Added isPermissionError() (access denied, 403, insufficient privileges) and isConnectionError() (connection refused, auth failure, network unreachable). formatInspectError() now routes these to actionable messages with credential/connectivity guidance.
  • inspector-panel.ts: Added 60s AbortController timeout to navigateInspectFromQuery(), 120s timeout to submitInspectForm(), _ts cache-bust param to prevent stale GET responses, and defensive error fallback for the retry loop exit path.
  • inspector-panel.test.ts: 12 new tests (17 total) covering permission errors, BigQuery 403, connection refused, Snowflake auth, and false-positive guards.
  • All 96 extension unit tests pass (7 test files).
  • 2026-03-23: Focused validation rerun — 96/96 tests pass on rebased branch. Acceptance mapping confirmed complete in Context table.

Review Feedback

  • [ ] Review cleared