Local visual regression artifacts, PR/task links, and task-serve Viz changes timeline

Problem

Rework visual regression approval: move away from Linux-only CI as the sole source of truth and toward local runs that produce a single review artifact with diffs or summary, linked from the PR and from task frontmatter. Non-blocking means no separate visual-approvals-style environment gate; reviewers still explicitly attest via the normal review flow. Goal is fast viewing of visual regressions plus a clear paper trail. Links should open a rendered view in task serve for easy scanning.

PR coupling: During the PR process, generate that review file and include it on the PR (e.g. commit on the PR branch) so the artifact lives with the code change and task serve / GitHub can resolve it to a specific PR number and revision.

Review timing and checklist: Ideally the report exists before human or agent review starts (author generates/commits it when snapshots change). The PR review surface should include checkboxes, for example: (1) I opened and reviewed the visual change report linked in this PR; (2) Visual differences are expected, intentional, and acceptable for this change. Wire these into .github/pull_request_template.md / lite template and any automated review checklist (e.g. RevBot) as appropriate.

Add a primary left nav tab Viz changes on the task serve UI with a nested second sidebar listing a chronological timeline of visual change records. Selecting an item loads the main pane with title, PR metadata and links, linked tasks, and the embedded or rendered change file so regressions can be traced over time.

Open decisions: exact path and naming for the review file, handling macOS versus Linux SVG drift if local is canonical, and how to index timeline entries via git, frontmatter field, or manifest.

Context

• Today: golden SVGs under tests/visual/snapshots/, Linux as CI source of truth; see docs/docs/contributing/visual-tests.md and just check-visuals / just diff-visuals.
• Human-gated approval workflow: .github/workflows/approve-visuals.yml and visual-approvals environment (see visual-tests doc).
• Task serve UI (activity page, side nav) is the surface for the new Viz changes tab and nested timeline sidebar.
• The viz review artifact should be first-class on the PR: same lifecycle as the branch (regenerate/update when snapshots change), so reviewers and the timeline can always anchor on PR + SHA.
• PR review templates: .github/pull_request_template.md and .github/pull_request_template_lite.md; align checkbox wording with RevBot / just review expectations if those checklists are enforced.

Possible Solutions

A. Markdown report committed on branch (Recommended)

Generate a single .visual-changes.md report under .visual-review/ on the PR branch. The report lists changed/new/deleted snapshots with inline SVG diffs (text-based). Indexed by git history for the task-serve timeline.

Pros: portable, git-native, easy to render in task serve and on GitHub, no external infra. Cons: file can be large if many snapshots change (mitigated by summary-only mode).

B. HTML report with embedded SVG side-by-side

Similar to A but produces an HTML file with <img> or inline SVG for visual comparison.

Pros: true visual diff in browser. Cons: larger file size, harder to review as raw diff, more rendering complexity in task serve.

C. External artifact store (S3/GCS) with links

Upload images to cloud storage, link from PR body.

Pros: no repo bloat. Cons: requires infra, breaks offline/local workflow, adds external dependency.

Decision: Option A — Markdown report committed on branch. Minimal, git-native, renders in task serve and GitHub. Can evolve to HTML later if needed.

Plan

Files to modify/create

1. scripts/visual-report — New script: runs visual tests, collects changed snapshots, generates .visual-review/report.md with diff summary and metadata (branch, commit, date, changed files list).
2. .github/pull_request_template.md and _lite.md — Add visual review checkboxes.
3. tasks/tools/tasks_server.py — Add /viz-changes route and /viz-changes/{entry} detail route.
4. tasks/tools/tasks_mkdocs_nav.py — No changes needed; the Viz changes tab uses a dedicated top-level route, not mkdocs nav.
5. tasks/mkdocs.yml — Add Viz Changes to nav.
6. docs/docs/contributing/visual-tests.md — Update to document the new local artifact workflow.

Implementation steps

1. Create scripts/visual-report that: - Runs just check-visuals or reads existing .actual.svg files - Compares against golden snapshots - Generates .visual-review/report.md with frontmatter (branch, commit SHA, date, PR number if known) - Lists each changed snapshot: name, status (changed/new/deleted), text diff summary
2. Add visual review checkboxes to both PR templates.
3. Add viz-changes route to task serve that: - Scans git log for commits touching .visual-review/report.md - Renders a timeline sidebar with entries by date/PR - Detail view shows the report content with PR links and task links
4. Wire PR/task linking: scripts/pr-create already updates task frontmatter with pr_url/pr_number; the visual report frontmatter includes the branch name so task serve can correlate.
5. Update visual tests docs.

Implementation Progress

Created files

• scripts/visual-report — Python script that compares tests/visual/snapshots/ against a base ref (default origin/main) using git diff --name-status. Generates .visual-review/report.md with YAML frontmatter (branch, commit, date, change counts) and per-file breakdown with text diffs in collapsible <details> blocks. Supports --check mode for CI (exit 1 if changes, no file written).
• tasks/tools/viz_changes.py — Module for the Viz Changes timeline in task serve. collect_viz_changes() walks git log for commits touching .visual-review/report.md, parses report frontmatter at each commit. render_timeline_html() and render_detail_html() produce the timeline list and detail views.

Modified files

• justfile — Added visual-report recipe.
• .github/pull_request_template.md and _lite.md — Added "Visual Review" section with two checkboxes: (1) reviewed the visual change report, (2) diffs are expected/intentional/acceptable.
• tasks/tools/tasks_server.py — Added /viz-changes (timeline) and /viz-changes/{commit_sha} (detail) routes. Added "Viz Changes" link to browse nav bar.
• tasks/tools/tasks_mkdocs_nav.py — Added viz_active parameter to render_nav_sidebar() and "Viz Changes" link in the left sidebar nav.
• docs/docs/contributing/visual-tests.md — Added "Visual Change Reports" section documenting the new workflow, commands, and task-serve integration.

Key decisions

• Markdown report format over HTML: simpler, git-friendly, renders in both GitHub and task serve.
• Git-native indexing: timeline entries are discovered by git log on the report file path, no separate manifest needed.
• Report frontmatter stores branch/commit/base/counts so task serve can show metadata without parsing the full markdown.
• PR linking: report subject line often contains (#NNN) from squash-merge; _pr_number_from_subject() extracts it for links.

QA Exploration

N/A — The Viz Changes tab is a read-only server-rendered page. The visual-report script is a CLI tool. Both are validated by the test suite and lint checks below. Browser QA would require an active task serve instance with committed visual reports in git history, which doesn't exist yet on this branch.

• [x] QA exploration completed (or N/A for non-UI tasks)

Review Feedback

• [ ] Review cleared