Audit cloud test coverage and reduce over-mocking
Problem
The current Cloud test suite leaned heavily on monkeypatched unit tests for view
logic and helper functions. That kept tests fast, but it also left a blind
spot: fake querysets and patched render/context helpers could validate the
shape of a function call while missing real Django ORM, queryset, URL, and
template failures. The /piedpiper/ crash was a concrete example: a real
Dashboard queryset raised on order_by("-updated_at"), but the existing test
passed because it used a fake queryset that accepted any field name.
Context
- Recent bug:
apps/cloud/apps/chat/dashboard_cards.pyorderedDashboardby-updated_at, butapps/cloud/apps/projects/models.py::Dashboardonly haslast_synced.- The failure surfaced only when loading the real org home/chat page.
- Existing Cloud tests were concentrated in monkeypatched request/view tests:
tests/cloud/test_org_chat_home.pytests/cloud/test_chat_persistence.pytests/cloud/test_dashboard_embed.py- Constraints:
- Keep the suite fast enough for regular local use and PR gating.
- Prefer the smallest realistic harness that exercises real Django ORM and template rendering on critical paths.
- Avoid jumping straight to a broad browser suite for server-rendered page-load failures.
Possible Solutions
- Keep patching individual regressions as they appear.
- Fastest short-term path.
- Not recommended because it preserves the same structural blind spot.
- Add a small ORM-backed/request-backed smoke layer for critical Cloud routes, while keeping narrow monkeypatched unit tests for pure branching logic.
- Recommended
- Exercises real models, querysets, view wiring, and templates for the highest-risk server-rendered routes without adding browser-test overhead.
- Build broad browser-level Playwright coverage immediately.
- Valuable later for JS and htmx flows.
- Too heavy as the first response to this failure class.
Plan
- Audit the existing
tests/cloudcoverage and identify tests that stub out the framework layers most likely to fail in production. - Add DB-backed request smoke coverage for high-value server-rendered routes: org home, project home, and dashboard view.
- Fix the concrete queryset regression by ordering recent dashboards on the
real
last_syncedfield. - Remove or narrow redundant tautological tests that only assert mocked ORM strings instead of real behavior.
- Document the recommended Cloud testing split so future tests do not drift back toward over-mocking.
Implementation Progress
- Audited the current Cloud tests and confirmed there was no DB-backed
request/template smoke coverage in
tests/cloud. - Fixed
apps/cloud/apps/chat/dashboard_cards.pyto order recent dashboards by-last_synced, which matches the realDashboardmodel. - Added
tests/cloud/test_cloud_route_smoke.py: - boots a minimal Django schema locally with migrations
- resets the DB between tests with
flush - exercises real ORM + real template rendering for:
- org home/chat page
- project home
- dashboard view
- adds an explicit regression assertion that
get_recent_dashboards()orders bylast_synceddescending - Removed the tautological fake-queryset ordering test from
tests/cloud/test_org_chat_home.pyand kept the remaining narrow monkeypatched tests focused on branching/render-selection behavior. - Updated
apps/cloud/README.mdwith a Cloud test split: - monkeypatched tests for pure logic
- ORM/request smoke tests for critical server-rendered routes
- browser tests only where JS/htmx behavior actually matters
QA Exploration
- N/A for this task. Coverage added at the Django ORM/request/template layer.
- [x] QA exploration completed (or N/A for non-UI tasks)
Review Feedback
- Initial review flagged the old fake-queryset ordering test as redundant and asked for an explicit ORM-backed ordering assertion.
- Addressed by deleting the tautological ordering test and adding a real
last_syncedordering regression test intests/cloud/test_cloud_route_smoke.py. - [x] Review cleared