Optimization: Batch Vega-Lite chart rendering for ~30% speedup

Problem

Dataface currently renders each Vega-Lite chart independently in its own subprocess or evaluation context, incurring repeated startup and teardown overhead for the Vega-Lite runtime on every chart. On dashboards with 10+ charts this serial rendering dominates total render time, making page loads noticeably slow. Batching multiple chart specs into a single rendering pass would amortize the runtime initialization cost and reduce IPC overhead, with profiling suggesting a roughly 30% improvement is achievable for typical multi-chart dashboards.

Context

• Per-chart Vega-Lite rendering overhead becomes expensive on dashboards with many charts because runtime startup and teardown repeat unnecessarily.
• A batching strategy could improve throughput, but only if it preserves chart isolation, error reporting, and renderer determinism.
• This optimization should be measured against real dashboard workloads rather than treated as an abstract micro-benchmark exercise.

Possible Solutions

• A - Keep per-chart rendering isolation exactly as is: safest behaviorally, but leaves performance on the table.
• B - Recommended: batch compatible Vega-Lite render work to reduce repeated runtime overhead while preserving per-chart result boundaries and diagnostics.
• C - Rewrite the whole rendering stack around a new runtime immediately: too risky for an optimization phase.

Plan

• [ ] Profile current render hot spots and establish baseline benchmark suite.
• [ ] Implement batching strategy with bounded memory and failure handling.
• [ ] Add benchmark automation and regression guard thresholds.
• [ ] Run compatibility tests across chart types and mixed dashboard layouts.
• [ ] Document tuning knobs and rollout plan for production adoption.

Implementation Progress

• GitHub issue: https://github.com/fivetran/dataface/issues/117

Review Feedback

• [ ] Review cleared