Dataface Tasks
    Heartbeat snapshot (JSON)

    Regression prevention and quality gates

    IDM4_V1_0_LAUNCH-DFT_CORE-02
    Statusnot_started
    Priorityp1
    Milestonem4-v1-0-launch
    Ownerhead-of-engineering

    Problem

    The execution adapters lack automated regression gates in CI. Changes to query generation, result parsing, or adapter connection logic can introduce regressions that are only caught manually or by end users after release. Without enforced quality gates — adapter-specific integration tests, output snapshot comparisons, cross-database compatibility checks — each release carries a risk of silently breaking query execution for one or more supported databases.

    Context

    • Manual review is not enough to protect the YAML contract, compiler/normalizer, execution adapters, and release/versioning once the change rate increases; regressions will keep shipping unless the highest-value checks become automatic.
    • This task should identify what needs gating in CI or structured review and what evidence is sufficient to block a risky change before it reaches users.
    • Expected touchpoints include dataface/core/, schema/compiled types, docs, and core test suites, automated tests, eval/QA checks, and any release or review scripts that can enforce the new gates.

    Possible Solutions

    • A - Add only a few narrow tests around current bugs: easy to land, but it rarely protects the broader behavior contract.
    • B - Recommended: define a regression-gate bundle around the core behavior contract: combine focused tests, snapshots/evals, and required review evidence for risky changes.
    • C - Depend on manual smoke testing before each release: better than nothing, but too inconsistent to serve as a durable gate.

    Plan

    1. Identify the highest-risk behavior contracts for the YAML contract, compiler/normalizer, execution adapters, and release/versioning and the types of changes that should be blocked when they regress.
    2. Choose the smallest practical set of automated checks and required review evidence that covers those contracts well enough to matter.
    3. Wire the new gates into the relevant test, review, or release surfaces and document when exceptions are allowed.
    4. Trial the gates on a few representative changes and tighten the signal-to-noise ratio before expanding the coverage further.

    Implementation Progress

    Review Feedback

    • [ ] Review cleared