Dataface Tasks
    Heartbeat snapshot (JSON)

    refactor: Consolidate config system - single source of truth in YAML

    IDISSUE-267
    Statusnot_started
    Priorityp1
    Milestonem2-internal-adoption-design-partners
    Ownerhead-of-engineering

    Problem

    Configuration values are currently loaded from multiple sources — environment variables, CLI flags, Python defaults, and YAML files — with no clearly defined precedence or single canonical path. This leads to situations where the same setting can be specified in two places with conflicting values, and the behavior depends on which code path runs first. Debugging config issues is time-consuming because there is no single place to inspect the resolved configuration. Consolidating into one YAML-driven source of truth with explicit override rules would eliminate these conflicts and make the system predictable for both developers and users.

    Context

    • Configuration is currently split across environment variables, CLI flags, Python defaults, and YAML files, which makes precedence hard to reason about.
    • The lack of one canonical source makes debugging and deployment behavior harder, especially when different environments resolve config differently.
    • This refactor should simplify the contract and precedence rules without breaking valid operational override paths.

    Possible Solutions

    • A - Leave the multi-source config system in place and document it better: lower disruption, but preserves complexity.
    • B - Recommended: move toward a YAML-centered single source of truth with explicit override rules for the few contexts that need them.
    • C - Eliminate all non-YAML overrides entirely: conceptually clean, but impractical for many runtime and deployment cases.

    Plan

    • [ ] Map current config pathways and precedence edge cases across products.
    • [ ] Implement unified schema/model with strict validation and helpful errors.
    • [ ] Migrate consumers to the new loader and remove duplicated parsing logic.
    • [ ] Add compatibility tests covering env, CLI, and YAML override behavior.
    • [ ] Publish migration notes and update examples to the unified config pattern.

    Implementation Progress

    Review Feedback

    • [ ] Review cleared