Functional and technical specification derived from research. This is a recommended direction for implementation planning, not committed scope.
DESIGN.md).type: inline as a normal built-in (Tier 0), ship early: Treat it like bar / line — always available, documented, tested — not a special “extension” feature. That alone covers most “custom chart” needs (VL/Vega in YAML, AI-authored specs, odd one-offs) and may remove or shrink the need for a heavyweight plugin registry, named template packs, or Python ChartExtension for many users.html arm of inline stays a sharp escape hatch, not a productized widget system (§6).inline fields and any later extension/registry machinery with M2 YAML versioning / migrations so contract changes are visible and migratable.type: inlinetype: inline with exactly one of vega_lite, vega, svg, html (§6).inline early (could land before a full declarative schema or plugin system). It uses the same vl-convert paths as other charts; work is mainly schema + compiler + render dispatch + tests, not new viz technology.vega_to_svg / vega_to_*; render_vega_spec today only calls vegalite_to_*. Wiring inline + vega is dispatch, not a new dependency.inline + vega_lite first; fall back to inline + vega, then inline + svg, then Python (Tier 2) only if needed.dataface.yml registration, AI catalog of allowed shapes) — the direction in extensible-schema-with-custom-elements-and-chart-types.md.template_id, org packs). Strict YAML policy (only these chart types / only these built-ins in the editor) is not assumed here — see §4.1 (M5 placeholder, may never).inline + query binding + themes are enough, skip or postpone large registry work. Do not block shipping inline on this.build_vega_lite, build_vega, or render_svg) — nicheinline cannot express the chart (heavy codegen, non-Vega SVG, integration with proprietary layout).DESIGN.md).Recommendation: Ship type: inline (Tier 0) first; add Vega dispatch alongside VL for inline; treat Tier 1 registry and Tier 2 Python as follow-ons driven by real demand, not prerequisites.
inlineDefault story: no plugin install — authors use type: inline. That may be sufficient for a long time.
If we still need named templates and/or Python chart extensions, split the problem into two layers that are easy to confuse:
uv / pip / pyproject.toml) or explicit project-local modules on disk (§4.1b). YAML does not run pip for you. Hynek-style: invest in dft compile / dft doctor verification — actionable errors (“add this dep”, “sync”, “fix this import”) — not a parallel package manager.allow_types, expose_builtins, template allowlists for “only these extensions may load.” Treat that as a placeholder on the M5 milestone backlog; we may never implement it if demand never appears. Armin-style: avoid building it pre-emptively.type: inline stays a built-in; it never lives in the extension registry. Later (M3+): reusable named template / registry work (templates + optional Python packs) if inline is not enough — separate from M5 policy placeholders.
How code is installed (recommended):
pyproject.toml), e.g.
dataface-charts-acme = { git = "https://github.com/acme/dataface-charts-acme.git", branch = "main" }
under [tool.uv.sources] / dependency-groups / [project.optional-dependencies] — whatever matches repo conventions.EXTENSIONS.md):[project.entry-points."dataface.charts"]
acme_stream = "acme_charts.stream:register_chart"
uv sync / pip install is what actually installs the code. There is no separate Dataface installer unless we later add a thin preflight that only checks versions.Verification (Hynek-style — real, not “policy lockdown”):
Optional required_packages documents what the repo expects installed; dft compile / dft doctor fails with actionable messages if a listed distribution is missing. That is env truth, not “you may only use these chart types.”
charts:
extensions:
required_packages:
- { name: "dataface-charts-acme", specifier: ">=1.0,<2" }
required_packages: when present, resolve with importlib.metadata (and lockfile where available); if missing, fail with exact uv add … / optional-deps hints — no silent skip.M5 placeholder — strict YAML policy (may never ship):
If one day we need enterprise-style lockdown (only these extension types, only these built-ins in Cloud editor / AI), we could add something like allow_types / expose_builtins / template allowlists. Park that on the M5 milestone as a maybe; assume we will not build it until a concrete customer forces the conversation. Until then: everything installed + everything in local_modules that resolves is usable.
# M5 backlog sketch only — do not implement by default
charts:
extensions:
allow_types: [acme_stream, acme_funnel]
expose_builtins: [line, bar, area, table, inline]
If this ever ships, apply Bruno-style diagnostics: collect all violations in one compile pass where practical.
Why not chart_deps: [github urls] alone?
pyproject.toml and run uv sync” — useful for docs and CI checks, but installation stays in pyproject/lockfile.Ensuring deps are installed:
| Approach | Role |
|---|---|
uv.lock / pip freeze |
Source of truth for CI and prod images |
dft compile / dft doctor |
Verify required_packages and entry points resolve; fail fast with actionable message |
| Cloud deploy | Same image that runs the app includes the extras; no runtime pip install from dashboard YAML |
Teams often want one-off or repo-private chart logic without publishing my-corp-charts to PyPI. That is still valid; it does not require a second installable distribution if the dashboard repo is already a Python package or you accept a path-backed import.
Recommended (still “real Python”): editable monorepo package
pyproject.toml with [build-system] / [project] and is installed editable: uv sync / pip install -e ..src/<pkg>/dataface_charts.py (or similar) using the same dataface.charts entry points in that pyproject — no second package, no PyPI; it is just your repo’s package.Alternative: explicit local module(s) (no entry point — dumb and explicit)
dataface.yml lists importable modules that call a documented register_charts(registry) (or return a list of descriptors) at compile time:charts:
extensions:
local_modules:
- my_dashboards.charts_custom # must be importable (project on PYTHONPATH or editable install)
sys.path only for that session when resolving local_modules, or requires uv run from the repo so imports work — document one blessed workflow.dbt Python models; not for untrusted YAML sources.Not recommended: auto-import every *.py under ./plugins/ without an explicit list — too magical (fails Armin’s “dumb loader” test). A single local_modules list or one register.py path is enough.
Relation to inline: locals are for reused Python logic across many charts; inline stays the default for one-off VL/Vega in YAML.
Reusable Vega-Lite / Vega JSON templates (files in repo, optional placeholders) do not need pip:
charts:
templates:
paths: ["./chart_templates/"] # or explicit ids → file mapping
Compiler discovers templates from disk using a dumb rule: glob / explicit path list / id → file map — no dependency graph between templates. Version control is git. Template allowlists (only these template ids) belong in the same M5 “maybe never” bucket as allow_types. Pairs with type: inline when authors paste spec vs template_id: org_streamgraph.
| Ecosystem | Pattern |
|---|---|
| Python plugins | Entry points (pytest, dbt adapters, Jupyter) — install via pip/uv; app discovers at import time |
| Grafana / Superset | Frontend bundles built into the host app or dropped in a plugins dir — not “YAML installs npm” |
| dbt | Packages in packages.yml + dbt deps — explicit second step (closest analog if we ever add dft deps for template packs only) |
Recommendation: For Python chart code, mirror entry points + pyproject/lockfile, plus optional required_packages verification. Do not plan YAML type allowlists for M2–M4 — M5 placeholder only, may never. For templates, git + paths (+ optional dft deps-style fetch later only for static assets, not arbitrary code execution). Do not promise “list GitHub URLs in default_config and charts auto-install” unless we build a deliberate, audited package story.
The hybrid sketch in extensible-schema-with-custom-elements-and-chart-types.md maps cleanly:
template_id on charts).inline → unchanged; not in registry.Dataface may ship bundled with dbt (vendor CLI, Cloud/Fusion-style images, locked installer). That does change the default story for Python extensions, not the conceptual model.
| Topic | Standalone Dataface / open venv | Locked dbt environment |
|---|---|---|
type: inline |
Same | More important — no extra packages; VL/Vega/svg/html in YAML is always available once Dataface runs. |
| Templates on disk (§4.2) | Git repo paths | Same; often delivered via dbt deps as static files inside a dbt package (e.g. chart_templates/*.vl.json) — no Python dependency edge, only dbt deps. |
| Entry-point chart packages | pip install / uv add anything |
Often not allowed ad hoc. Extensions must be pre-installed in the bundle or exposed as blessed optional extras of the same distribution (pattern: dbt-<dist>[dataface-charts-acme]) — vendor-controlled, like adapters. |
local_modules / editable monorepo (§4.1b) |
Fine for internal dev | OSS / self-managed projects only; Cloud images typically ignore or disallow arbitrary repo Python unless documented. |
required_packages verification |
“Run uv add” |
Errors may need to read: “not included in this dbt release” / contact admin — same check, different remediation copy. |
What we learn: Inline + templates are the portable extensibility layer across dbt-co-released and standalone installs. Python entry-point plugins remain valid but are second-class wherever the runtime is frozen; there, treat them like dbt adapters — versioned, tested, shipped with the product, not user-pip-installed at compile time.
dbt packages vs Python packages: Keep the distinction explicit: packages.yml is ideal for declarative template assets; Python chart code stays in the interpreter’s dependency graph (pyproject / bundle extras).
# Conceptual — not an implementation commitment
class ChartExtension:
id: str
def validate(self, chart: DotDict) -> None: ...
def build_vega_lite(self, chart: DotDict, data: list[dict]) -> dict: ...
# OR — when VL is not enough:
def build_vega(self, chart: DotDict, data: list[dict]) -> dict: ...
# OR — bypass Vega entirely:
def render_svg(self, chart: DotDict, data: list[dict]) -> str: ...
Implementations pick one build path per extension (VL vs Vega vs raw SVG) depending on what the chart needs. Distinct from type: inline, which is built-in dispatch, not an installed plugin.
Registration via entry points (see EXTENSIONS.md) or explicit path in project config for monorepos.
type: inline — one built-in chart, optional payloadsDefault built-in — same class as bar / line, not an extension. Prefer one chart type: value instead of four separate “raw_” types. Authors set exactly one payload field; the compiler rejects zero or more than one* (clear errors, no silent precedence).
charts:
inline_chart:
type: inline
vega_lite: # mutually exclusive with vega / svg / html
$schema: https://vega.github.io/schema/vega-lite/v5.json
mark: bar
encoding: { x: { field: a, type: nominal }, y: { field: b, type: quantitative } }
data: { values: [] } # or rely on explicit query binding per schema design
Same chart with another payload slot:
vega_lite — mapping; pipeline: inject data/theme/size → vegalite_to_svg (today) / png / pdf.vega — mapping; pipeline: inject data → vega_to_svg (vl-convert already exposes it; see Tier 2 note in §3).svg — string or multiline text (fragment or full document per rules we document); embed in SVG-first output without vl-convert.html — string; HTML output path or foreignObject when embedding in SVG; same portability and XSS caveats as before.inline typetype: inline + exactly one payload key”).raw_vega_lite, raw_html, …).vega_lite, vega): lower XSS risk than html; still trusted-author / reviewed AI; cap size if needed; query binding must stay explicit (DESIGN.md).svg / html: markup risks as in the prior spec; optional project flag to enable; default no sanitization unless we add an explicit mode later.vega_lite → vega → svg before html when the goal is still a chart in the normal pipeline.Payload values could alternatively be strings interpreted as paths (e.g. ./specs/foo.vl.json) — same exclusivity rule, one key set.
Authors can use Jinja (or the same template engine Dataface already uses elsewhere) to parameterize inline payloads, as long as we pin when templates run and what context is visible.
| Payload | Templating use | Caveats |
|---|---|---|
vega_lite / vega |
Usually avoid putting raw query rows through Jinja to build JSON — easy to produce invalid JSON or sneak in logic that belongs in SQL. Prefer normal query → structured data → inject data in code (today’s pipeline). Jinja is still reasonable for scalars: titles, colors, limits, width/height, variable-driven field names — or for a string payload that is rendered then parsed as JSON using a tojson-style filter so the result is valid. |
|
svg / html |
Natural fit: treat payload as a template string; render with a documented context (variables, maybe bounded row snippets if we ever allow it). html: use autoescape defaults appropriate for the context. |
Rule of thumb: Templating should not become a second data transformation layer that violates “data belongs to queries” (DESIGN.md). Use Jinja for authoring parameters and presentation strings; use queries for dataset shape.
An html payload might contain <form action="…"> to a customer API — markup only; Dataface does not handle POST or auth. json-render / Lowdefy remain mental models, not something we replicate as a framework.
dataface/core/render/chart/ as core built-ins + shared VL builder utilities.inline to the built-in chart dispatcher (types enum, compiler, renderer) first.inline is not enough — do not require it for the first extensibility release.inline (VL, Vega, minimal svg/html cases); extension/registry tests when that layer exists.| Milestone | Work |
|---|---|
| M2 (or earlier if scoped small) | Ship type: inline as a documented default: validation, query/data binding rules, vega_lite + vega vl-convert dispatch, tests. Fold inline into schema version / migrations when the YAML version field lands so payload keys evolve cleanly. |
| M2 | Schema version field, migrations, namespace rules — applies to inline and any future custom: types. |
| M3+ | Optional: named template registry, Python extensions — per extensible-schema-with-custom-elements-and-chart-types, reduced scope if inline covers most cases. |
| M5 | Placeholder (may never): YAML policy lockdown — allow_types, expose_builtins, template allowlists, etc. Do not schedule unless a concrete need appears; default assumption is permissive forever. |
Research / design stays useful for external comparisons; implementation can lead with inline without waiting for the full catalog/registry design.
type: inline as Tier 0 default early, and whether Tier 1 registry / Tier 2 Python stay in M3 or slip until demand is proven.inline + vega_lite first, then inline + vega if needed.