Code
Everything you see on this site — the Explore
charts and every Data download — is produced by the small
FastAPI + Jinja + Plotly app below, from the JSON/CSV
inventory artifacts shipped under app/data/. Stock-only stack,
no pandas.
Run it, get identical output. Every JSON API response
and every download is computed deterministically from the shipped
inventory files — no hidden state, no network call, no random seed. Install
the pinned deps, run uvicorn app.main:app, and the charts and
download bytes regenerate byte-for-byte.
1 · The JSON API over the real corpus inventory
The charts fetch these endpoints. Each returns a real on-disk inventory structure (per-cluster document counts, year spans, extracted-table counts) — genuine corpus measurements, no transformation beyond sort/slice.
@app.get("/api/panels")
def api_panels() -> JSONResponse:
"""Top panel-capable series clusters — real per-cluster measurements."""
return JSONResponse(INVENTORY["panels"])
@app.get("/api/richest")
def api_richest() -> JSONResponse:
"""Series clusters with the most extracted tables — real counts."""
return JSONResponse(INVENTORY["richest_by_tables"])
@app.get("/api/totals")
def api_totals() -> JSONResponse:
return JSONResponse({
"project": HEADLINES["project"],
"inventory": INVENTORY["totals"],
"provenance": INVENTORY["_provenance"],
})
2 · The charts (vendored Plotly + Arcanum Site Kit theme, offline)
The explorer draws the fetched clusters with the vendored Plotly build (no
CDN), routed through the kit's ArkPlotly so every chart shares one
transparent, deep-red-accent, dark/light-aware theme and a modebar with PNG +
CSV export. Labels are de-underscored/de-truncated for legibility. The bars are
document counts and extracted-table counts per real cluster — not a time series.
fetch('/api/panels').then(r => r.json()).then(rows => {
const top = rows.slice(0, 18).reverse();
// ArkPlotly = the kit's shared Plotly theme: transparent bg, --ark-* accent +
// colorway, dark/light-aware, trimmed modebar (PNG + "Download data (CSV)").
ArkPlotly.plot('panels-chart', [{
type: 'bar', orientation: 'h',
x: top.map(r => r.n_docs),
y: top.map(r => r.display_name || r.label), // English label (display_name)
// r.original_label carries the Russian source title (shown in the hover/table)
hovertext: top.map(r => `${r.display_name}<br>${r.n_docs} docs (${r.span})` +
(r.original_label ? `<br><i>${r.original_label}</i>` : '')),
hovertemplate: '%{hovertext}<extra></extra>'
}], {
title: 'Panel-capable clusters by document count (top 18)',
margin: {l: 300, r: 20, t: 40, b: 40},
height: 600,
xaxis: {title: 'documents in cluster'}
}, {filename: 'vlad_panel_capable_clusters'});
// Inline CSV download of the same table (real panels_csv artifact, text/csv).
ArkDownloads.buttons({ csv: '/api/download/panels_csv' });
});
3 · The downloads (correct content-types)
The Data page serves files verbatim or as faithful CSV serializations of an existing real JSON structure. Each format carries its true media type — the "CSV-served-as-JSON" failure is avoided by deriving the content-type from the artifact's declared format.
MEDIA = {
"csv": "text/csv; charset=utf-8",
"json": "application/json",
"zip": "application/zip",
}
def _clusters_csv(key: str) -> bytes:
"""Serialize a real cluster list from inventory.json (panels / richest)."""
inv = _load("inventory.json")
buf = io.StringIO()
w = csv.writer(buf)
w.writerow(["series_cluster", "example_document", "n_docs", "n_years",
"span", "extracted_tables", "body_chunks"])
for r in inv[key]:
w.writerow([r["series"], r["label"], r["n_docs"], r["n_years"],
r["span"], r["tables"], r["body"]])
return buf.getvalue().encode("utf-8")
def render_artifact(art: dict) -> tuple[bytes, str, str]:
raw = _bytes_for(art)
return raw, MEDIA[art["fmt"]], _download_filename(art)
4 · How the inventory is built (and translated)
The 266-cluster inventory is measured off the validated HDARP knowledge base
by the project clusterer, then summarized into the shipped
inventory.json by build_data.py. It only counts and
groups real extractions — it never invents a value. The same build step applies
the curated English-translation map (translations.json): every
displayed Russian source-document title becomes an English
display_name, with the Russian original kept as
original_label. Untranslated Russian and dropped garbled labels are
reported on every build.
with open(INV, newline="", encoding="utf-8-sig") as fh:
for r in csv.DictReader(fh):
rows.append({
"series": (r.get("series") or "").strip(),
"label": (r.get("sample") or "").strip() or (r.get("series") or ""),
"n_docs": _int(r.get("n_docs")), "n_years": _int(r.get("years")),
"span": (r.get("span") or "").strip(),
"tables": _int(r.get("tables")), "body": _int(r.get("body")),
"sample": (r.get("sample") or "").strip(),
})
# panel-capable = a real cluster spanning >= 2 distinct years
panel = [x for x in rows if x["span"] and x["n_years"] >= 2]
# English-first display: translate the Russian source title, keep the original.
def _localize(row, tmap):
s = row["sample"]
if _is_cyrillic(s) and s in tmap:
row["display_name"] = tmap[s] # English (primary)
row["original_label"] = _clean(s) # Russian (secondary, shown in parens)
else:
row["display_name"] = _clean(s) # already-English slug
row["original_label"] = None
5 · Series-construction code Roadmap — not yet written
The harmonized economic series do not exist yet, so neither do their
construction scripts. The plan is fixed: each panel-capable cluster will be
built with the uniform Anu pipeline (a single source of truth per
series, series_registry.json, with full provenance). These
scripts land in the project repository as clusters are built, starting
with the Cluster A fiscal pilot (1856–1990) — see
Methodology and the
construction roadmap.
| Planned stage (Anu pipeline) | Will produce | Status |
|---|---|---|
| anu-research | Variable + source map from each cluster's KB dirs | scoped |
| anu-ingestion | Parse per-chunk table CSVs into a raw long table (year × category × value) | scoped |
| anu-extension / extenbook | Stitch multi-part / multi-year editions into one series | scoped |
| anu-variant | Regime / currency / orthography variants (1897 + 1922/1947/1961 reforms) | scoped |
| anu-review / anu-adequacy | Coverage + quality gate; register in series_registry.json | scoped |
None of these scripts are shipped in the repro bundle because none are written yet — labeling them "available" would be fabrication. When the pilot lands, this section gains real, runnable construction code.