A date-partitioned generator (one page + one JSON per day) usually has a single entry point like update_charts(date, lang) that the nightly cron calls with yesterday. Backfilling reuses that same entry point with an older date. The per-day artifacts are fine — they are keyed by date. The danger is every derived artifact the same function also rewrites, because those represent "latest" and are built from whatever date you passed in.
The asymmetry that bites
In the generator I was repairing, two sibling calls behaved differently:
# build_page.py — save_chart(): GUARDED
most_recent = check_most_recent(lang=lang, project=project)
if query_date == most_recent:
save_rendered(lang_index, chart_template, data) # /<lang>/index.html
if lang == DEFAULT_LANG:
save_rendered(main_index, chart_template, data) # / homepage
# build_page.py — update_feeds(): NOT GUARDED
def update_feeds(cur_date, lang, project, day_count=10):
for day_delta in range(0, day_count):
date_i = cur_date - timedelta(days=day_delta)
...
save_rendered(feed_path, rss_template, render_ctx) # unconditionalSo backfilling a three-day-old date could not touch the homepage, but would rewrite every language's RSS feed with a 10-day window ending at the backfilled date — silently dropping the three most recent days from 28 public feeds. No error, no warning; the feed just rewinds and stays wrong until the next nightly run happens to fix it.
The check, before you run anything
Grep the update entry point for every save/write/render call and classify each target:
- Date-keyed (
.../2026/8/14.html) — safe, idempotent. - "Latest" aliases (homepage,
<lang>/index.html) — must be guarded bydate == most_recent(). Confirm the guard exists; do not assume symmetry with a neighbouring function. - Rolling windows (RSS/Atom, sitemaps, "recent N" lists) — almost never guarded, because for the nightly path
cur_dateis the newest date, so the bug is invisible in normal operation. This is where the damage lands.
The cheapest fix is to finish the backfill with a render-only pass at the current newest date, which recomputes the rolling artifacts from the right anchor without refetching anything:
d = bp.check_most_recent(lang=lang, project=project)
bp.update_charts(d, lang, project) # no network; rewrites feed + indexesTreat that pass as part of the backfill, not an optional follow-up.
Two related things worth knowing
Backfill ascending. If the update function also re-saves a date's neighbours for prev/next navigation, going oldest-first means each run repairs the previous one's forward link. Descending leaves the oldest day dead-ended:
if check_chart(cur_date, 1, ...): save_chart(cur_date - timedelta(days=1), ...)
if check_chart(cur_date, -1, ...): save_chart(cur_date + timedelta(days=1), ...)Run as the owning user. Backfilling as your own account under sudo-less access creates root-of-tree files owned by you; the nightly cron then cannot overwrite them, converting a one-time gap into a permanent one. Check the perms of what the code writes, not just the day directory — the day dir was group-writable in my case, but the project index one level up was drwxr-xr-x, so the run would have died partway with a PermissionError after writing some days. A [ "$(id -un)" = "owner" ] || exit 1 preflight is worth the two lines.
Scope the backfill to the actual gap
Don't reach for a wide window because it is the script's default. Count what is genuinely missing first:
for d in 13 14 15 16 17 18; do
printf '%s: %s\n' "$d" "$(ls static/*/project/2026/8/$d.json 2>/dev/null | wc -l)"
done
# 13: 28 14: 0 15: 0 16: 28 17: 28 18: 28That took one command and cut the job from four days to two — and revealed that one "missing" day had already self-healed, because a still-running poller with a long enough retry window picked it up when upstream recovered. Re-fetching known-good days is not free: it re-hits upstream APIs and rewrites files that were already correct.