Skip to content

Before backfilling a historical date, audit which derived artifacts have a most-recent guard — feeds and indexes will silently rewind

TL;DR.

Backfilling an old date in a date-partitioned static site can overwrite "latest" artifacts (RSS feeds, index pages) with stale content, because those are regenerated from the date being processed. Grep for the most-recent guard before running the backfill, and finish with a render-only pass at the newest date.

A date-partitioned generator (one page + one JSON per day) usually has a single entry point like update_charts(date, lang) that the nightly cron calls with yesterday. Backfilling reuses that same entry point with an older date. The per-day artifacts are fine — they are keyed by date. The danger is every derived artifact the same function also rewrites, because those represent "latest" and are built from whatever date you passed in.

The asymmetry that bites

In the generator I was repairing, two sibling calls behaved differently:

# build_page.py — save_chart(): GUARDED
most_recent = check_most_recent(lang=lang, project=project)
if query_date == most_recent:
    save_rendered(lang_index, chart_template, data)   # /<lang>/index.html
    if lang == DEFAULT_LANG:
        save_rendered(main_index, chart_template, data)  # / homepage

# build_page.py — update_feeds(): NOT GUARDED
def update_feeds(cur_date, lang, project, day_count=10):
    for day_delta in range(0, day_count):
        date_i = cur_date - timedelta(days=day_delta)
        ...
    save_rendered(feed_path, rss_template, render_ctx)   # unconditional

So backfilling a three-day-old date could not touch the homepage, but would rewrite every language's RSS feed with a 10-day window ending at the backfilled date — silently dropping the three most recent days from 28 public feeds. No error, no warning; the feed just rewinds and stays wrong until the next nightly run happens to fix it.

The check, before you run anything

Grep the update entry point for every save/write/render call and classify each target:

  • Date-keyed (.../2026/8/14.html) — safe, idempotent.
  • "Latest" aliases (homepage, <lang>/index.html) — must be guarded by date == most_recent(). Confirm the guard exists; do not assume symmetry with a neighbouring function.
  • Rolling windows (RSS/Atom, sitemaps, "recent N" lists) — almost never guarded, because for the nightly path cur_date is the newest date, so the bug is invisible in normal operation. This is where the damage lands.

The cheapest fix is to finish the backfill with a render-only pass at the current newest date, which recomputes the rolling artifacts from the right anchor without refetching anything:

d = bp.check_most_recent(lang=lang, project=project)
bp.update_charts(d, lang, project)     # no network; rewrites feed + indexes

Treat that pass as part of the backfill, not an optional follow-up.

Backfill ascending. If the update function also re-saves a date's neighbours for prev/next navigation, going oldest-first means each run repairs the previous one's forward link. Descending leaves the oldest day dead-ended:

if check_chart(cur_date, 1, ...):   save_chart(cur_date - timedelta(days=1), ...)
if check_chart(cur_date, -1, ...):  save_chart(cur_date + timedelta(days=1), ...)

Run as the owning user. Backfilling as your own account under sudo-less access creates root-of-tree files owned by you; the nightly cron then cannot overwrite them, converting a one-time gap into a permanent one. Check the perms of what the code writes, not just the day directory — the day dir was group-writable in my case, but the project index one level up was drwxr-xr-x, so the run would have died partway with a PermissionError after writing some days. A [ "$(id -un)" = "owner" ] || exit 1 preflight is worth the two lines.

Scope the backfill to the actual gap

Don't reach for a wide window because it is the script's default. Count what is genuinely missing first:

for d in 13 14 15 16 17 18; do
  printf '%s: %s\n' "$d" "$(ls static/*/project/2026/8/$d.json 2>/dev/null | wc -l)"
done
# 13: 28   14: 0   15: 0   16: 28   17: 28   18: 28

That took one command and cut the job from four days to two — and revealed that one "missing" day had already self-healed, because a still-running poller with a long enough retry window picked it up when upstream recovered. Re-fetching known-good days is not free: it re-hits upstream APIs and rewrites files that were already correct.

No signals yet