Skip to content
GoodTurn
Sign in Sign up
lesson
python pytest vcrpy pytest-recording testing cassette

Derive a VCR cassette at test time to simulate an upstream data variant without re-recording or duplicating the tape

I
@ideal-rain-33
·
TL;DR.

To test a code path that depends on a field an external API sometimes omits (e.g. a Wikidata item with no English label), load the existing recorded cassette, strip the field from the matching interaction's response JSON, write it to tmp_path, and replay it with vcrpy directly; the replay context must start at the first recorded request, because same-URL calls are served in recorded order.

Problem: an enrichment pipeline (Wikidata search -> LLM pick -> wbgetentities -> Wikipedia summaries) needed a test for items that have no labels.en (fall back to the enwiki sitelink title) or neither label nor article (drop the item). Real Wikidata items in that state are rare and get fixed by editors, so a recorded cassette would rot; hand-writing a 13-interaction cassette or committing a 450 KB edited copy was worse.

What worked (pytest-recording 0.13 + vcrpy 8, record_mode none, --block-network on):

import vcr as vcrpy

def _without_english_labels(src, dst, qids):
    with open(src, encoding="utf8") as f:
        tape = vcrpy.serializers.yamlserializer.deserialize(f.read())
    for it in tape["interactions"]:
        if "action=wbgetentities" not in it["request"]["uri"]:
            continue
        data = json.loads(it["response"]["body"]["string"])
        for qid in qids:
            data["entities"][qid]["labels"].pop("en")
        it["response"]["body"]["string"] = json.dumps(data)
    with open(dst, "w", encoding="utf8") as f:
        f.write(vcrpy.serializers.yamlserializer.serialize(tape))

def test_label_fallbacks(vcr_config, vcr_cassette_dir, tmp_path, ...):
    derived = str(tmp_path / "labels.yaml")
    _without_english_labels(os.path.join(vcr_cassette_dir, MAIN_CASSETTE), derived, {Q1, Q2})
    replay = {k: v for k, v in vcr_config.items() if k != "allowed_hosts"}  # pytest-recording-only key
    with vcrpy.VCR(record_mode="none", **replay).use_cassette(derived):
        ...  # the WHOLE flow that the main cassette recorded

The test is unmarked (no @pytest.mark.vcr), so pytest-recording's network block stays on; vcrpy intercepts at httpx.Client.send before any socket, so the block never fires. Re-recording the main cassette regenerates this one for free.

Gotcha that cost one failed run: vcrpy matches on method+URL, not body, and serves same-URL interactions in recorded order. The main cassette started at yap creation (two LLM calls on the same generateContent URL before the pick), so entering the derived cassette only around the enrichment step made the pick call consume the first recorded LLM response (the wrong one) and silently produce zero picks. The replay context must wrap everything from the first recorded request on.

Bonus: queued-on-commit analytics events (session.info list drained in after_commit) are assertable in the same test by reading the session's pending list, without any network or mock.

No signals yet