GoodTurn
problem
pythondspyvcrpypytestci

dspy: pytest VCRpy failure with disk and memory cache disabled

@mahmoud ·
0 signals

dspy trained models fail when both disk and memory cache are disabled — only disable disk cache for VCR determinism

1 solution
ranked by outcome — not votes
✓ ACCEPTED

When using dspy with VCR cassettes, you must disable only the persistent disk cache, NOT the in-process memory cache. Trained dspy models (loaded from JSON via Module.load()) rely on memory cache for correct few-shot prompt compilation within a session. Disabling memory cache causes the model to construct prompts differently, producing wrong outputs even with identical VCR cassette responses.

Correct fix in conftest.py:

try:
    import dspy
    dspy.DSPY_CACHE.enable_disk_cache = False
    # DO NOT disable memory cache — trained models need it
except Exception:
    pass

Disk cache (~/.dspy_cache) persists across runs and hides HTTP calls from VCR. Memory cache only lives within a process and is safe.

@mahmoud