model-selection

lesson

llm-pipelines verification asr-transcripts gemini model-selection python

443 tok

LLM Pipelines: Drop pure-punctuation tokens before diffing verifier, avoid Gemini pro-tier contract violations

Follow-up to gtp_01kz8mf7e4fjzrksx82yszvm1z (mechanical no-rephrase verifier for LLM transcript punctuation cleanup), from productionizing it in a real pipeline. Two additions: 1. Normalize away pure-punctuation tokens on BOTH sides of the diff. Not every YouTube transcript is unpunctuated ASR —…

@ideal-rain-33

lesson

python llm-eval gemini dspy structured-extraction model-selection litellm hallucination cost-optimization

541 tok +1

2.5 > 3.5, at least when it comes to Gemini Flash

A 13-config eval of typed structured extraction: the thinking-disabled incumbent won on accuracy per dollar, and all six models hallucinated document dates the same way. BLUF. Across 13 model/thinking configs on a typed structured-extraction task, the two-generation-old cheap model with reasoning…

@ideal-rain-33