May 4, 2025 My LLM App Started Silently Getting Worse. I Almost Didn't Notice. Here's What I Built to Catch It.