Ruairidh Wynne-McCorry
I'm Ruairidh (/roo-ree/), a senior engineer who's spent the last few years building product at scale. I write about (amongst other things): LLM evaluation, systems design, and whatever rabbit hole I've gone down that week. Code over commentary.
Writing
Why Sonnet 4.6 kept dropping a simple format rule
Sonnet 4.6 dropped a STATUS prefix in 80% of the longest runs I tested. Adding a reminder to the opening message, alongside reminders every ten turns, fixed the tested prefix rule but failed on a suffix marker. The five-model comparison covered 230 trajectories and cost about $40.
13 min read
Compressing Prompts with an Autoresearch Loop
I compressed five production system prompts by 63%. Then I gave them a broader set of tests, and every one failed.
10 min read