benchmarks
2 posts — newest first.
-
Your Chunking Benchmark Is Measuring Chunk Size
14 chunking strategies on SQuAD with answer-span ground truth. Semantic chunking lost to every alternative, and fixed-k scoring hid the thing that mattered.
-
Frontier models still fail half your incidents: reading ITBench-AA like an SRE
ITBench-AA put frontier models against 59 real Kubernetes incident diagnoses — all scored below 50%. What the benchmark measures and how to use it.