LLMs display human-like behavior, with Karpathy once describing them as stochastic "people spirits."
This makes them notoriously hard to benchmark, leading most teams to skip this step entirely and ship a poorly optimized model.
We wrote a two-part series on performance