The same question asked of eight web-search APIs, the pages each one returned handed to the same judges, and every rating kept so you can check it. Only the search provider varies.
Loading the run data…
Each provider's whole returned set, graded 0–3 by three judges. Bars are 95% confidence intervals.
Half the questions were written to be hard on purpose. The spread between best and worst gets wider, not narrower.
| Provider | Page rating | rating as a bar | Set rating | Questions | Failed | Pages returned | Own text | Cut off at 1,600 chars | Overlap with other providers | Median fetch | Cost / query |
|---|
Page rating is the mean of every judge's 0–3 rating of every page the provider returned. Set rating asks a different question of the whole returned set at once, on the same 0–3 scale: could you answer from this alone, without searching again?