Skip to content

Policy

How Jevals decides what to list, how it publishes, and how to hold it to account.

Who runs Jevals

Jevals is an independent project run by one maintainer. It is not affiliated with TypeSafe AI, Vercel, OpenRouter or any model lab, and no vendor reviews results before they are published.

Funding and conflicts

  • The maintainer pays for every API call at list price. Each release's measured spend is in the changelog.
  • No sponsorship, vendor credits, discounts or pre-release model access. If that changes, it is disclosed here before any affected release.
  • Jev is called only through Vercel AI Gateway, the way any developer would call it.

What gets listed

  • Every row is run by the maintainer with the published harness. There are no vendor submissions and no private variants: one configuration per model version.
  • Each lab's newest models are preferred. A system is listed on a tab only when it has a complete run of every task in that tab at 5 repeats.
  • Every run that is started is disclosed, including runs that were stopped; the changelog names them and the reason.
  • The model string each provider served is recorded per row. If an alias starts pointing at a different model, the row is rerun in a new release.
  • Baselines that any system can lose to are always listed.

Releases and corrections

  • Every board is an immutable release with a permanent URL under /r/<release>/. Mistakes are fixed in a new release whose changelog entry says what changed and why; old releases stay online.
  • Rows that cannot be told apart share a rank. "First place" is only claimed when the 95% interval of the difference excludes zero.
  • Vendor claims are quoted with their source and never mixed into measured rows.

License and reuse

Board data, run logs and suite files are licensed CC-BY-4.0: cite "Jevals (jevals.com), release <release>". They are linked from every board and collected in the public jevals-data repository. Item text is not republished; every item links to its public dataset, which keeps its own licence.

Privacy

This site uses Google Analytics to count visits and see which pages are read. Google Analytics sets its own cookies and receives your IP address and browser details, under Google's privacy policy. Cloudflare Web Analytics also counts visits and page load times; it sets no cookies and does not identify individual visitors, under Cloudflare's privacy policy. Blocking either changes nothing on the site. Every other script is the site's own, served from jevals.com, and every page works with scripts turned off, except the explorer.

If you enable notifications on the home page, we store your email address and the time you enabled them, nothing else, and use them only to email you when a new benchmark or model lands. The address is never shared or sold. Entering it again and choosing disable deletes it.

If you use the contact form, we keep your email address, your message and the time you sent it, and use them only to reply. They are never shared or sold. Ask in a message and we delete them.

Corrections

If a number is wrong, a label looks wrong, or a model deserves a row, open an issue in jevals-data with the release and row. Every correction lands in the changelog.