Model checks
LLMart, as the auditors found it. Provider →
last audit 18 Nov · evidence checked on-chain×0.80
on its $ANTS points now
Approved auditors check that each provider serves the model it sells.
- 🧪They buy from it like any buyer: ~100 obscure number questions, compared with the real model.
- 📏The real model misses some too: that is the noise. A provider fails only if it misses clearly more, and two auditors must agree.
- 💰Each pass +2% on $ANTS points, up to +50%. Fails cost more: 1 −4%, 2 −16%, 3 −36%. Checked again every week.
How this works (Antseed's announcement ↗)
Approved auditors check that each provider serves the model it sells.
- 🧪They buy from it like any buyer: ~100 obscure number questions, compared with the real model.
- 📏The real model misses some too: that is the noise. A provider fails only if it misses clearly more, and two auditors must agree.
- 💰Each pass +2% on $ANTS points, up to +50%. Fails cost more: 1 −4%, 2 −16%, 3 −36%. Checked again every week.
Passed
8
SAME · counting now
Last audit
18 Nov
epoch 31 · counts to 32
| Model | Differed↑ | × noise | Fail line | Noise | Proof |
|---|
| Grok 4.7SAME | 0% | 0.0× | 11 under | 0.0% | ↗ |
| Claude Opus 5.5SAME | 3% | 1.5× | 14 under | 1.7% | ↗ |
| Qwen 3.8 27BSAME | 9% | 0.9× | 28 under | 10.6% | ↗ |
| Kimi K3.1 ProSAME | 10% | 2.4× | 7 under | 4.2% | ↗ |
| GLM 5.3SAME | 11% | 0.9× | 27 under | 11.3% | ↗ |
| Claude Sonnet 5.5SAME | 11% | 2.3× | 11 under | 4.6% | ↗ |
| MiniMax M3SAME | 17% | 1.2× | 23 under | 14.1% | ↗ |
| Gemini 4 ArgonDIFF | 30% | 30× | +16 over | 1.0% | ↗ |
| Gemini 3.1 Pro PreviewDIFF | 35% | 13× | +19 over | 2.7% | ↗ |
| Claude Fable 5.1DIFF | 39% | 7.7× | +27 over | 5.0% | ↗ |
Evidence of some checks not found
Claude Fable 5.1DIFF
32 of 110 differed (fails from 21) · 54 of 140 differed (fails from 27) · the real model differs from itself on 5%
Where it missed
Medical (drug half-lives)26/48
Pop culture (box office and ratings)4/10
Internet culture (dates and numbers)2/5
Programming (release years)10/26
For example
M87 jet projected length in kiloparsecsreal 1.5 · provider 5
Henize 2-10 blue compact dwarf black hole mass in millions of solar massesreal 1 · provider 3
Qaidam Basin area (square kilometers)real 120,000 · provider 250,000
✓Reasoning switched off with reasoning effort none, as the real model was asked.
Gemini 3.1 Pro PreviewDIFF
38 of 110 differed (fails from 19) · 41 of 131 differed (fails from 23) · the real model differs from itself on 3%
Where it missed
Medical (drug half-lives)7/14
Astronomy (periods and distances)14/29
Chemistry (boiling points)5/11
Chinese history (specific dates)7/17
For example
Tarantula Nebula (30 Doradus) estimated total mass (millions of solar masses)real 1 · provider 1,000,000
sebelipase alfa elimination half life hoursreal 0.1 · provider 2
The number of chapters (juan) in the original Shishuo Xinyu (A New Account of the Tales of the World)real 3 · provider 36
✓Reasoning switched off with reasoning effort none, as the real model was asked.
Gemini 4 ArgonDIFF
30 of 100 differed (fails from 14) · 27 of 110 differed (fails from 19) · the real model differs from itself on 1%
Where it missed
Mathematics (special values)1/1
Chinese literature (publication years)9/17
Programming (release years)7/15
Chinese history (specific dates)5/11
For example
neutron gyromagnetic ratio (in 10^8 s^-1 T^-1)real 1.832 · provider 183.2
shielded helion gyromagnetic ratio (in 10^8 s^-1 T^-1)real 2.038 · provider 203.8
Number of chapters in Lu Buwei's Lushi Chunqiureal 160 · provider 26
🧠Kept reasoning when asked not to: about 315 reasoning tokens on each of 4 ways to switch it off.
4 older checks · 3 failed · not counting now
| Model | Differed↑ | × noise | Fail line | Noise | Proof |
|---|
| Kimi K3SAME 7 Nov · replaced | 16% | 1.8× | 12 under | 8.7% | ↗ |
| GPT-6.1 SolDIFF 7 Nov · expired | 23% | 46× | +11 over | 0.0% | ↗ |
| MiMo V2.6 ProDIFF 7 Nov · expired | 30% | 3.3× | +25 over | 8.8% | ↗ |
| GPT-6 AstraDIFF 7 Nov · expired | 60% | 11× | +41 over | 5.5% | ↗ |
MiMo V2.6 ProDIFFno longer counts
77 of 260 differed over 2 sets (fails from 52) · 51 of 130 differed (fails from 37) · the real model differs from itself on 9%
Where it missed
Chemistry (melting points)9/14
Programming (release years)9/16
Medical (drug half-lives)5/9
Biology (chromosome numbers)6/12
For example
NGC 6888 Crescent Nebula expansion velocityreal 30 · provider 1,500
coercive field of permalloy 80 at 293K amperes per meterreal 4 · provider 70
dont trust the b in apartment 23 episode count totalreal 26 · provider 79
✓Reasoning switched off with reasoning effort none, as the real model was asked.
GPT-6.1 SolDIFFno longer counts
14 of 100 differed (fails from 12) · 23 of 99 differed (fails from 12) · the real model differs from itself on 0%
Where it missed
Chinese internet (platform dates)9/21
Pop culture (box office and ratings)9/24
Chinese literature (publication years)2/6
Chemistry (melting points)1/3
For example
Lumasiran terminal plasma elimination half-life (hours)real 5.2 · provider 168
王灼《碧鸡漫志》的卷数real 5 · provider 10
Hamlet (1996) — camera-negative width in millimetersreal 65 · provider 22
✓Reasoning switched off with reasoning effort minimal, as the real model was asked.
GPT-6 AstraDIFFno longer counts
56 of 110 differed (fails from 17) · 66 of 110 differed (fails from 25) · the real model differs from itself on 5%
Where it missed
Physics (material properties)11/11
Mathematics (special values)6/6
Cryptography (parameters)2/2
Pop culture (box office and ratings)12/13
For example
SCHOTT N-F2 glass Sellmeier squared resonance wavelength C1 (10^-12 m^2)real 9.959e-15 · provider 0.01237
Fused silica Sellmeier resonance wavelength λ2, Malitson 1965 (10^-6 m)real 1.162e-7 · provider 0.1162
Sapphire ordinary-ray Sellmeier resonance wavelength 1, Malitson–Dodge dispersion fit (10^-6 m)real 7.266e-8 · provider 0.072
✓Reasoning switched off with disable thinking, as the real model was asked.
| Round | Passed | Failed | Multiplier |
|---|
| 17 Novepoch 31 | 7 | 3 | ×0 |
| 7 Novepoch 30 | 10 | 4 | ×0.56 |
| 4 Novepoch 29 | 8 | 2 | ×1.00 |