Proof
What do the published numbers show?
Each bar sets Knos against its baseline and opens the file its number comes from.
How did Knos do in each outside test?
Knos lost 1 of 7 outside tests and won 1; the rest are ties or not run yet.
| Test | Question | Arms | Numbers | 95% interval | Result |
|---|---|---|---|---|---|
| Where demand is | Where is a kind of business most common per person? | Knos with Qloo, Claude alone | 0.193 against 0.407 (n 30) | [-0.360, -0.060] | loss |
| Films fans also love | Which films do a film's fans also love? | Knos with Qloo, Claude alone | 0.320 against 0.307 (n 30) | [-0.060, +0.087] | tie |
| Crowdfunded launches, 250 held out | Did a past project reach its goal? | open signals with Qloo, always success | 78.0% tuned (59.6% as one verdict) against 78.8% (n 250) | [-0.063, +0.039]* | tie |
| Crowdfunded launches, 300 ranked | Does the score rank projects by money raised? | open signals with Qloo, open signals alone | Spearman 0.108 against 0.113 (n 300) | [-0.037, +0.025] | tie |
| 200 research awards | Did a past award succeed? | open signals, a coin | 47.5% against 50% (n 200) | [-0.093, +0.044]* | tie |
| Each Qloo signal removed | Does one Qloo answer change the launch city, the target or the verdict? | each recorded card with every Qloo answer, then with one call family removed | 18 of 59 removals change one; all removed, 6 of 6 (6 cards) | [+0.203, +0.431]† | win |
| Launch cities, pre-registered | Do Qloo's lifts find the metros where a kind of business is densest? | top 15 metros by people reranked by lift, people alone, Claude alone, shuffled lift | not run: Qloo has refused every call since 25 Oct (HTTP 429) | none | not run |
An interval above 0 is a win and one below 0 a loss. One that holds 0 is a tie. * Wilson interval of the accuracy, less the base rate, from the published counts. † Wilson interval of the share of removals that change one. A test that has not run has no result.
How many devnet transactions has each program made?
Each row below is read now from devnet, at most 1,000 per program.
| Program | Transactions | Newest |
|---|---|---|
| Reading devnet… | ||
Read now from devnet, at most 1,000 per program.