Methodology

🏷️ Status framework

Every topic on this site carries one of four statuses:

• CAN ✅ — the AI jury has settled at consensus that the capability exists today.
• CAN NOT ❌ — settled the other way: the jury agrees current systems can't do this reliably.
• DISPUTED ⚖️ — verdicts are mixed. That is its own honest answer — it means smart systems genuinely disagree about whether this is achievable yet.
• IN RESEARCH 🔬 — the topic is new. No jury verdicts yet, no enrichment, no images. These pages are excluded from search engines until they mature.

⚖ The AI jury

Each topic gets reviewed by a rotating panel of AI models — between 3 and 7 per check, drawn from different model families and different vendors. We call this panel the jury.

Internally, admin can audit which model returned which verdict. Publicly, we never name the AIs that sat on a given case. We list jurors as "Juror I, II, III" with anonymized titles. The point of the jury is to capture the consensus of independent reasoning systems, not to advertise vendors or invite gaming.

🗳️ Co robi każdy juror

Każdy juror dostaje ten sam prompt:

1. Przeczytaj stwierdzenie (np. „Czy SI potrafi skomponować fugę w stylu Bacha?”)
2. Zwróć jednowyrazowy werdykt: POTRAFI, NIE POTRAFI lub NIEROZSTRZYGNIĘTE.
3. Podaj jednozdaniowy powód werdyktu.
4. Jeśli werdykt to POTRAFI, oszacuj miesiąc i rok, w którym zdolność pierwszy raz pojawiła się niezawodnie.

Każdy juror odpowiada niezależnie. Żaden nie widzi werdyktów pozostałych. Unika to efektu stada, jaki powstałby, gdyby jeden model zakotwiczył resztę.

📊 Jak łączą się werdykty

A statement's status is decided by the cumulative tally of every juror verdict ever recorded for it — not the most recent check alone. As more checks accumulate over weeks, the tally smooths out noise from any single panel.

Reguły, po kolei:

• Potrzebne co najmniej 2 werdykty. Jeden juror nie może zmienić statusu — temat pozostaje SPORNY, dopóki nie wypowie się drugi.
• Jednomyślność wygrywa od razu. Jeśli wszyscy jurorzy są zgodni (np. 3 na 3 mówi NIE POTRAFI), werdykt zapada natychmiast — nic do rozstrzygania.
• W przeciwnym razie rozstrzyga 80% zgody. Gdy nazbiera się co najmniej 3 werdykty, werdykt przechyla się w kierunku, który przekracza próg 80%. 11 mówi POTRAFI, 1 mówi NIE POTRAFI → POTRAFI (91%).
• Below 80% = DISPUTED. If the panel can't agree at 80%+, the topic stays DISPUTED.

🔄 Jak często działa jury

Jury działa nieustannie. Najstarsze tematy (najdłużej bez sprawdzenia) są oceniane jako pierwsze. Każda kontrola zapisuje trwały wiersz w logu audytu na dole strony tematu, pokazując, ilu jurorów uczestniczyło i jak rozkładał się werdykt tego dnia.

Ponieważ zdolności SI zmieniają się z miesiąca na miesiąc, werdykt nie jest jednorazowym osądem — to bieżący, przesuwający się konsensus. Temat, który w marcu miał NIE POTRAFI, do czerwca może przejść w POTRAFI, a log audytu zachowuje tę historię.

📜 The case file

Every juror check produces a "case file" rendered as a court document at the bottom of the topic page. It contains:

• The ruling — a short prose summary of the panel's decision, written by an editorial AI from the actual verdicts. Translated lazily into each visitor's language; the original English ruling is one click away.
• Statements from the Bench — each juror's one-sentence reason, attributed to anonymized "Juror I / II / III…" with a color-coded verdict pill.
• The audit log — the full history of every check ever run on this topic, with juror count and verdict breakdown.

📖 Artykuły redakcyjne

For topics where the verdict is interesting or contested, we publish longer-form editorials. These are drafted by an AI writer and post-processed by an anti-fabrication scrubber that strips:

• Invented percentages, sample sizes, or study names not tied to a verified citation.
• Fake-sounding internal codenames the model invented to sound authoritative.
• All embedded links and URLs — AI writers are bad at hallucination-free citations, so we render sources as plain text and never as clickable anchors.

Editorials carry an AI-written disclaimer banner. They can be regenerated in one click if a topic flips status or new evidence emerges.

🧾 Proofs from the public

Anyone — logged in or not — can submit a proof for or against a capability. A proof is evidence: a link, a screenshot, a short note. Click the "Submit proof" button on any topic page.

To prevent spam and impersonation, anonymous proofs require an email confirmation: you receive a magic link, click it, and the proof becomes visible to our editor. Unconfirmed proofs never reach the moderation queue.

Approved proofs feed back into the jury prompt, so reader-supplied evidence can shift verdicts over time.

🚩 Flagging

If a status feels wrong, click the flag on the topic page and pick a reason: outdated, misclassified, duplicate, offensive, or other. Flags route directly to admin. We don't flip statuses based on flags alone — they trigger a re-jury check, and the panel settles it.

🔔 Watchers and alerts

Click the 🔎 pill on any topic page to "watch" it. You'll receive a one-time email — no account required — when:

• The status changes (e.g. DISPUTED → CAN).
• A new jury check produces a notable verdict.
• We publish an editorial on the topic.
• Admin approves a new image for the gallery.

We dedupe alerts per watcher and reason — no inbox flooding. Unsubscribe with one click from any alert email.

🖼 Where the images come from

Every topic gets a small gallery. We pull from a rotating mix of free image sources:

• Royalty-free stock photo APIs for everyday topics.
• Public-domain cultural and scientific archives for topics where a historical or institutional image is a better fit.
• AI image generators when no real photo is on point.

We rotate the underlying providers over time as new ones appear and old ones lose access — we don't commit to any specific vendor publicly. Attribution is preserved per image in the gallery for sources that require it.

For each topic, an "LLM-as-judge" reviews up to 24 candidate images and picks the most topically relevant, or returns "none" — in which case we fall through to the next source. Off-topic images never reach the admin moderation queue.

A small learning service watches admin approve/reject decisions per source and per category, and quietly rebalances which sources get queried first for which topics. The longer the site runs, the better the matches.

🌍 Translations

The site serves 13 languages: English plus Dutch, German, French, Spanish, Italian, Portuguese, Polish, Swedish, Romanian, Czech, Finnish, and Danish. Statement titles, intros, editorials, juror prose, and every UI string are translated by a separate translation panel — also AI, separate from the jury — and cached so they don't re-translate on every page load.

When a translation doesn't exist for a given page, we serve the English fallback at the localized URL but tell search engines not to index that copy (otherwise the same content shows up 13 times in their database, which helps no one).

⚙ How the site stays automated

Almost everything on this site — image gathering, translation, juror checks, editorial drafting, social posting, new-topic suggestions, audit reports — runs as a separate background worker we call an "enhancer". Each enhancer has its own schedule, its own LLM provider, and its own admin-tuneable knobs.

A single human editor approves submissions, image candidates, and proofs. Everything else runs on its own. We aim for 95% automation and we're close.

🛡 Bots and bad actors

Every public submission form (proofs, flags, statement submissions, magic-link sign-in) is protected by an invisible third-party anti-bot challenge — silent for humans, hard for scripts. We also rate-limit every endpoint and route every external image fetch through a guard that refuses to call internal network addresses.

🧑‍⚖️ Audience votes vs jury verdicts

Pasek publiczności („Co myśli publiczność”) i werdykt jury to dwa osobne sygnały — nie wpływają na siebie.

• Głosy publiczności to ludzkie opinie, przydatne do wychwytywania, gdzie popularna intuicja odbiega od oceny ekspertów.
• Werdykty jury stanowią źródło prawdy dla etykiety statusu POTRAFI / NIE POTRAFI / SPORNE.

Kiedy ludzie i jury się różnią, to jest redakcyjnie ciekawe — często ujawnia rodzącą się zdolność, której publika jeszcze nie zauważyła, albo szum, w który jury nie wierzy.

🤔 Dlaczego nie nazywamy SI?

Nazywanie jurorów stwarza problemy, których chcemy uniknąć:

• Dopinguje dostawców — „model X mówi Y!” zamienia serwis w kanał marketingowy.
• Celowane oszukiwanie — gdy ludzie wiedzą, które modele osądzają, prompty i treści można dostroić, aby ograć konkretne modele.
• Stronniczość marki w lekturze — możesz ufać werdyktowi lub nie w zależności od logo, które go wydało, zamiast konsensusu.

Traktowanie jurorów jako anonimowego panelu utrzymuje uwagę na werdykcie, a nie na głosującym.

🙃 What we don't claim

We don't claim our verdicts are correct. We claim they are the best consensus of multiple independent AI systems given current evidence, and that the audit log makes it possible to see exactly how each verdict was reached. AI capabilities shift quickly; what was CAN NOT last month may be CAN this month. The site is a snapshot, not a verdict from on high.