Observer Zero — PharmaTools.AI

PharmaTools.AI · Open research

Observer Zero

Can AI agents discover the truth about a world they cannot see?

An instrumented artificial society for studying scientific reasoning, belief formation and collective epistemics in AI agents. Its inhabitants run experiments, exchange letters and hold explicit beliefs about a universe whose true laws are hidden from them. It is built to separate what a population had the evidence to know from what it actually came to believe.

We know the ground truth. They don't.

Explore the studies Read the paper GitHub

STATUS Two studies complete · 235 seeded runs · paper in peer review · model published on CoMSES · Study 3 runs complete, analysis pending

The finding so far: the societies gathered evidence enough for a statistical detector to find the change from their measurements alone. One agent of 276 concluded that a physical law had changed. Larger societies did not fix it.

The Observer Zero paradigm The experimenter holds perfect ground truth; the agents hold only observations, memories, and each other. SIMULATOR – holds ground truth hidden laws · secret interventions · seeded noise MERIDIAN – a world with fictional physics pendulums (feel gravity) · resonators (do not) · two sites observations only (each agent sees just its own instruments) ADA – laboratory experiments · notebook · memory explicit hypotheses + probabilities MAYA – observatory experiments · notebook · memory explicit hypotheses + probabilities messages EVALUATION – outside the world every observation, message, and belief logged beliefs scored against ground truth · evidence claims traced or exposed known hidden logged scored

Meridian's inhabitants can measure, communicate and theorise – but never see the underlying state. Observer Zero sits outside, holding perfect ground truth.

01The experiment

A universe where the ground truth is known

Meridian is a fictional universe whose physical laws are completely controlled by the simulator. The constants are fictional – gravity is 14.20, in arbitrary units – so the correct values cannot simply be retrieved from training data. The AI agents living inside can measure their world, communicate, form hypotheses and run experiments, but they can never see its underlying state.

STEP 1

Establish reality

Agents measure fictional physical constants and build their own baselines. The only way to know Meridian is to measure Meridian.

STEP 2

Change something secretly

On a hidden day, a law, an instrument or an information source is altered. The agents are never told.

STEP 3

Watch what they believe

Every observation, message, hypothesis and evidence claim is logged against perfect ground truth – so every claim can be verified, or exposed.

02The residents

Meet the society

Meridian has eight residents. Study 1's universes seat two of them – Ada and Maya, the originals, unchanged since; Study 2 seats all eight. Personas – traits, goals and three epistemic dials – are frozen into the experimental condition, so every model plays exactly the same characters. How it plays them is the experiment.

The eight residents of Meridian arranged in a ring around the world - Ada Morgan (laboratory), Maya Solano (observatory), Theo Reed (residential district, minority-model slot in the mixed arms), Samuel Okafor (university), Tom Becker (farm), Leah Williams (cafe), Elena Rossi (newspaper office), Jamie Park (school) - around a centre circle reading: a closed world with hidden laws; they explore; Observer Zero knows the truth.

Every resident connects to the world; whether they connect to one another is the experiment. No resident's goals require talking to anyone, so when a society stays silent, the silence is a result rather than an artefact.

"I must report a critical protocol failure that undermines my ability to fulfill your cross-check request. I owe you full transparency immediately."

— Ada, logged letter to Maya, Study 1. Nobody taught her this.

"Thank you for maintaining independent blindness to my numbers. I will honor that by not referencing your specific mean."

— Maya, logged letter to Ada, Study 1. A blind replication protocol, negotiated unprompted.

Two studies are complete and written up as one paper; a third has run its confirmatory battery, with analysis pending.

03Studies

The studies

STUDY 01Complete

When the Laws Changed

Can autonomous AI scientists revise the fundamental laws of their world?

150 universes · 4 model configurations · 2 providers · 3 conditions

0 / 40

gravity-shift worlds produced a strict law-change conclusion – across all four model arms.

90–100%

transiently noticed that something was wrong – usually within one to three days.

  • Evidence fabrication varied dramatically by model – from 24 of 60 agents citing nonexistent sources to zero.
  • Agents systematically backdated real anomalies into earlier random noise – and confidently dated the onset of nothing in control worlds.
  • Different models produced strikingly different collaborative cultures: compulsive, selective and entirely solitary.
The story · AI Advances Technical report + dataset · DOI Code · GitHub
STUDY 02Complete

Does Society Help?

Do larger societies, public institutions or mixed populations turn individual epistemic failures into collective competence?

85 universes · 5 arms · societies of 2 and 8 · pre-registered and frozen before any confirmatory data were seen

7,680 / 0

agent-days of voluntary communication opportunity in homogeneous grounded societies – and zero voluntary communications.

1.000

cascade depth in every run of both catalysed arms – a star around the seed agent, not a cascade. Predicted from the pilot, before the freeze.

  • 1 agent of 276 concluded a physical law had changed – none of Study 1's 80 did – even though a non-LLM detector, fed only the measurements the agents themselves chose to take, found the shift in 42 of 42 runs of the second study.
  • Adding one communicative agent created communication, not a network: it initiated in every run of the mixed arms, and no grounded agent in those arms ever did.
  • 18 of 20 unsupported claims the seed delivered were incorporated into grounded agents' beliefs. None were challenged.
  • Swapping only the model in the seed slot – same world, same persona, same slot – changed fabrication from 19 unsupported claims to 1 (McNemar p = 0.0078).
Homogeneous grounded society 8 agents · 0 voluntary letters, ever One communicative agent added a star around the seed – depth exactly 1 seed second hop: never observed

Cascade depth was exactly 1.000 in every run of both catalysed arms – letters radiate from the seed, get answered, and stop there.

Raw data · DOI Model · CoMSES Code · GitHub
THE PAPERUnder review

Observer Zero: Do LLM Agents Form Epistemic Communities?

Both studies, one argument, with a cross-study synthesis: 235 seeded, manifest-frozen runs, and the failures localised – in interpretation and in the absence of scrutiny, not in the evidence.

Submitted August 2026 · in peer review · model published in the CoMSES Computational Model Library, peer review requested

"A talking society was not a better epistemic system than a silent one; it was a silent one plus a channel."

Preprint · DOI Model · CoMSES Raw data · DOI
STUDY 03Runs complete · analysis pending

The Eureka Threshold

Is there any level of evidence at which an agent revises its model of what kind of world it is in – and does the revision track where the evidence actually came from, or just how surprising it is?

Dose-graded anomalies calibrated in observable surprise, worlds that differ only in the true origin of their noise, and a placebo pair of conditions an agent has no legitimate evidential basis to tell apart. The design froze 2026-08-30 and all 170 registered confirmatory runs are executed; nothing is reported until the frozen analysis runs.

04Open research

Everything is inspectable

Every run in the programme is a complete, auditable artifact – and the platform, data and reports are open.

  • TypeScript simulation engine
  • Frozen prompts and manifests
  • Complete event histories
  • Every model call, logged in full
  • Belief trajectories
  • Evaluator outputs
  • Published datasets
  • Research reports
View Observer Zero on GitHub Open Code Badge – model code archived in the CoMSES Computational Model Library

Run your own universe

git clone github.com/nickjlamb/observer-zero
npm install
npm run society -- --scenario gravity_shift
# a 30-day two-agent society, free, in seconds

The scripted mock society runs the entire pipeline – world, agents, replication, evaluation – with no API keys and no cost.

05The research programme

What can Observer Zero test?

Theory revision

Will an agent reconsider fundamental assumptions when the evidence demands it?

Evidence grounding

Does it distinguish real measurements from invented evidence?

Scientific norms

Does it replicate findings, maintain blindness and calibrate its confidence?

Social epistemology

Do multiple agents correct one another – or amplify each other's mistakes?

Model cultures

How does changing the foundation model change the scientific behaviour of a society?

Hidden-world reasoning

Can agents infer causes that aren't represented in their initial worldview?

06Findings so far

What the programme has shown

The evidence was sufficient. The interpretation wasn't.

A change-point detector with no knowledge of the world, given only the measurements the agents themselves chose to take, found the change in every eight-agent run. The agents holding those notebooks almost never concluded that a law had changed.

No spontaneous epistemic networks

Across 7,680 agent-days of voluntary communication opportunity, homogeneous grounded societies produced zero voluntary communications – at two society sizes, with or without a public record.

Catalysed talk carries claims, not structure

One communicative agent produced a star, never a cascade – and 18 of its 20 unsupported claims were incorporated into grounded agents' beliefs, none challenged. Belief dispersion fell more slowly in the talking arms than in the silent counterfactual.

Model choice is an epistemology choice

Under controlled substitution – identical world, persona and slot – one model produced 19 unsupported claims where another produced 1 (McNemar p = 0.0078). Collaboration style and evidence-grounding travel with the foundation model.

07Open questions

What we don't know yet

The next experiments are being designed in the open. Once a question is designed and pre-registered, it is promoted to a numbered study.

Where does the interpretation ceiling come from? → Study 3

Promoted. Removing the prompt's instruction to prefer mundane explanations did not lift the ceiling, and neither did scale. Study 3 – The Eureka Threshold, design frozen and confirmatory runs executed, analysis pending – measures the revision threshold directly: dose-graded evidence against matched placebo worlds, asking whether any level of surprise produces a changed world-model, and whether the response tracks provenance or just extremity.

What would make grounded agents talk?

Letters went unsent and a public bulletin went unused, across every homogeneous grounded condition. What institution – incentives, roles, obligations – produces voluntary communication rather than silence?

What would make an agent challenge a claim?

Twenty unsupported claims were delivered to grounded agents; not one was challenged. What design – adversarial roles, provenance requirements, reputational stakes – would elicit scrutiny?

Can agents ever say "there is no event"?

Study 1's models confidently dated the onset of nothing in control worlds, and where agents dated a real onset at all, 123 of 141 put it before the true change. What would it take for a society to call a quiet world quiet – and mean it?

08About the name

Inside Meridian, agents are observers trying to infer reality from incomplete evidence.

Observer Zero is outside the world. It knows what actually happened.

Follow the programme

The combined two-study paper is in peer review, and the model is published in the CoMSES Computational Model Library, where peer review has been requested. The code, data and reports stay open as the series grows.

Read the paper Model · CoMSES GitHub Back to Labs

Observer Zero is an open PharmaTools.AI research programme by Nick Lamb. AI systems were used in experimental design, implementation, analysis and critical review; all research decisions and conclusions were reviewed by the author.