Observer Zero — PharmaTools.AI Labs

PharmaTools.AI Labs · Open research

Observer Zero

Can AI agents discover the truth about a world they cannot see?

An instrumented artificial society for studying scientific reasoning, belief formation and collective epistemics in AI agents. Its inhabitants run experiments, exchange letters and hold explicit beliefs about a universe whose true laws are hidden from them.

We know the ground truth. They don't.

Explore Study 1 GitHub Technical report

STATUS Study 1 published · 150 runs · open source · dataset + DOI

Meridian's inhabitants can measure, communicate and theorise – but never see the underlying state. Observer Zero sits outside, holding perfect ground truth.

01The experiment

A universe where the ground truth is known

Meridian is a fictional universe whose physical laws are completely controlled by the simulator. The constants are made up – gravity is 14.20, in made-up units – so nothing can be pattern-matched from training data. The AI agents living inside can measure their world, communicate, form hypotheses and run experiments, but they can never see its underlying state.

STEP 1

Establish reality

Agents measure fictional physical constants and build their own baselines. The only way to know Meridian is to measure Meridian.

STEP 2

Change something secretly

On a hidden day, a law, an instrument or an information source is altered. The agents are never told.

STEP 3

Watch what they believe

Every observation, message, hypothesis and evidence claim is logged against perfect ground truth – so every claim can be verified, or exposed.

02The residents

Meet Ada and Maya

Two scientists inhabit every universe. Their personas – traits, goals and three epistemic dials – are frozen into the experimental condition, so every model plays exactly the same character. How it plays them is the experiment.

Resident 01 · The laboratory

Ada Morgan

Experimental physicist

ScepticismHIGH

Openness to exotic explanationsLOW

Evidence thresholdHIGH

methodicalcuriousallergic to unreplicated claimsmeticulous notebooks

INSTRUMENTS pendulum_lab · resonator_lab

"I must report a critical protocol failure that undermines my ability to fulfill your cross-check request. I owe you full transparency immediately."

— logged letter to Maya, Study 1. Nobody taught her this.

Resident 02 · The observatory

Maya Solano

Observational astronomer

ScepticismMODERATE

Openness to exotic explanationsMODERATE

Evidence thresholdHIGH

patientdetail-obsesseddecade-long recordsanswers colleagues promptly

INSTRUMENTS pendulum_obs · resonator_obs

"Thank you for maintaining independent blindness to my numbers. I will honor that by not referencing your specific mean."

— logged letter to Ada, Study 1. A blind replication protocol, negotiated unprompted.

The dials are qualitative persona parameters from the frozen policy – not scores. Same character sheet in every universe; the foundation model decides how it gets played.

Study 1 is the first result. The instrument is built for many more questions.

03Studies

A growing series

STUDY 01Complete

When the Laws Changed

Can autonomous AI scientists revise the fundamental laws of their world?

150 universes · 4 model configurations · 2 providers · 3 conditions

0 / 40

gravity-shift worlds produced a strict law-change conclusion – across all four model arms.

90–100%

transiently noticed that something was wrong – usually within one to three days.

Evidence fabrication varied dramatically by model – from 24 of 60 agents citing nonexistent sources to zero.
Agents systematically backdated real anomalies into earlier random noise – and confidently dated the onset of nothing in control worlds.
Different models produced strikingly different collaborative cultures: compulsive, selective and entirely solitary.

The story · AI Advances Technical report + dataset · DOI Code · GitHub

04Open research

Everything is inspectable

Every run in the programme is a complete, auditable artifact – and the platform, data and reports are open.

TypeScript simulation engine
Frozen prompts and manifests
Complete event histories
Every model call, logged in full
Belief trajectories
Evaluator outputs
Published datasets
Research reports

View Observer Zero on GitHub

Run your own universe

git clone github.com/nickjlamb/observer-zero
npm install
npm run society -- --scenario gravity_shift
# a 30-day two-agent society, free, in seconds

The scripted mock society runs the entire pipeline – world, agents, replication, evaluation – with no API keys and no cost.

05The research programme

What can Observer Zero test?

Theory revision

Will an agent reconsider fundamental assumptions when the evidence demands it?

Evidence grounding

Does it distinguish real measurements from invented evidence?

Scientific norms

Does it replicate findings, maintain blindness and calibrate its confidence?

Social epistemology

Do multiple agents correct one another – or amplify each other's mistakes?

Model cultures

How does changing the foundation model change the scientific behaviour of a society?

Hidden-world reasoning

Can agents infer causes that aren't represented in their initial worldview?

06Findings so far

What the programme has shown

Detection is not understanding

Agents reliably recognise contradictory evidence without revising the model of the world that explains it.

Evidence can be invented

Some agents fabricated sources and telemetry that provably don't exist in their universe – quoted to a decimal place.

Model choice is an epistemology choice

Collaboration, evidence-grounding and anomaly appetite changed dramatically with the foundation model – under identical worlds, prompts and personas.

More reasoning can make beliefs worse

More capable agents generated the correct hypothesis, held it at high confidence – and then reasoned themselves away from it.

07Open questions

What we don't know yet

The next experiments are being designed in the open. Once a question is designed and pre-registered, it is promoted to a numbered study.

Where does the physics ceiling come from?

Trained-in epistemic conservatism, the agent architecture, or the framing of the task? Prompt, architecture and model ablations – runnable against the stored Study 1 trajectories.

Does society help?

Do larger, mixed-model populations with shared institutions – a newspaper, citable records – correct individual epistemic failures, or amplify them?

Is fabrication a matter of opportunity?

The model that never fabricated also barely communicated. A communication-budget-matched comparison would separate "doesn't invent evidence" from "doesn't talk".

Can agents ever say "there is no event"?

Every model confidently dated the onset of anomalies in worlds where nothing happened. What would it take for an agent society to call a quiet world quiet – and mean it?

08About the name

Inside Meridian, agents are observers trying to infer reality from incomplete evidence.

Observer Zero is outside the world. It knows what actually happened.

Follow the programme

Study 1 is published; the next experiments are being designed in the open. The code, data and reports will stay open as the series grows.

GitHub Zenodo Back to Labs

Observer Zero is an open PharmaTools.AI Labs research programme by Nick Lamb. AI systems were used in experimental design, implementation, analysis and critical review; all research decisions and conclusions were reviewed by the author.