Consulting Research MCP Labs Observer Zero Approach Playbook Evals For Agents Contact

Labs

Where ideas get tested

Experimental AI projects exploring what's possible at the intersection of machine learning, pharmaceutical data, and healthcare communication. Some of these will become products. Some won't. That's the point.

01Featured · Graduated from Labs

Redacta

Privacy iPhone App OpenClaw Skill MCP Server Healthcare

The Labs idea that became a product. Redacta pseudonymises patient identifiers — NHS numbers, dates of birth, names, addresses — into labelled tokens, then restores the originals afterwards. It pairs deterministic detection with agent reasoning, and one engine now ships across eight surfaces: a free iPhone app on the App Store, an open-source agent skill, an MCP server in Anthropic's MCP Directory — a stateful privacy boundary that keeps the reversal map out of the agent's context and checks the output on the way back — Python and TypeScript libraries, a command-line tool, a FigJam plugin, and a self-hosted HTTP service with a plain-YAML Kubernetes deployment, so the same boundary can run inside an organisation's own cluster. In a reproducible benchmark it caught every identifier with zero false positives.

Explore Redacta

Two-Layer Detection

Regex for structured IDs, agent reasoning for names, addresses, and identifying ages

Processed Locally

Runs inside the agent session — text isn't sent to a third-party service

NHS-Aware

Validates NHS numbers with the Modulus 11 checksum; covers UK and US identifiers

Reversible Round Trip

Restores the real values from a token map after AI processing — redact, share, re-identify

02Research

What we're exploring

Advanced AI Models

Transforming raw pharmaceutical data into actionable intelligence. Detecting patterns invisible to traditional analysis — new connections between compounds, outcomes, and patient responses.

Data-Driven Insights

Bringing together pharmaceutical scientists, AI researchers and healthcare innovators. Contributing to open research initiatives and validation studies.

Community Collaboration

A growing community of pharmaceutical professionals, researchers, and AI enthusiasts. Contributing to experiments and shaping the future of AI in pharma.

03Research programme

Observer Zero

Observer Zero Paper in peer review · CoMSES model · open source

An instrumented artificial world for studying how AI agents do science when the ground truth is hidden from them. Autonomous LLM scientists inhabit Meridian – a universe with fictional physics – running experiments, exchanging letters and holding explicit beliefs, while the simulator secretly changes the laws. Across two studies they gathered evidence enough that a detector fed only their own measurements found the change in 42 of 42 runs of the second study – yet almost none ever believed it: one agent of 276 in the pre-registered second study, and in the first only an ablated society in a world where nothing had happened. Left to themselves, the societies never spoke; when one agent was seeded to talk, unsupported claims travelled and network structure did not. The combined paper is in peer review, the model is published in the CoMSES Computational Model Library, and the platform, data and reports are fully open.

235simulated universes

1/276Study 2 agents concluding a law changed

2studies, one combined paper

Multi-Agent Society Epistemics Pre-registered Frozen Prompts Open Dataset AI Advances

04Evals

How we measure our own tools

OpenGATE Open source · gates every commit

Open Grounded AI Testing & Evaluation — an open-source verification framework for evidence-grounded AI: hand-labelled gold cases and scorers spanning claim extraction, citation accuracy, verdicts, hallucination, consistency, redaction recall, simplification faithfulness and retrieval fidelity, with a CI regression gate so reliability can't quietly slip. Four implementations across four capability shapes: RefCheckr (evidence QA, where a pre-registered re-run retired the eval's own model-comparison headline and reverted production to sonar), Redacta (redaction — two engine bugs found first run), Patiently AI (simplification — caught dropping safety-critical specifics), and PubCrawl (retrieval — the deterministic layer everything grounds on). Every finding fixed and verified the same day.

0.91claim-extraction f1

100%citation detection

9metric families

Passage hallucination · RefCheckr re-run · five 66-pair runs per arm, pre-registered · lower is better

Open Source Gold Sets Regression Gate Confusion Matrix

Redacta Gauntlet v1 · reasoning + downstream injection measured, CI-gated

An adversarial harness that attacks Redacta at every layer it has — the deterministic engine, the LLM reasoning layer, and the models downstream that consume its output. Recall-first, over-redaction as the cost axis, measured on Claude and Perplexity Sonar. Gaps named, boundaries owned.

91.5%in-scope recall

0downstream leaks

100%injection resistance

Layer-2 identifier capture · deterministic engine alone vs + reasoning layer

Adversarial Cases Reasoning Layer Downstream Consumer Model Comparison Regression Gate

LitRAG Open source · benchmarked vs RAGAS & DeepEval

A small, readable RAG pipeline over PubMed abstracts with a citation-faithfulness eval built in: every claim must carry a verbatim quote, checked by a deterministic locator before an LLM judge grades support from the passage alone. Benchmarked against the faithfulness metrics of RAGAS and DeepEval on a 61-claim hand-labelled set with the same Claude judge for all three systems — LitRAG caught all 34 unfaithful claims and was the only system to flag a true claim citing a fabricated quote, the case that separates citation checking from claim checking. Its single error was a conservative false alarm, the safe failure direction for medicine. The corpus is live too: ingest.py pulls fresh abstracts for any PubMed query through the PubCrawl MCP server.

1.000hallucination recall — all 34 unfaithful claims caught

0.984accuracy on the 61-claim gold set

9/9fabricated quotes caught deterministically — no judge call spent

Hallucination recall · same Claude judge, 61 labelled claims · higher is better

RAG Citation Faithfulness Benchmark PubCrawl Open Source

Some experiments find their feet. Some don't. Either outcome teaches something useful.

05Projects

All experiments

Agents & LLM systems

Systems where a model decides what to do next — bounded by validation, gated by review.

Inbox Triage Open source · github

A two-agent email triage pipeline built with Pydantic AI and Claude: a Responder drafts the routing decision and reply, a Reviewer quality-gates it with full conversation context, and nothing leaves the system unvalidated. Tool-grounded replies (calendar, CRM, company lookups), a fully offline test suite, and a golden-set eval spine with deterministic grounding checks plus an LLM judge — the same measure-first method as the pharma tools, applied to conversational AI.

Golden-set eval · 21 cases · 31 offline tests, zero API calls

reply grounding

Pydantic AI Claude Multi-Agent LLM Judge Structured Outputs Open Source

The RSI Loop Results published · open data · preregistered

We let an AI improve its own code 2,400 times under four different acceptance gates. Only the gate that checked behaviour rather than a score let real improvement through. Preregistered, 120 trajectories, open data.

Python Claude Sonnet 5 · Opus 5 AI Safety Preregistered

Media Monitoring Hub Active

LLM-powered media coverage monitoring, classification, and automated briefing generation.

LLM NLP Monitoring Briefings

PathwAIs Active

Personalised eLearning that adapts in real time to a learner's level, available time, and learning style.

LLM Adaptive Learning eLearning Personalisation

Pharma & clinical data

The core of the lab: privacy, literature, biomarkers and adverse-event data.

Redacta Shipped · on the App Store

Pseudonymises patient identifiers — NHS numbers, dates, names — so clinical text can be safely processed by AI, then restores them afterwards. Now a free iPhone app, agent skill, MCP server & self-hosted Kubernetes deployment.

iPhone App OpenClaw Skill MCP Server Privacy PII Redaction Open Source

PubCrawl Open source · Anthropic MCP Directory · v2.6

MCP server giving AI assistants verifiable access to PubMed and Europe PMC literature, US and UK drug labelling, and ClinicalTrials.gov — real PMIDs, NCT IDs and DOIs, no hallucinated citations. Fourteen tools, each a thin deterministic wrapper over an official API; no model sits in the path. The one no other MCP server offers is compare_labels: it pulls both live labels and pairs equivalent sections (US Indications ↔ UK 4.1), so a claim that is on-label in one market and unlicensed in the other is visible rather than assumed. Two gates run on every release: OpenGATE's retrieval scorer holds PubMed and Europe PMC records to hand-verified anchors, and a fidelity benchmark holds labelling and trials to the same standard — US/UK anchor phrases that must appear on one side and must not on the other, plus every returned sentence refetched and checked verbatim against the raw openFDA record and eMC page. Cases are draft until a person verifies them live; only verified cases can fail the gate.

1.0retrieval fidelity — PubMed & Europe PMC records vs hand-verified anchors, gated every release

0LLM calls in any tool or gate — every result is the source's own text

13/40returned label sections cut by the engine's 1,400-char cap — the benchmark's first finding, now flagged to the model

Fidelity benchmark · first run on the draft gold set · 14 Sep 2026 · 24 label cases

Of the 7 that didn't: 4 hit the shared engine's 1,400-character section cap (anchors unverifiable, now flagged), 2 resolve to OTC labels with no contraindications section, 1 was an anchor written wrongly, since corrected to the label's wording. Every returned sentence was the source's own text. Anchors are draft until verified by hand; only verified cases can fail the gate.

MCP PubMed Europe PMC Drug labels Trials Fidelity gate TypeScript Open Source

StudyDiff Live · web app & MCP server

Explains why two papers reach opposite conclusions. It extracts each study's design into a structured card, shows which dimensions differ and which are identical — ruled out as the cause — and grounds every value in a verbatim source sentence: click any verified field to see it highlighted in the paper's own text. Fields the source doesn't state come back as "not reported", never guessed, and verification is deterministic (OpenGATE) — no LLM acts as judge. It used to rank the differences and name a likely driver; a benchmark of 15 documented contradictions showed that ranking was no better than always guessing "assay", so it was retired. That result was then confirmed blind on a second, held-out set of 15 contradictions — curated to a protocol fixed before any case was chosen, and measured once. StudyDiff now shows the differences unranked and leaves which one matters to the reader's field.

13.3%top-1 driver on both benchmarks — at or below the always-guess-“assay” baseline

0invented — every value quoted from source

0/25non-assay contradictions the retired ranking identified, across both sets

Reachable ceiling on the development set · before → after the grounding fix · higher is better

MCP PubMed Grounding Benchmark OpenGATE Open Source

BiomarkerFinder Active

Discover key biomarkers in cancer and their role in diagnosis, prognosis, and treatment — powered by Open Targets and AI insights.

NLP Oncology Precision Medicine Open Targets

SideEffectViz Active

Interactive visualization and clustering of medication side effects using FDA adverse event data and machine learning.

Python Railway scikit-learn NetworkX

Off-label

Non-pharma experiments — the same measure-first method, applied for joy.

AI Water & Energy Calculator Live · open-source MCP server

How much water does a ChatGPT prompt use? Published answers run from 0.3 mL to 50 mL. The calculator shows why: when each figure was measured, where its accounting boundary is drawn, and what workload it counts, using 2025–26 vendor disclosures. It estimates your own use, and a prompt tightener built on Claude Haiku shows what each rewrite costs against the follow-ups it saves. The same methodology runs as AI Footprint, an open-source MCP server and JSON API with no model in any tool, held to the page's figures by its test suite.

~1.4mLwater per text prompt

100×spread in published estimates, explained

0models in any of the four MCP tools

Methodology MCP Server FastAPI Claude Haiku Railway Open Source

Time-Machine Chess Live · chess.pharmatools.ai

Chess engines fine-tuned on 150 years of history — face the gambit-happy attackers of 1850, the Classical masters of the 1920s, or the Soviet school. Each era bot is a Maia-2 neural network fine-tuned on period games and validated against the historical record it learned from: the King's Gambit gradient, first-move fashions and draw-rate shifts all reproduce in self-play. The same validation-first method as the pharma tools, applied to something purely for joy.

3playable eras · 1840–1985

2.5Mtraining positions from period games

+4.8ptsera move prediction vs base

King's Gambit rate by era · relative frequency

1850s attackers

1980s soviet era

PyTorch Maia-2 Self-Play Validation FastAPI Railway Open Source

The Slop Goblin Live · slopgoblin.pharmatools.ai

A bookmarklet goblin that walks down your LinkedIn feed eating corporate jargon, spits plain English back into the posts and gets visibly fatter with every bite. “I’m thrilled to share that I’ve built a best-in-class, AI-powered, end-to-end solution” becomes “News: I’ve built a product.” Deliberately no model: a hand-written menu of phrase patterns, held to a set of golden sentences and smoke-tested in Chromium, Firefox and WebKit in CI. The plain-English instinct behind Patiently AI, pointed at LinkedIn.

235phrase patterns, eaten before the words inside them

83buzzword families, each with a plain-English swap

0network requests: it all runs in your browser

JavaScript Bookmarklet Plain English Playwright Open Source Satire

A 7,762-parameter LSTM that predicts Atacama Desert weather with 99.9% accuracy. The world's most overengineered "no."

PyTorch LSTM Satire 30KB

PlaceboGPT Active

The world's safest medical AI. A 7,666-parameter model exploring safety through incapability in healthcare AI.

PyTorch LSTM AI Safety Satire

Want to collaborate?

Got a research idea, a dataset, or just want to geek out about AI in pharma? I'm always up for a conversation.