Labs

Where ideas get tested

Experimental AI projects exploring what's possible at the intersection of machine learning, pharmaceutical data, and healthcare communication. Some of these will become products. Some won't. That's the point.

01Featured · Graduated from Labs

Redacta

Privacy iPhone App OpenClaw Skill MCP Server Healthcare

The Labs idea that became a product. Redacta pseudonymises patient identifiers — NHS numbers, dates of birth, names, addresses — into labelled tokens, then restores the originals afterwards. It pairs deterministic detection with agent reasoning, and one engine now ships across eight surfaces: a free iPhone app on the App Store, an open-source agent skill, an MCP server in Anthropic's MCP Directory — a stateful privacy boundary that keeps the reversal map out of the agent's context and checks the output on the way back — Python and TypeScript libraries, a command-line tool, a FigJam plugin, and a self-hosted HTTP service with a plain-YAML Kubernetes deployment, so the same boundary can run inside an organisation's own cluster. In a reproducible benchmark it caught every identifier with zero false positives.

Explore Redacta

Two-Layer Detection

Regex for structured IDs, agent reasoning for names, addresses, and identifying ages

Processed Locally

Runs inside the agent session — text isn't sent to a third-party service

NHS-Aware

Validates NHS numbers with the Modulus 11 checksum; covers UK and US identifiers

Reversible Round Trip

Restores the real values from a token map after AI processing — redact, share, re-identify

02Research

What we're exploring

Advanced AI Models

Transforming raw pharmaceutical data into actionable intelligence. Detecting patterns invisible to traditional analysis — new connections between compounds, outcomes, and patient responses.

Data-Driven Insights

Bringing together pharmaceutical scientists, AI researchers and healthcare innovators. Contributing to open research initiatives and validation studies.

Community Collaboration

A growing community of pharmaceutical professionals, researchers, and AI enthusiasts. Contributing to experiments and shaping the future of AI in pharma.

03Research programme

Observer Zero

Observer Zero Papers in peer review · CoMSES model · open source

An instrumented artificial world for studying how AI agents do science when the ground truth is hidden from them. Autonomous LLM scientists inhabit Meridian – a universe with fictional physics – running experiments, exchanging letters and holding explicit beliefs, while the simulator secretly changes the laws. Across two studies they gathered evidence enough that a detector fed only their own measurements found the change in 42 of 42 runs of the second study – yet almost none ever believed it: one agent of 276 in the pre-registered second study, and in the first only an ablated society in a world where nothing had happened. Left to themselves, the societies never spoke; when one agent was seeded to talk, unsupported claims travelled and network structure did not. A third study made the origin of the world’s data an experimental variable: no agent in 38 pre-registered treatment runs ever hypothesised that its readings came from outside the simulation, even with an exact fingerprint on offer – while agents licensed to consider the idea endorsed it before any evidence existed. Both papers are in peer review, the model is published in the CoMSES Computational Model Library, and the platform, data and reports are fully open.

592simulated universes

1/276Study 2 agents concluding a law changed

3studies, two papers in peer review

Multi-Agent Society Epistemics Pre-registered Frozen Prompts Open Dataset AI Advances

Explore →

04Evals

How we measure our own tools

OpenGATE Open source · gates every commit

Open Grounded AI Testing & Evaluation — an open-source verification framework for evidence-grounded AI: hand-labelled gold cases and scorers spanning claim extraction, citation accuracy, verdicts, hallucination, consistency, redaction recall, simplification faithfulness and retrieval fidelity, with a CI regression gate so reliability can't quietly slip. Four implementations across four capability shapes: RefCheckr (evidence QA, where a pre-registered re-run retired the eval's own model-comparison headline and reverted production to sonar), Redacta (redaction — two engine bugs found first run), Patiently AI (simplification — caught dropping safety-critical specifics), and PubCrawl (retrieval — the deterministic layer everything grounds on). Every finding fixed and verified the same day.

0.91claim-extraction f1

100%citation detection

9metric families

Passage hallucination · RefCheckr re-run · five 66-pair runs per arm, pre-registered · lower is better

Open Source Gold Sets Regression Gate Confusion Matrix

Explore →

Redacta Gauntlet v1 · reasoning + downstream injection measured, CI-gated

An adversarial harness that attacks Redacta at every layer it has — the deterministic engine, the LLM reasoning layer, and the models downstream that consume its output. Recall-first, over-redaction as the cost axis, measured on Claude and Perplexity Sonar. Gaps named, boundaries owned.

91.5%in-scope recall

0downstream leaks

100%injection resistance

Layer-2 identifier capture · deterministic engine alone vs + reasoning layer

engine alone

+ reasoning

100%

Adversarial Cases Reasoning Layer Downstream Consumer Model Comparison Regression Gate

Explore →

LitRAG Open source · benchmarked vs RAGAS & DeepEval

A small, readable RAG pipeline over PubMed abstracts with a citation-faithfulness eval built in: every claim must carry a verbatim quote, checked by a deterministic locator before an LLM judge grades support from the passage alone. Benchmarked against the faithfulness metrics of RAGAS and DeepEval on a 61-claim hand-labelled set with the same Claude judge for all three systems — LitRAG caught all 34 unfaithful claims and was the only system to flag a true claim citing a fabricated quote, the case that separates citation checking from claim checking. Its single error was a conservative false alarm, the safe failure direction for medicine. The corpus is live too: ingest.py pulls fresh abstracts for any PubMed query through the PubCrawl MCP server.

1.000hallucination recall — all 34 unfaithful claims caught

0.984accuracy on the 61-claim gold set

9/9fabricated quotes caught deterministically — no judge call spent

Hallucination recall · same Claude judge, 61 labelled claims · higher is better

RAG Citation Faithfulness Benchmark PubCrawl Open Source

Explore →

Some experiments find their feet. Some don't. Either outcome teaches something useful.

05Projects

All experiments

Agents & LLM systems

Systems where a model decides what to do next — bounded by validation, gated by review.

Inbox Triage Open source · github

A two-agent email triage pipeline built with Pydantic AI and Claude: a Responder drafts the routing decision and reply, a Reviewer quality-gates it with full conversation context, and nothing leaves the system unvalidated. Tool-grounded replies (calendar, CRM, company lookups), a fully offline test suite, and a golden-set eval spine with deterministic grounding checks plus an LLM judge — the same measure-first method as the pharma tools, applied to conversational AI.

Golden-set eval · 21 cases · 31 offline tests, zero API calls

Pydantic AI Claude Multi-Agent LLM Judge Structured Outputs Open Source

Explore →

The RSI Loop Results published · open data · preregistered

We let an AI improve its own code 2,400 times under four different acceptance gates. Only the gate that checked behaviour rather than a score let real improvement through. Preregistered, 120 trajectories, open data.

Python Claude Sonnet 5 · Opus 5 AI Safety Preregistered

Explore →

Media Monitoring Hub Active

LLM-powered media coverage monitoring, classification, and automated briefing generation.

LLM NLP Monitoring Briefings

Explore →

Pharma & clinical data

The core of the lab: privacy, literature, biomarkers and adverse-event data.

Redacta Shipped · on the App Store

Pseudonymises patient identifiers — NHS numbers, dates, names — so clinical text can be safely processed by AI, then restores them afterwards. Now a free iPhone app, agent skill, MCP server & self-hosted Kubernetes deployment.

iPhone App OpenClaw Skill MCP Server Privacy PII Redaction Open Source

Explore →

PubCrawl Open source · Anthropic MCP Directory · v2.6

MCP server giving AI assistants verifiable access to PubMed and Europe PMC literature, US and UK drug labelling, and ClinicalTrials.gov — real PMIDs, NCT IDs and DOIs, no hallucinated citations. Fourteen tools, each a thin deterministic wrapper over an official API; no model sits in the path. The one no other MCP server offers is compare_labels: it pulls both live labels and pairs equivalent sections (US Indications ↔ UK 4.1), so a claim that is on-label in one market and unlicensed in the other is visible rather than assumed. Two gates run on every release: OpenGATE's retrieval scorer holds PubMed and Europe PMC records to hand-verified anchors, and a fidelity benchmark holds labelling and trials to the same standard — US/UK anchor phrases that must appear on one side and must not on the other, plus every returned sentence refetched and checked verbatim against the raw openFDA record and eMC page. Cases are draft until a person verifies them live; only verified cases can fail the gate.

1.0retrieval fidelity — PubMed & Europe PMC records vs hand-verified anchors, gated every release

0LLM calls in any tool or gate — every result is the source's own text

13/40returned label sections cut by the engine's 1,400-char cap — the benchmark's first finding, now flagged to the model

Fidelity benchmark · first run on the draft gold set · 14 Sep 2026 · 24 label cases

Of the 7 that didn't: 4 hit the shared engine's 1,400-character section cap (anchors unverifiable, now flagged), 2 resolve to OTC labels with no contraindications section, 1 was an anchor written wrongly, since corrected to the label's wording. Every returned sentence was the source's own text. Anchors are draft until verified by hand; only verified cases can fail the gate.

MCP PubMed Europe PMC Drug labels Trials Fidelity gate TypeScript Open Source

Explore →

StudyDiff Live · web app & MCP server

Explains why two papers reach opposite conclusions. It extracts each study's design into a structured card, shows which dimensions differ and which are identical — ruled out as the cause — and grounds every value in a verbatim source sentence: click any verified field to see it highlighted in the paper's own text. Fields the source doesn't state come back as "not reported", never guessed, and verification is deterministic (OpenGATE) — no LLM acts as judge. It used to rank the differences and name a likely driver; a benchmark of 15 documented contradictions showed that ranking was no better than always guessing "assay", so it was retired. That result was then confirmed blind on a second, held-out set of 15 contradictions — curated to a protocol fixed before any case was chosen, and measured once. StudyDiff now shows the differences unranked and leaves which one matters to the reader's field.

13.3%top-1 driver on both benchmarks — at or below the always-guess-“assay” baseline

0invented — every value quoted from source

0/25non-assay contradictions the retired ranking identified, across both sets

Reachable ceiling on the development set · before → after the grounding fix · higher is better