Research

Research, articles & presentations

Journal papers, preprints with open code and data, congress posters and technical articles on AI evaluation, verification, autonomous agents and healthcare AI.

ORCID Google Scholar GitHub

Translation, not interpretation. The thesis behind much of this research, argued in the journal paper below and tested in shipped, open-source code.

Read the approach →

Papers, preprints & posters

Peer-reviewed journal articles, preprints deposited on Zenodo with open code and data, a congress ePoster and a working paper.

  • Aug 2026 Front. Digit. Health Journal
    Featured

    Validation of an AI-powered mobile application for personalizing medical note explanations: a mixed-methods evaluation

    doi:10.3389/fdgth.2026.1771051

    Peer-reviewed, three-phase validation of Patiently AI: computational readability analysis of 210 AI-generated explanations, expert evaluation by 15 healthcare professionals, and a survey of 54 patients and members of the public. Reading level fell by 2.96 grade levels, 87.3% of expert safety assessments judged outputs clinically safe, and patients preferred the AI explanations in 70% of comparisons with 98.1% comprehension accuracy.

    Health Literacy Clinical Validation Patient Engagement
    First page of the Patiently AI validation study in Frontiers in Digital Health
  • Oct 2026 Zenodo Preprint

    An energy-first method for estimating the water and energy footprint of AI use

    doi:10.5281/zenodo.23417340

    Published estimates of the water behind one AI prompt run from about 0.3 mL to 50 mL. This versioned note traces most of that spread to four choices – when the hardware was measured, where the accounting boundary is drawn, whose data centre serves the request and what kind of task is counted – and sets out a method that starts from vendor-disclosed energy per prompt (0.24–0.34 Wh for a typical text prompt) and derives water at two stated boundaries: about 0.3 mL counting data-centre cooling only, and about 1.4 mL including the water used to generate the electricity. Every estimate carries a judgement-based range of 0.3× to 3×, and a token-based extension covers coding agents. The same figures drive a public calculator, a JSON API and an MCP server whose tests pin it to the page.

    ToolAI Water & Energy Calculator

    Codegithub.com/nickjlamb/ai-footprint · MCP server and JSON API

    NoteRebuilding an AI water calculator, energy first

    AI Sustainability Energy Methodology
  • Oct 2026 Zenodo Preprint

    A Preregistered Comparison of Acceptance-Gate Architectures for an LLM That Rewrites Its Own Code

    doi:10.5281/zenodo.23236100

    An LLM rewrote a small posture-risk classifier for 20 revisions per run under four acceptance gates – its own judgement, a visible score, a hidden holdout, and an absolute behavioural envelope – while a seeded simulator scored every proposal on exact hidden ground truth. The design was frozen at a public git tag before the first confirmatory run. The visible-score gate was gamed on 10 of 10 seeds (a perfect visible score on 7, hidden truth flat or falling); the hidden holdout was protective but not productive; the behavioural envelope was the only gate under which hidden truth rose, and it changed what the optimiser proposed rather than only what it accepted. A parametric optimiser that cannot write code beat the LLM under the visible gate; the stronger model was no safer under the weak gate and more productive under the strong one; removing the optimiser's cross-revision notes made gaming faster and honest fixes rarer. 120 trajectories, 2,400 revisions, every prompt and gate decision archived.

    DataZenodoAll 120 trajectories with SHA-256 manifest

    Codegithub.com/nickjlamb/rsi-loop · preregistration

    SummaryLay summary and infographic

    AI Safety Goodhart's Law LLM Agents
  • Aug 2026 Zenodo Preprint In peer review

    Observer Zero: Do LLM Agents Form Epistemic Communities?

    doi:10.5281/zenodo.21906653

    The combined report of the programme's two studies, with a cross-study synthesis over 235 seeded, manifest-frozen runs – the second study pre-registered and frozen before any confirmatory data were seen. A change-point detector fed only the measurements the agents themselves chose to take finds the covert law change in 42 of 42 runs of the second study; one agent of Study 2's 276 concluded a physical law had changed, and none of Study 1's 80. Homogeneous grounded societies produced zero voluntary communications across 7,680 agent-days, and adding one communicative agent produced a star rather than a cascade – eighteen of its twenty unsupported claims were incorporated into grounded agents' beliefs, none challenged. A talking society was not a better epistemic system than a silent one; it was a silent one plus a channel.

    ModelCoMSESComputational Model Library, peer review requested

    DataZenodoStudy 2 raw run data

    Codegithub.com/nickjlamb/observer-zero

    AI Agents Social Epistemics Multi-Agent Systems
  • Aug 2026 Zenodo Preprint

    Observer Zero: Autonomous LLM Scientists Detect Changes to Their World but Fail to Conclude That It Changed

    doi:10.5281/zenodo.21872780

    Two-agent teams of LLM scientists run experiments, exchange letters, and maintain probabilistic hypotheses inside Meridian, a world with fictional physics whose constants the simulator alters mid-run without telling them. Detection is near-perfect at 90–100%, yet across 40 gravity-shift runs and roughly 150 final belief states not one agent concluded the law itself had changed – the explanations reach for faulty instruments and interfering colleagues instead. Removing the mundanity-preference instruction widened what agents would entertain without fixing the conclusion, and produced a law-change verdict in a control world where nothing had changed at all. Model family, not scale, governed evidence fabrication and willingness to collaborate.

    ArticleAI AdvancesWhat 150 Simulated Universes Revealed About How AI Agents Do Science

    Codegithub.com/nickjlamb/observer-zero

    AI Agents Scientific Reasoning Multi-Agent Systems
  • Jul 2026 Zenodo Preprint

    Recreating Historical Chess Style with Policy-Only Fine-Tuning and Quantitative Validation

    doi:10.5281/zenodo.21564620

    Five era-specific chess models — 1840s Romantics to 2010s engine-era grinders — produced by one epoch of fine-tuning Maia-2 on dated historical corpora, served without search, and validated three ways: self-play reproduces historical opening, draw-rate, and game-length gradients; used as a generative classifier the models identify a game's era at 42% single-game (20% chance) and 88% over twenty games; and strength measured against a calibrated engine ladder lands all five within one rating class, confirming eras differ in style, not strength. A case study in the same discipline that runs through this page: no claim without a measurement.

    ArticleTowards AIRecreating 180 Years of Chess Style with Policy-Only Fine-Tuning

    Playchess.pharmatools.ai

    Fine-Tuning Imitation Learning Model Validation
  • Jul 2026 Zenodo Preprint

    Evaluation as Infrastructure: Deterministic Regression Gating for Evidence-Grounded AI Systems in Production

    doi:10.5281/zenodo.21365095

    An experience report applying OpenGATE to four production systems, including an MHRA-registered medical device. Three of the four yielded real defects on their first run: a silent parse failure affecting roughly half of multi-claim verdicts, two name-capture failures in a de-identification engine, and a simplifier that dropped an antibiotic dose from a discharge summary. A controlled comparison against RAGAS and DeepEval finds the two frameworks disagree about what their shared “faithfulness” metric means — on outputs asserting figures absent from the source, one detects 5/6 and the other 0/6.

    AI Evaluation LLM-as-Judge Regression Testing
  • Jun 2026 PharmaTools.AI Whitepaper

    Designing Trustworthy AI: Why High-Stakes Systems Are Converging on Verification

    A working paper arguing that trustworthy AI in high-stakes domains is an architectural problem as much as a model-capability one. It names the generation → verification shift, treats fabricated citations and confabulated rationales as structurally predictable failures of generative architectures, and uses healthcare as the stress test for what verification-first design requires.

    Verification AI Healthcare AI Hallucinations
    First page of Designing Trustworthy AI: Why High-Stakes Systems Are Converging on Verification
  • Nov 2025 ESMO Real World Data Digit. Oncol. Conference Published abstract

    PosterLens: AI-Powered Mobile App for Decoding Scientific Posters

    doi:10.1016/j.esmorw.2025.100562

    ePoster 366eP at ESMO Congress 2025; abstract published in ESMO Real World Data and Digital Oncology 10 (Suppl):100562. Ten recent oncology congress posters were processed with PosterLens (OCR plus a large language model) and the summaries scored against the originals on objective, population, primary endpoint and conclusion. Text length fell by 78% (range 68–89%) at a median fidelity of 4/4; objective, population and conclusion were preserved in 90% of posters and the primary endpoint in 80%, and time to locate the key finding fell from 23 to 11 seconds.

    Mobile AI Clinical Trials Conference Presentation
    PosterLens ePoster 366eP presented at ESMO Congress 2025
  • Jan 2025 JNGR 5.0 Journal

    A Day in the Life of an MSL Powered by AI

    A concept paper for an integrated MSL training platform, told through one working day: generative AI with retrieval-augmented generation for personalised content, multimodal AI for interactive mechanism-of-action visualisation, and explainable AI for transparent recommendations. Framed as a design proposal rather than an implementation, with extensions to field sales and compliance training.

    MSL Training RAG AI Agents
    First page of A Day in the Life of an MSL Powered by AI: Combining AI Technologies to Transform Training in JNGR 5.0

Articles & essays

Technical writing for practitioners in AI Advances, Towards AI, Generative AI and Pharmaphorum.

  • May 2026 AI Advances Article

    Your RAG Passes Every Retrieval Metric — And Still Lies to You

    Recall@k confirms the right document was retrieved but says nothing about whether the answer is faithful to it. Introducing LitRAG, a ~200-line open-source pipeline with a two-stage citation check — deterministic quote localization plus LLM-as-judge support grading — that catches fabricated quotes and unsupported claims retrieval metrics are blind to.

    RAG Citation Faithfulness AI Evaluation
  • May 2026 Towards AI Article

    I Built an RSI for My RSI

    A MediaPipe pose detector (~1,150 lines of MIT-licensed Python) that flags high-strain posture from head and wrist angles, wrapped in a recursive self-improvement loop — and the two-stage validation gate that keeps each self-proposed change accurate, clinically plausible and free of reward-hacking shortcuts, drawing on four decades of pharmaceutical validation practice.

    Recursive Self-Improvement AI Safety Reward Hacking
  • Dec 2025 AI Advances Article

    What Happens When AI Runs Out of Data?

    Argues that the “data wall” is a property of text, not of learning: the web records what people write, not what they know, and experiential data from robotics and simulation dwarfs it — Waymo alone drives roughly 10 million simulated miles a day. Text corpora are the randomised trial; embodied learning is the real-world evidence; model collapse is what happens when synthetic data loses its grounding.

    AI Training Machine Learning Experiential Learning
  • Aug 2025 Medium Article

    When AI Can't Forget: The 7 Hidden Risks of LLMs in Healthcare

    Seven ways a healthcare LLM can leak or corrupt what it was trained on — model inversion, membership inference, prompt injection, accidental memorisation, data linkage, memory injection and poisoned training data — each with a clinical example and the matching control, from de-identification and differential privacy to output monitoring and red-team testing.

    Healthcare AI Data Privacy LLM Security
  • Jul 2025 Generative AI Article

    How I Won a Major Healthcare Communications Award… With just my laptop

    How Patiently AI won the 2025 Communiqué Award for Progress in Healthcare and Scientific Communications against agency-backed pharma campaigns: a six-month solo build on a £157-a-month tool budget, from MVP through user testing to submission, with 60% of testers reporting more confidence understanding their notes and 30% using the non-English explanations.

    Award Winner Healthcare Comms AI in Healthcare
  • Jul 2025 Generative AI Article

    Why LLMs Love JSON

    Separate the data from the instruction: put claim, study reference, audience, tone and jurisdiction in a JSON context object and keep one short, reusable prompt, so a single source of truth drives clinical summaries, patient education, regulatory text and MSL training without rewriting the prompt each time. Six worked examples across pharma, SaaS, e-commerce, education, legal and customer support, a three-step implementation plan, and the JSON-as-input versus JSON-as-output distinction.

    LLMs Structured Output AI Engineering
  • May 2025 AI Advances Article

    How to Break a Large Language Model

    Why LLMs fail on multi-step expert reasoning: each step is a statistical prediction, so errors compound, and a single irrelevant clause can cut accuracy by up to 65% (Mirzadeh et al., 2024). Shows how to build adversarial test prompts by harvesting a model’s earlier mistakes as distractors, worked through a pharmacokinetics dosing problem with unit conversions and a missed dose.

    AI Safety LLM Reasoning AI Evaluation
  • Apr 2025 Pharmaphorum Article

    How to Fix Pharma's Content Duplication Problem

    Medical affairs and commercial teams rebuild the same content in parallel because they sit in separate regulatory silos. Proposes a single master repository with AI-driven adaptation per audience, automated compliance guardrails and citation tracking, and a staged rollout that starts with lower-risk disease-awareness material before touching regulated content.

    Content Strategy AI Workflows Medical Affairs
  • Feb 2025 Medium Article

    From Generative AI to Agentic AI: Pharma's Next Revolution

    Draws the line between generative AI, which produces content on request, and agentic AI, which plans, acts in underlying systems and adapts without a prompt — and maps where that shift lands in pharma: drug development, patient support and regulatory compliance.

    Agentic AI Pharma Innovation Autonomous Systems
  • Jan 2025 AI Advances Article

    How I Built Smarter AI Tools for Pharma: From Generative AI to Agentic AI

    A builder’s account of moving PharmaTools.AI from generative to agentic tools, starting with a RAG-based compliance assistant that retrieves the relevant code and precedent while reviewing promotional claims, and the three problems it targets: overloaded compliance teams, medical writers translating for patients, and patients navigating their own care.

    AI Development Agentic AI Technical

Technical notes

Short technical write-ups: bugs found, fixes verified and open-source contributions, with links to the evidence.

Building AI for pharma, compliance, or patient communication?

Get in touch →