Why Does AI Keep Failing When It Meets Real Lab Data? with John F. Conway

About this episode

The reproducibility crisis in industrial science is estimated to waste thirty to fifty billion dollars a year, and John F. Conway says almost none of it is a technology problem. He is Chief Visioneer Officer at 20/15 Visioneers and has spent twenty-five years on scientific data — first building the models, then realising the world had a data problem so severe that the models could not save it. Jonathan Green talks with him about why smaller foundational models beat giant ones in the lab, why half of academic data quality leaks into commercial pharma, why he refuses to start any project without mapping the process first, and why "screen everything" turned out to be the wrong answer after fifteen years of trying. Key Takeaways: • The bottleneck is model-quality data, not model size. In science, smaller foundational models outperform the giants because you are not overfitting on a problem where barely any data exists in the world. Save the large language models for writing the paper about it. • Capture the failures, not just the wins. An experiment that failed is data — it stops the next person repeating it, and analysing why it failed is often where the value sits. • Unusable scient