The instrument that reads everything

The most useful way to think about AI in science is as a new kind of instrument. A telescope does not have ideas; it lets you see things you could not see, and the ideas follow. Learned models do something similar for problems where a mapping exists in nature but is too expensive to compute directly.

The three cases where it clearly worked

Protein structure. Sequence goes in, three-dimensional shape comes out. This had been an open problem for fifty years, and the solution turned a task that took a doctoral project into one that takes minutes. The prerequisites were a large curated database of experimentally determined structures and a problem with a single correct answer. The 2024 chemistry Nobel recognised it.

Weather. Numerical weather prediction solves fluid equations on a grid, which takes a supercomputer hours. Models trained on decades of reanalysis data now produce forecasts of comparable or better skill in seconds. They do not understand the atmosphere; they learned the mapping from one state to the next, and for forecasting that is enough.

Materials and chemistry screening. Predicting properties of candidate compounds fast enough to filter millions of them before anyone synthesises anything. The value is not in the prediction being perfect; it is in narrowing an intractable search to a testable shortlist.

The common shape of every success

Look at all three and the same conditions appear. A large, consistent, machine-readable dataset built by decades of unglamorous experimental work. A well-posed question with a checkable answer. And a problem where a good approximation is genuinely useful, because the alternative is an expensive simulation or no answer at all.

Where any of those is missing, the results are much weaker. This is why AI has transformed structural biology and has not transformed, say, sociology.

The replication problem it created

Machine learning has spread into fields whose researchers have no training in its failure modes, and the consequences are visible. Data leakage between training and test sets produces spectacular results that vanish on new data. Models trained on one hospital's scanners learn the scanner rather than the disease. Published accuracy figures on benchmarks that were themselves assembled with a bias flatter every method applied to them.

Several systematic reviews have found that a large share of published machine learning results in some applied fields do not hold up. That is a methods problem, not a technology problem, and it is fixable with pre-registration, held-out data from different sources and code release. It is also slow to fix, because the incentives point the other way.

What AI does not do

It does not choose the question. It does not design the experiment that produces data nobody has. It does not know when the world has changed in a way that invalidates the training distribution, which in science is often the interesting moment. And it does not provide the mechanism: a model that predicts a material is stable does not tell you why, and why is what lets a chemist generalise.

Automated laboratories that run the physical experiments and feed results back are the serious attempt at closing this loop. They work in narrow, well-instrumented domains and are a long way from a general capability.

What to watch

Watch for a drug developed from an AI-predicted structure completing late-stage trials, which would be the first end-to-end validation. Watch for weather models being adopted by national services rather than published by labs. And watch how many fields adopt held-out external validation as a publication requirement, because that will decide how much of this literature survives.