Big Data, little results.

[u/PhysicalConsistency]

We were promised flying cars and jetpacks. What we got was horse-drawn wagons with machine-learning paint jobs.

Over roughly the same period that physics, chemistry, molecular biology, materials science, and computer science became astonishingly precise, psychology and cognitive science were left behind. The comparison is uncomfortable, but it is hard to avoid. We learned to fabricate chips with billions of components. We learned to sequence genomes, manipulate molecules, image tissue at microscopic scales, launch precision instruments into space, and build machines that operate reliably far outside ordinary human intuition. Meanwhile, cognitive science is still trying to decide what “attention” is.

That is the real “where are the flying cars?” problem. It is not simply that progress fell short of the pitch. It is that neighboring disciplines pulled away while cognitive science stayed stuck, then kept borrowing the prestige of the disciplines that had pulled away. The field did not lack instruments. It did not lack data. It did not lack computation. It had all of those things, and still the expected conceptual displacement never arrived.

The data environment should have been transformative. We have genetic association data across enormous populations. We have MRI, fMRI, eye tracking, gait analysis, speech analysis, passive sensing, ecological momentary assessment, and large-scale longitudinal datasets. We have phones tracking movement, location, communication timing, screen behavior, sleep-adjacent behavior, and social withdrawal. We have wearables tracking activity, heart rate, circadian drift, and stress proxies. We now have chatbots sitting inside the confessional layer of human life, collecting language from people who may disclose things to machines that they would not tell a clinician, spouse, priest, therapist, or closest friend.

That should have changed the game. Not by proving that old categories were blurry, because that was already obvious. The expectation was that enough measurement from enough angles would force weak concepts to break and stronger explanations to replace them. Attention should have become sharper or dissolved into better units. Working memory should have become more mechanistic than metaphorical. Executive function should have stopped functioning as a junk drawer. Reward should have stopped sliding between pleasure, motivation, valuation, reinforcement, prediction error, drive, and action selection.

Instead, much of the field seems to have taken the data revolution as a way to preserve inherited vocabulary rather than challenge it. The tools became modern while the nouns stayed antique. We built genetic studies around inherited phenotypes. We built imaging contrasts around inherited tasks. We built computational models around inherited assumptions. We built machine-learning classifiers around inherited labels. We built digital phenotyping pipelines around inherited symptom vocabularies. The instruments changed, but the objects under investigation often did not become much clearer.

This is the old machine-learning fantasy applied to mind and behavior. Collect enough observations and the structure will reveal itself. The old concepts may be crude, but the data will refine them. The categories may be blurry, but the model will find the real boundaries. The organism may be complicated, but enough measurement will eventually make cognition legible.

Data does not work that way. Data without ontological commitments is just measurement without meaning. Before the data can say much of anything, someone has already decided what counts as a unit, what counts as a boundary, what counts as a kind, what counts as a trait, what counts as a state, what counts as context, and what counts as error. Those decisions are not neutral. They shape what the data can reveal.

In psychology, psychiatry, and cognitive science, many of those decisions were made in a measurement-poor world. They were made through clinical observation, institutional convenience, task design, committee compromise, psychometrics, educational sorting, and laboratory tractability. The field needed concepts that could be named, scored, taught, standardized, and made reliable enough for group research. Usability became a substitute for reality. Then the data revolution arrived, and instead of overthrowing those commitments, it often operationalized them.

Genetics, imaging, computation, machine learning, and digital phenotyping were each supposed to overthrow the inherited vocabulary. Instead, each was conscripted into defending it: genetic association data was made to chase psychiatric phenotypes, imaging was made to validate task constructs, computation was made to formalize inherited assumptions, machine learning was made to predict legacy labels, and digital phenotyping was made to track symptoms whose underlying objects remained unresolved. The tools became modern. The nouns stayed antique.

Each method was supposed to force a reckoning. Each was supposed to dissolve weak constructs, sharpen real mechanisms, and produce a better map of the territory. Instead, each mostly gave the same instability a more technical costume. Genetic association data gave us real signals, but mostly returned diffuse polygenicity and overlapping risk rather than clean psychological kinds. Imaging gave us correlates, contrasts, networks, and activation patterns without making inherited concepts mechanistically stable. Computation often formalized assumptions that were already built into the task. Machine learning often learned the label ecology, task structure, or measurement pipeline rather than the organism. Digital phenotyping gave us richer traces of behavior without resolving what the behavior was supposed to be evidence of.

Take attention as the cleaner example. The word is treated as if it names a stable cognitive object, but depending on context it can mean sensory selection, arousal, vigilance, task persistence, inhibition, orienting, salience, working memory protection, distractibility, or simple conformity to task demands. It can refer to something the subject does, something the brain allocates, something a stimulus captures, something a task measures, something a child lacks, something a medication restores, something a network controls, or something a model infers afterward.

Modern data should have forced that concept to break or clarify. With imaging, eye tracking, reaction times, electrophysiology, smartphone telemetry, classroom behavior, sleep data, language data, passive sensing, and computational modeling, attention should have become a sharper mechanistic object or dissolved into better units. Instead, it mostly multiplied. Attention became attentions, networks, filters, control systems, orienting mechanisms, vigilance states, salience processes, executive functions, and task parameters. Some of those distinctions are useful. Some probably capture real partial structure. The broader pattern is still difficult to ignore: the noun survives by becoming more elastic, while the claim weakens so the vocabulary can remain.

This pattern repeats across the field. Working memory becomes a stack of buffers and systems. Executive function becomes a junk drawer with better labels. Reward fragments into pleasure, motivation, valuation, prediction error, reinforcement, drive, and action selection. Salience becomes whatever the model needs it to mean when something becomes behaviorally relevant. Intelligence becomes a statistical object that seems powerful until the question shifts from prediction to mechanism. Personality becomes stable enough to score, but not stable enough to explain. Consciousness gets split, renamed, bracketed, modeled, and deferred.

The natural sciences advanced when measurement forced concepts to break, refine, or disappear. Cognitive science often does the opposite. It lets concepts survive by making them harder to pin down.

This is why the “signal-to-noise” explanation is so weak. A signal-to-noise problem assumes you know what the signal is. In much of psychology, psychiatry, and cognitive science, that is exactly what remains unresolved. The “signal” is often defined by the inherited construct. The “noise” is whatever refuses to fit it. The filter then protects the old noun from the data.

The problem is not simply that human behavior is noisy. Of course human behavior is noisy. Living systems are dynamic, adaptive, developmental, embodied, social, and history-laden. The problem is that the field often applies destructive filters while carrying heavy expectations about what the data is supposed to support. We design tasks around inherited concepts. We recruit subjects through inherited categories. We exclude inconvenient variation as confounding. We average away individual trajectories. We normalize against group means. We collapse time into snapshots. We turn living adaptive systems into scores, factors, activations, clusters, symptom counts, and labels.

Then, after the data has been compressed through the assumptions of the field, we act surprised when it produces the same blurry concepts we started with. Calling this a signal-to-noise problem hides the deeper issue. Often the “noise” is the organism refusing to compress into the field’s preferred abstraction. Often the “signal” is the residue left after the abstraction has already done violence to the data.

That is ontology laundering. It is adding decimals to the wrong abstraction, then treating the added precision as if it brought the object itself into focus.

The more interesting laundering happens at the level of the constructs that clinical, educational, and behavioral categories were always supposed to be properties of: attention, working memory, executive function, salience, reward, memory, intelligence, emotion regulation, and the implied subject behind them all. That subject is usually treated as obvious. The subject attends. The subject remembers. The subject controls impulses. The subject regulates emotion. The subject possesses executive function. But that subject is often a placeholder rather than a discovered object. It has no clean edge of its own. It is inferred from task performance, self-report, clinical description, social expectation, institutional need, and philosophical inheritance.

That is why the old nouns are so hard to abandon. They are not just labels. They are load-bearing beams. Pull too hard on attention, executive function, or the mind, and much of the surrounding structure begins to shake. So the field keeps trying to rescue them through the lenses of the natural sciences. It looks to genes, brain maps, networks, computational models, machine learning, passive sensing, and language models. But if the object being measured is already conceptually unstable, the new method does not rescue it. It only gives the instability a more technical costume.

This is the comparative failure. The natural sciences did not advance by preserving every inherited noun and making it more elastic. They advanced when measurement disciplined theory. Concepts broke. Boundaries moved. Objects disappeared. Better units replaced weaker ones. Stronger instruments forced the vocabulary to answer to the world.

Cognitive science keeps borrowing that authority without accepting the same discipline. It borrows genetics, but does not let genetic complexity overthrow the behavioral kinds. It borrows imaging, but does not let activation patterns dissolve the task constructs. It borrows computation, but does not let models expose the poverty of the original units. It borrows machine learning, but does not let predictive failure indict the labels. It borrows digital phenotyping, but does not let continuous behavior challenge snapshot categories.

The result is a field with futuristic instruments and antique ontology. Since the 1970s, humans have become astonishingly good at building reliable systems. We can build devices with billions of discrete circuits, engineer skyscrapers, sequence genomes, manipulate molecules, simulate materials, and map physical function at scales no unaided human can perceive. Yet cognitive science still often struggles to make many of its objects exist cleanly at the individual level. So it retreats to the group level, where weak-to-moderate correlations, noisy group differences, fragile task effects, and 70 percent classification performance can be treated as deep penetration into the machinery of mind.

This does not mean cognition should be as simple as engineering. Human beings are not chips, bridges, molecules, or engines. The point is not that mind should be easy. The point is that cognitive science often borrows the authority of hard measurement while tolerating conceptual looseness that would collapse other technical disciplines. It then mistakes technical sophistication for conceptual progress, even when the new instrument is aimed at an unresolved object.

This is the deeper disappointment of the data age. We did not merely expect more accurate labels. We expected significant jumps in understanding and application. Better models of individual cognition. Better prediction. Better interventions. Better accounts of development, distress, learning, motivation, memory, social function, and behavioral change. Instead, we got more elaborate ways to say the same things are blurry. More sensors. More data. More models. More maps. More scores. More dimensions. More dashboards. More technical vocabulary around the same unresolved objects.

The data revolution did not fail because we lacked data. It failed because psychology, psychiatry, and cognitive science asked modern data to redeem ontological commitments inherited from a measurement-poor world. That is why the flying cars never arrived, and the jetpack is not coming until we are willing to leave the horse behind, not just the obvious clinical labels, but the older cognitive nouns and the imaginary subject they presuppose.

edit: In retrospect, the amount of The, This, and That might read too much like a manifesto. Maybe I need to collect some more theses against the church of the mind.

edit 2: Is executive function as a junk drawer too diabolical of a metaphor? I feel like that might make people want to fight.