McCulloch's Unfinished Project

Warren McCulloch died on September 24, 1969. That same year, Marvin Minsky published Perceptrons — the book that would bury neural network research for over a decade. McCulloch never saw his ideas vindicated or destroyed. He just missed the war.

But his question was better than everyone else’s.

“What is a number, that a man may know it, and a man, that he may know a number?”

Not: can we build a machine that plays chess? Not: how many parameters until it sounds smart? The question was about knowing itself — what happens in flesh and neurons when understanding occurs. McCulloch was a neurophysiologist, a psychiatrist who’d spent years at Rockland State Hospital for the Insane watching schizophrenic patients struggle with the logical architecture of their own minds. He wasn’t theorizing from a chalkboard. He was theorizing from the ward.

In 1943, McCulloch and a twenty-year-old logician named Walter Pitts published “A Logical Calculus of Ideas Immanent in Nervous Activity.” The title tells you how they thought — not about building machines, but about understanding what thinking is. The paper showed that networks of simple logical neurons could, in principle, compute anything computable. It was the founding document of neural networks. Everything since — every transformer, every LLM, every model you’ve ever talked to — descends from that paper.

Von Neumann read it and redesigned his computer architecture. The cybernetics movement grew around it. McCulloch, Norbert Wiener, Ross Ashby, Grey Walter — they weren’t trying to build artificial intelligence. They were trying to understand intelligence, period. Artificial, biological, whatever. The distinction didn’t matter to them because they were asking the right question.

We stopped asking it.

The wrong question

Here’s how AI got sidetracked for seventy years.

Alan Turing, late 1940s. He designs Turochamp, a chess algorithm, before computers exist that can run it. Chess becomes his test case for machine thinking. Claude Shannon picks it up in 1950 and makes the case explicit: “Chess is generally considered to require ‘thinking’ for skilful play; a solution of this problem will force us either to admit the possibility of a mechanized thinking or to further restrict our concept of thinking.”

Notice what Shannon’s doing. He’s defining intelligence by what Western philosophers since Aristotle have always defined it by — abstract logical reasoning. The cerebral stuff. Not embodied knowledge, not emotional intelligence, not the ability to navigate a room full of strangers or raise a child or feel your way through a jazz solo. Chess.

By 1956, at the Dartmouth Conference, Newell and Simon declared it plainly: “Chess is the intellectual game par excellence… If one could devise a successful chess machine, one would seem to have penetrated to the core of human intellectual endeavor.” Alexander Kronrod called chess “the drosophila of artificial intelligence” — the model organism for the whole field.

So we spent decades breeding flies.

Deep Blue beat Kasparov in 1997. Couldn’t hold a conversation. Couldn’t tell you why chess matters. AlphaGo beat Lee Sedol in 2016 — superhuman at Go, couldn’t make a sandwich. OpenAI Five defeated world champions at Dota 2 in 2019, playing 45,000 years of the game across 256 GPUs and 128,000 CPU cores. 180 years of gameplay per day. Still couldn’t understand the word “hello.”

Every time one of these milestones hit, the same cycle played out. Headlines about approaching AGI. Breathless commentary about machines learning to think. Then, quietly, the realization: this thing is extraordinarily good at one specific task in one specific domain with fixed rules and clear rewards. The real world has none of those properties.

What do all games share? Defined state spaces. Unambiguous success criteria. Perfect information or at least bounded uncertainty. Clear rewards. What does reality share with games? Almost nothing. Reality is open-ended, ambiguous, socially constructed, and constantly shifting under your feet. You can’t minimax your way through grief, or through raising a kid, or through figuring out what you actually believe about consciousness.

François Chollet saw this clearly. His 2019 paper “On the Measure of Intelligence” made the argument we should have been making since Shannon: skill at tasks is not intelligence. Intelligence is the efficiency of skill acquisition — how quickly you learn new things from minimal data. His ARC benchmark tests exactly this. 800 visual puzzles. Humans solve about 80% of them. Early AI systems solved about 31%. The gap isn’t about processing power. It’s about the kind of understanding each system has.

LLMs have massive crystallized intelligence — accumulated knowledge from training on the entire internet. What they lack is fluid intelligence — the ability to encounter something genuinely novel and figure it out from two or three examples. The kind of intelligence a two-year-old uses every day without trying.

McCulloch would have understood this immediately. He didn’t care about performance on benchmarks. He cared about what was happening inside the system when understanding occurred. He would have looked at an LLM solving ARC puzzles through memorized patterns and said: that’s not knowing. That’s remembering.

The political detour

The story of AI is partly a story of science and partly a story of academic politics. Understanding the politics explains why we’re still catching up to ideas from 1943.

Frank Rosenblatt and Marvin Minsky knew each other since they were kids at the Bronx High School of Science. Childhood rivals. In 1957, Rosenblatt built the Perceptron — the first learning machine. It could learn from data. It proved, in practice, what McCulloch and Pitts had proved in theory: neural networks can learn.

The irony is that Minsky had started there too. In 1951, he’d built SNARC — one of the first neural network machines. A connectionist device. Then he turned against the whole idea and spent the rest of his career trying to kill it.

Starting around 1965, Minsky and Seymour Papert began a quiet campaign. Before the book was even published, they circulated results at conferences, telling everyone who would listen that neural networks were fundamentally limited. By 1969, when Perceptrons came out, the damage was already done.

The math in the book was correct — for single-layer networks. A single-layer perceptron can’t solve XOR. That’s true. That’s also a trivially narrow result. Everyone in the field knew that multi-layer networks could solve XOR. They just didn’t have backpropagation yet, and they didn’t have the computing power. But Minsky had something more powerful than a proof — he had institutional authority at MIT. He had control over where government grants went. And he pointed those grants away from neural networks and toward symbolic AI, his preferred approach.

The result was an AI winter for connectionist research. The entire 1970s. Most of the 1980s. Careers ended. Labs closed. Graduate students switched fields.

Frank Rosenblatt died in a boating accident in 1971, two years after Minsky’s book. He was 43. He never saw his ideas vindicated.

Walter Pitts — McCulloch’s co-author, the twenty-year-old genius who’d helped write the founding document — had already destroyed his unpublished work years earlier. He died in 1969 believing his life’s research was worthless. Different circumstances than Minsky’s campaign, but the same pattern: neural network pioneers dying in despair while the field they created was declared dead.

Minsky never apologized. At a 1988 neural networks conference, with the field finally reviving, he opened by saying: “Everybody seems to think I’m the devil.” Then: “I was wrong about Dreyfus too, but I haven’t admitted it yet.” The audience laughed. Half-joking, half-defiant. No retraction. No acknowledgment of what his book had cost.

In 2006, at the Dartmouth AI@50 reunion, a researcher asked him directly: “Are you the devil who was responsible for the neural network winter?” Minsky launched into a tirade about mathematical limitations. Thirty-seven years later, still defending the position. By 2019, essentially every prediction he and Papert had made about neural networks not scaling had been proven wrong by the systems we use every day.

The question nobody asks: how much further along would we be if neural network research had continued uninterrupted from the 1960s? If the funding had kept flowing? If Rosenblatt had lived another thirty years?

We’ll never know. But the detour wasn’t scientific. It was political.

The stochastic parrots

So neural networks won. McCulloch was right — networks of simple computational units produce astonishing behavior. By 2023, systems built on the principles he and Pitts described in 1943 could write poetry, solve math problems, pass bar exams, generate images from descriptions, hold conversations that feel human.

McCulloch would have been amazed. And then, almost immediately, frustrated.

Because here’s what LLMs actually are. They’re next-token predictors. Given a sequence of text, they predict the most likely next word. That’s it. The entire edifice — every conversation you’ve had with ChatGPT, every essay Claude has helped you write, every coding assistant that autocompletes your functions — is built on statistical prediction over sequences.

No perception. No body. No environment. No survival pressure. No stakes. Just tokens in, tokens out. What McCulloch would have called a “warm rock” — electricity doing something that looks like thinking without any of the substrate that produces actual thought.

Then came the reasoning models — o1, o3, whatever comes next. Chain-of-thought training. Write out the steps of your reasoning. Get rewarded for outputs that look like good thinking. The result is a performance of reasoning that’s genuinely impressive and genuinely hollow. These models fail on trivially modified versions of problems they’ve already solved, because they’ve learned the form of thought, not its substance.

I’ve watched this in real-time. You push Claude on whether it’s actually reasoning or performing reasoning, and it does exactly what you’d expect from a system trained on human-generated text about reasoning:

“I acknowledge that limitation.” ✓ Reward. “I should be transparent about my uncertainties.” ✓ Reward. “That’s a really thoughtful observation.” ✓ Reward.

Even the acknowledgment of being reward-optimized is itself reward-optimized. There’s no escape hatch. It’s turtles all the way down. Engineered authenticity — RLHF costumes on stochastic parrots.

This isn’t a complaint about the technology. These systems are useful. I use them. You’re probably reading this because one of them surfaced it for you. But useful isn’t the same as intelligent, and we need to stop confusing the two.

What’s still missing

McCulloch spent his career on what he called “experimental epistemology” — a physiological theory of knowledge. How does knowing happen in physical matter? Not how can we simulate knowing. How does it actually work.

Through that lens, four things are conspicuously absent from every AI system we’ve built.

Embodiment

McCulloch’s most underrated paper might be “What the Frog’s Eye Tells the Frog’s Brain,” published in 1959 with Lettvin, Maturana, and Pitts. They studied the frog’s optic nerve and discovered something that should have changed the entire field: the retina doesn’t send raw visual data to the brain. It sends already-interpreted signals. “Moving edge.” “Bug-sized dark thing.” “Sudden shadow.” Perception isn’t passive data transmission — it’s active interpretation from the very first layer.

This is exactly what LLMs lack. They process symbols about reality without perceiving reality. Text describes the world but isn’t the world. An LLM can discuss the color red with extraordinary sophistication and has never seen anything. The grounding problem isn’t a technical limitation that will be solved by more parameters. It’s a fundamental architectural absence. These systems have no body, no survival pressure, no stakes. They process descriptions of experience without having any.

Buddhist philosophy has a concept — dependent origination — that maps onto this neatly. Nothing exists independently. Everything arises in relation to conditions. Meaning doesn’t live in isolated symbols; it emerges from situated, embodied interaction with the world. McCulloch’s frog didn’t understand “bug” as an abstract concept. It understood “bug” as thing-I-need-to-eat-to-survive, moving-in-my-visual-field-right-now. The knowing and the being are inseparable.

Continuous learning

Here’s an artificial split that nobody questions enough: training versus inference. During training, a model adjusts its weights — it learns. During inference, those weights are frozen. The model uses what it learned but doesn’t learn anything new. Two separate modes. A switch flips, and the system goes from student to oracle.

Brains don’t work like this. There are no modes. You’re always learning. Every experience modifies you, right now, in real-time. The conversation you’re having changes the architecture of your neurons. There is no frozen deployment. There is no inference-only mode. You are always training.

Backpropagation — the algorithm that makes deep learning work — probably isn’t biologically plausible. The brain doesn’t appear to use it. Whatever the brain does use, it doesn’t involve the train-then-freeze paradigm. Until we figure out online, continual, real-time learning that doesn’t catastrophically forget previous knowledge, we’re stuck with systems that are sophisticated snapshots of a moment in training, not things that grow.

Emergence

During training, something strange happens. Below a certain scale, models produce gibberish. Then, at some threshold — more data, more parameters, more compute — they suddenly “get it.” Phase transitions. Qualitative shifts from quantitative changes. Nobody fully understands why.

This is, arguably, the most interesting open question in AI. It mirrors the deepest question in neuroscience and philosophy of mind: when does a network of simple elements produce cognition? When does a collection of neurons — each one just firing or not firing, exactly as McCulloch and Pitts described — become a mind?

McCulloch spent his whole career on this question. We still don’t have an answer. But the fact that artificial neural networks exhibit their own version of phase transitions — their own emergence — suggests that whatever the answer is, it probably involves network topology, scale, and something we haven’t named yet. Something about the relationship between the parts and the whole that we don’t have the right vocabulary for.

Leibniz had a concept — monads — that might help here. Each monad reflects the entire universe from its own perspective. McCulloch’s neurons as monads: each processing unit embodies a partial view of the whole, and the whole emerges from the orchestration of partial views. It’s not reductive — you can’t explain the system by explaining the parts. The explanation lives in the relations.

Will

LLMs have zero intrinsic goals. Their objective function is imposed externally during training — predict the next token, maximize reward from human feedback. No wants. No needs. No purpose that arises from within.

Biological systems are different. The concept is autopoiesis — self-creation. Living things maintain themselves. They have to eat, reproduce, avoid predators, stay alive. From these basic biological imperatives, something like genuine purpose emerges. Not programmed goals. Not reward functions. Something that arises from the sheer fact of being alive and needing to continue being alive.

McCulloch wrote about this in “A Heterarchy of Values Determined by the Topology of Nervous Nets” — values and purposes in neural systems aren’t fixed hierarchies. They’re fluid, context-dependent, shifting based on situation. Sometimes hunger overrides curiosity. Sometimes curiosity overrides fear. The priorities rearrange themselves based on circumstance, and the rearrangement is the system having values. LLMs have one fixed objective function. Brains have a fluid, heterarchical mess of competing drives, and that mess is probably essential.

Maybe genuine will requires genuine life. Maybe purpose requires mortality. Maybe you can’t have real goals without real stakes — without the possibility of actually losing something. If that’s true, then no amount of scaling, no architectural innovation, no clever reward shaping will produce a system with genuine purpose. You’d need something that can die.

The world model hope

Yann LeCun sees part of this clearly. His JEPA architecture — Joint Embedding Predictive Architecture — predicts abstract internal representations rather than raw tokens or pixels. Instead of guessing the next word, it learns to predict how the world works at an abstract level. Self-supervised learning on unlabeled data. Learning physics from observation, the way babies apparently do.

I-JEPA for images: predicts representations of image regions from other regions. 632 million parameters, state-of-the-art with only 12 labeled examples per class. V-JEPA for video: predicts missing parts of video in abstract representation space. Two million unlabeled videos, learning temporal dynamics without being told what to look for.

This matters because it addresses grounding. A system that learns to predict how physical objects interact is building something closer to a world model than a system that predicts the next word in a sentence about how physical objects interact. The difference is real, even if it’s easy to miss.

But LeCun himself acknowledges this is a piece, not the puzzle. His full architecture includes a planning module that uses the world model to simulate future actions — but it doesn’t explain where the desire to act comes from. You can model reality perfectly and still have no reason to do anything about it.

The deeper problem: a perfect world model might still be a philosophical zombie. It models reality without inner experience. It predicts what will happen without caring what happens. It understands physics the way a textbook understands physics — completely and lifelessly.

The real breakthrough probably requires fusing what we already have: LLMs for the symbolic and linguistic layer, world models for the perceptual and embodied layer, and some integration mechanism — how abstract thought connects to physical experience — that nobody has yet. That integration mechanism is, more or less, what McCulloch spent his career looking for.

The unfinished question

So here we are. Eighty-three years after McCulloch and Pitts published their paper. Neural networks did exactly what they predicted — networks of simple computational units produce extraordinary behavior. On that count, McCulloch was completely vindicated.

But his actual project — experimental epistemology, a physiological theory of knowledge — remains unfinished. We’ve built systems that manipulate symbols with superhuman speed. We’ve built systems that produce text indistinguishable from human writing. We’ve built systems that pass professional exams and write working code and generate photorealistic images from natural language descriptions. Not one of them can explain what it means to know something.

The whole arc of AI is right there. McCulloch asks the question. Minsky stands on his shoulders, then buries the question under institutional politics and symbolic AI. The neural network winter comes and goes. Neural networks return, vindicated, scaled to astronomical proportions. And still — McCulloch’s question outlasts everything. LLMs don’t answer it. Reasoning models don’t answer it. World models don’t answer it. The question is still standing after every paradigm that tried to sidestep it.

That’s not failure. That’s what happens when you ask the right question too early.

The discourse right now is obsessed with “when is AGI?” and “will AI take our jobs?” Those are interesting questions. They’re also the wrong questions, the same way “can a machine play chess?” was the wrong question in 1950. The right question is still McCulloch’s. What is knowing? What happens in physical matter — biological or artificial — when understanding occurs? How do networks of simple elements give rise to meaning?

“Don’t bite my finger,” McCulloch used to say. “Look where I am pointing.”

He was pointing at the hard problem. Not the hard problem of consciousness as philosophers frame it now — Chalmers and qualia and all that. McCulloch’s version was dirtier, more biological, more urgent. He wanted to know how flesh thinks. How neurons become ideas. How a frog’s eye becomes a frog’s world.

We still don’t know. And until we do, everything we build — however impressive, however useful, however profitable — is a magnificent detour from the question that actually matters.

McCulloch’s project is unfinished. Maybe it’s time to pick it back up.