Where the Bitter Lesson Fails
The limits of scale, and why the last mile of AI requires a different approach
Rich Sutton's "Bitter Lesson" is the most influential essay in modern AI. Its core claim: methods that scale with compute eventually beat methods that encode human knowledge. Stop being clever. Just scale.
The lesson has been vindicated repeatedly. Hand-crafted features lost to neural networks. Expert systems lost to transformers. Symbolic reasoning lost to scale. Every clever trick to "help" models perform better eventually becomes a liability as models get big enough to learn the solution themselves.
This lesson shapes how the entire industry thinks. Scale compute. Scale data. Scale parameters. The bitter lesson says this will work. And for general capabilities, it has.
But there's a domain where the bitter lesson fails. And it happens to be the domain where most of the economic value lies.
The 170 IQ Paradox
Imagine a 170 IQ genius—proficient in every field of human knowledge, capable of reasoning through any problem, with access to more information than any human could process.
Now drop this genius into Lazard Capital as a new employee. Can they perform effectively?
Absolutely not.
Despite genius-level intelligence and universal knowledge, they lack:
- Institutional knowledge and procedures
- Organizational workflows and tribal knowledge
- Understanding of "how we do things here"
- Relationship context with clients and colleagues
- Knowledge of recent decisions and their rationale
This isn't a capability problem. Our hypothetical genius could learn all of this eventually. It's a context problem. Without the right context, even unlimited intelligence produces wrong outputs.
This is exactly what's happening with AI deployment in enterprises. We have models that score 99.7% on public benchmarks. They can explain physics, write code, reason through complex problems. Yet they fail catastrophically on real company tasks.
We're measuring the wrong things.
Why Scale Doesn't Solve Context
The bitter lesson works when the task can be specified completely in the input.
"Translate this text to French." All the necessary context is in the prompt. More scale, more training data, better results.
"Write code to implement this algorithm." The algorithm specification contains everything needed. Scale works.
"Explain quantum entanglement." General knowledge domain. Scale definitely works.
Now consider: "Draft a response to this customer complaint using our standard escalation process, considering that this customer is in a strategic account with whom we have an upsell opportunity pending, and the VP of Sales has a personal relationship with their CTO."
Where is this context? Not in the prompt. Not in any training corpus. It exists in your Salesforce records, your Slack threads, your meeting transcripts, and critically, in the institutional knowledge that's never been written down anywhere.
Scaling compute doesn't help if the necessary context isn't accessible to the model.
This is the fundamental limit. The bitter lesson assumes the task can be learned from data. Enterprise tasks depend on context that isn't in any training data and can't be inferred from public information.
The Context Gap
Here's what's actually happening in enterprise AI deployments:
Demo environment: Model sees clean prompt, produces impressive output, evaluator says "wow, AI is amazing."
Production environment: Model sees prompt missing 90% of necessary context, produces plausible-but-wrong output, employee spends 30 minutes fixing it, concludes "AI isn't ready yet."
The gap isn't model capability. It's context availability.
McKinsey reports that 95% of AI demos fail in production. This isn't because production is "harder." It's because production requires context that wasn't present in the demo.
What Actually Wins
In enterprise AI, the winning strategy isn't scaling compute. It's:
1. Integration over intelligence
The model that's connected to your systems beats the smarter model that isn't. A 70B parameter model with access to your CRM, your email, your Slack, and your documents outperforms a 400B model working from just the prompt.
This is heresy according to the bitter lesson. Building integrations is "hand-engineering." The bitter lesson says wait for the model to get smart enough that integrations don't matter.
But integrations provide context. And context is the binding constraint, not capability.
2. Capture over retrieval
RAG (retrieval-augmented generation) is the industry's answer to the context problem. Embed your documents, retrieve relevant chunks, stuff them in the prompt.
But RAG only works if the context exists in retrievable form. The decision rationale from last month's sales call isn't in a document anywhere. The institutional knowledge about how to handle this type of customer isn't written down. The context that matters most is often the context that was never captured.
Capture matters more than retrieval. You can't retrieve what doesn't exist.
3. Expert feedback over training scale
The bitter lesson says more training data beats less. But for enterprise tasks, the marginal value of another billion tokens of general text is approximately zero. What matters is feedback from experts who understand your specific context.
One genuine expert correction in a real production setting provides more learning signal for your use case than a million generic examples.
The Integration Era
The AI industry is still in the "scale will solve it" mindset. Bigger models. Longer context windows. More training data. Just keep scaling.
For general capabilities, this works. For enterprise deployment, it's insufficient.
The companies that will actually extract value from AI are not waiting for better models. They're building:
- Deep integration: Connecting AI to where institutional knowledge actually lives
- Continuous capture: Ensuring valuable context gets recorded as it's created
- Expert feedback loops: Learning from real decisions, not synthetic benchmarks
- Permission-aware systems: Making sensitive context safely accessible
This looks like "hand-engineering" if you're measuring against general benchmarks. But general benchmarks don't measure what enterprises need.
A Nuanced Lesson
The bitter lesson isn't wrong. It's incomplete.
For tasks that can be specified in the input, scaling wins. Don't hand-engineer features. Don't encode heuristics. Just scale.
For tasks that require contextual knowledge not in the input, scaling alone isn't sufficient. You need integration, capture, and feedback systems that bring relevant context to the model.
The former is the domain of benchmark improvements. The latter is the domain of production deployments that actually work.
Enterprise AI is in the latter domain. The bitter lesson is necessary but not sufficient. What's missing isn't more compute. It's the right context, available at the right time, with the right permissions.
The Market Implication
This explains the AI deployment gap. Why 85% of organizations can't use AI effectively despite having access to the same models as everyone else.
Models are commoditizing. Anyone can access GPT-4. Anyone can access Claude. The models are smart enough.
What's not commoditized:
- Your institutional knowledge
- Your integration to where context lives
- Your capture systems for decision traces
- Your expert feedback loops
This is where defensible value is created. Not in model capability, but in context infrastructure.
The bitter lesson says compute wins. In enterprise AI, context wins.
The companies that understand this are building context infrastructure today. The companies that don't are waiting for the next model release to solve their problems.
One of these strategies will work.
At Context, we're building what the bitter lesson misses: the context infrastructure that makes AI useful in production. Learn more at context.inc
Share this article