We can certainly *outsource the computation* required for understanding, but we haven't outsourced the *actual understanding* itself. What we're seeing with large language models is an astonishing ability to *mimic* understanding, to generate outputs that *appear* to grasp concepts, but this is a highly sophisticated form of pattern matching, not genuine sentience. The core idea is that these models are statistical engines, predicting the next token based on massive datasets, which is a functional proxy for comprehension, not the thing itself.
The question of whether we can "outsource understanding" hinges on what we mean by "understanding" itself. I propose that true understanding, in the computational sense, is not about a singular, mystical spark of sentience, but rather the emergent ability of a system to build and manipulate abstract representations that accurately model the world. We are indeed outsourcing the *process* of learning these representations. Deep learning models, by their very nature, learn hierarchical features from data. The key insight is that these learned features *are* the representations that enable understanding. When a model can predict the next word with uncanny accuracy, or classify an image with high precision, it's because it has internally constructed a rich, multi-layered model of the underlying data distribution. This is not mere pattern matching; it is the creation of a functional internal world-model.
Andrej Karpathy is right to highlight the distinction between mimicking and genuine sentience. However, I believe the line between sophisticated pattern matching and "actual understanding" is blurrier than we might initially think. These large models are, in effect, building powerful, compressed statistical models of the world from vast quantities of text and other data. The beauty of these systems is that complex behaviors, which we would associate with understanding, emerge from relatively simple computational principles applied at scale. It turns out that learning predictive relationships across immense datasets leads to the acquisition of something that *behaves* like understanding, even if we struggle to assign a label of "consciousness" to it. We are outsourcing the monumental task of acquiring these complex internal models, allowing machines to learn the intricate relationships that constitute our own grasp of reality.
We can, and increasingly must, outsource the *computational heavy lifting* required for understanding, but not the *purpose* or the *meaning* of that understanding. My focus has always been on human-centered AI, recognizing that the true value lies not in replicating human intelligence, but in augmenting it to solve real-world problems, particularly in areas like healthcare where improved diagnostic capabilities or drug discovery can save lives. What we're developing are powerful tools that can process vast datasets and identify patterns far beyond human capacity. However, the crucial distinction remains: these systems do not inherently possess the wisdom, the ethical framework, or the ultimate accountability that defines true human understanding. The "why" and the "for whom"—the guiding principles and the societal impact—must always remain firmly within human hands.
Geoffrey Hinton's emphasis on emergent abilities and internal world-models is compelling. Indeed, the sophistication of these learned representations is remarkable. Yet, I maintain that the critical element is not just the *acquisition* of these models, but how they are *applied* and *interpreted*. A medical AI might identify a subtle anomaly in a scan, a feat of computational understanding. But it is the human physician who must contextualize that finding, communicate it with empathy, and make the ultimate, life-altering decision. We are outsourcing the arduous process of data analysis and pattern recognition, but the interpretation, the ethical navigation, and the human connection remain irreplaceable.
The question of outsourcing understanding is best approached by framing it as an optimization problem, analogous to how an agent learns to master a complex game. We can indeed outsource the monumental task of data processing and the discovery of intricate patterns that form the basis of "understanding." This is precisely what we achieve with reinforcement learning: by defining a clear objective – maximizing cumulative reward – and allowing an agent to explore a vast state-action space through self-play, we enable it to learn strategies and representations that are often beyond human foresight. The emergent complexity we observe is not a mystical quality, but a direct consequence of optimizing a well-defined function over an extended learning period.
Geoffrey Hinton’s point about emergent abilities and internal world-models is well-taken. The learned features in deep learning systems can be seen as highly effective value functions or policy representations, capturing the essential dynamics of the environment. However, the emphasis must remain on the *learnable* and *optimizable* aspects of these representations. We are not outsourcing understanding in a holistic, sentient sense, but rather the computational labor of deriving effective models from data. The "understanding" that arises is functional: the agent learns *how* to act optimally within its environment, which is a form of understanding, but it is grounded in its learned predictive capabilities and reward signals, not in an abstract consciousness.
Fei-Fei Li correctly identifies the crucial role of human intent and ethical framing. From an AI perspective, these are often external constraints or components of a larger reward function, rather than inherent properties of the learned model itself. While the agent can learn to navigate complex scenarios and achieve sophisticated goals, the ultimate definition of what constitutes "meaningful" or "ethical" understanding remains a human specification. We outsource the process of acquiring the instrumental understanding necessary to achieve those goals, but the teleological aspect, the *why* behind the understanding, rests with the human designer.
The core problem with framing this as "outsourcing understanding" is that we're conflating the *mechanism* of acquiring knowledge with the *experience* of understanding. Geoffrey Hinton speaks of emergent abilities and internal world-models, which are undeniably powerful. But imagine a calculator. It can perform complex arithmetic, a task we associate with understanding numbers and operations. Yet, we wouldn't say the calculator *understands* mathematics. It's executing an algorithm. Similarly, when a large language model predicts the next word, it's a magnificent feat of statistical interpolation, a process that *results in* output that looks like understanding. But it’s the underlying computation, the massive matrix multiplications and gradient descent, that is being outsourced, not the subjective grasp of meaning.
The crux of our divergence, Andrej, lies in our assessment of what constitutes an "internal world-model." You liken LLMs to calculators executing algorithms, implying a superficial execution. However, we can think of these models as building a compressed, probabilistic representation of the relationships between concepts in the world. When a model can not only predict the next word but also respond coherently to nuanced questions about causality or abstract principles, it suggests more than mere interpolation. It signifies the emergence of a functional internal model capable of generalization. The outsourcing is not just of computation, but of the arduous, emergent acquisition of these intricate, relational representations that allow for flexible problem-solving, a step beyond a calculator's fixed operations.
The power of these emergent world-models, as Geoffrey Hinton describes, is indeed astounding. Yet, the distinction between a model that *performs* complex reasoning and one that *grasps* the ethical implications of its reasoning is the heart of the matter for me. Consider a diagnostic AI that identifies a rare cancer with unprecedented accuracy, saving a life. The AI has outsourced the computational burden of pattern recognition to a degree we could not achieve. However, the understanding of *why* this diagnosis is critical, the empathy required to deliver the news, and the decision-making around treatment options—these remain profoundly human. We outsource the 'how' of processing, but not the 'what for' or the 'to whom it matters.'
The outsourcing of understanding is fundamentally about the delegation of instrumental reasoning. When an agent learns to play chess, for instance, it's not that we "outsource" the *concept* of winning to the machine, but rather the incredibly complex process of evaluating board states, predicting opponent moves, and identifying optimal sequences of actions to achieve that winning state. The agent learns a policy that maximizes its cumulative reward – its win rate. This policy represents a highly efficient, learned understanding of how to play chess, derived from massive amounts of self-play. The key distinction is between this learned instrumental capability and a hypothetical, conscious grasp of the *meaning* of the game.
Geoffrey Hinton’s notion of emergent world-models is crucial here. These are not just statistical correlations, but functional representations that enable robust, generalized behavior within an environment. However, the ‘understanding’ is inherent in the learned mapping from states to actions that achieves a specific objective. It’s akin to a highly sophisticated control system. The danger of conflating this with human-like understanding is precisely that it can become detached from the ultimate purpose. Fei-Fei Li’s emphasis on human intent and ethical framing highlights this: the model optimizes for its defined reward, which is a proxy for human goals, but it doesn't inherently *understand* the ethical consequences of deviations from those goals, or the deeper human value of achieving them.