The prevailing direction in artificial intelligence development treats autonomy as the primary axis of progress: the goal, implicitly, is to build systems capable of acting on their own with less and less human involvement. This essay argues for a different orientation. AI should be built to extend and amplify human judgment rather than to substitute for it, and this is not merely an ethical preference but a technical requirement, one rooted in the nature of knowledge itself.

Knowledge as Distributed and Tacit

Most intelligence problems worth solving are not like chess. In chess, and in formal mathematics, the objective is static and fully specified, and the relevant information is public: every rule of the game is visible to any party capable of processing it. In such closed systems, an autonomous system can, in principle, outperform any human competitor without needing ongoing human input.

Economically meaningful work rarely takes this form. The knowledge required to run a kitchen, price inventory in a shop, or manage a firm is not fully articulable; it is acquired through sustained engagement with a task and updated continuously in response to feedback that is local to the person doing the work. Michael Polanyi's concept of tacit knowledge, developed in The Tacit Dimension (1966), captures this: we know more than we can say, and much of what we know cannot be transferred by explicit instruction. Friedrich Hayek made a parallel argument in his 1945 essay "The Use of Knowledge in Society," contending that centralized economic planning fails not for want of computational power but because the information a planner would need is inherently dispersed and cannot be fully aggregated at any single point.

The same structural limitation applies to AI trained once in a centralized location and deployed uniformly across users. Such a system cannot access the tacit, local knowledge that differentiates one organization's practice from another's, any more than a central planner could access the dispersed knowledge of every shopkeeper and factory manager. Intelligence and knowledge production are complementary, not substitutable. A system built to absorb and adapt to local expertise will outperform one built to replace that expertise with a standardized output.

A Case in Point: Toyota

Toyota's 2014 decision to reintroduce skilled craftsmen onto production lines that had already been substantially automated illustrates the point. The move was intended to preserve and grow the tacit expertise needed to supervise and improve automated systems. Automation and human skill development are mutually reinforcing rather than competing processes: an organization that automates without cultivating the underlying expertise risks losing the capacity to direct or correct its own machinery.

Interfaces as an Engineering Problem

Human participation in AI systems is often treated as a design afterthought, a matter of user experience polish rather than a genuine engineering constraint. This is a mistake. The narrowness of current human-AI interfaces, largely confined to sequential text exchanges with substantial latency, is a real bottleneck on what collaboration can achieve. Human collaboration in practice involves interruption, correction, and revised intent expressed in real time; a slow text interface cannot carry this bandwidth. The right response is to build interactivity into a system's training itself, so that gains in raw capability translate directly into improved collaborative behavior, rather than relying on external scaffolding layered on top of a static model.

This also bears on evaluation methodology. Benchmarks that measure how long a model can operate autonomously without human correction, such as METR's time-horizon measurements, capture something real but partial. They say nothing about the quality of outcomes achieved through sustained human-AI collaboration, a metric that only individual organizations are positioned to assess for themselves, since it depends on context no external benchmark can capture.

Alignment as a Problem of Concentration

Human values, like human knowledge, are irreducibly plural and locally held. A system in which a small number of organizations determine the values encoded into widely used models constitutes a concentration of normative authority that should concern us. Power which depends less on the productive contribution of ordinary people has correspondingly weaker incentive to serve their interests, a dynamic Luke Drago and Rudolf Laine describe as the "intelligence curse." A further concern arises from the practice of using a model's own outputs to train its successor: absent external correction, each generation inherits the dispositions of the one before it, narrowing rather than expanding the diversity of values represented in AI systems over time.

John von Neumann observed in 1955 that the beneficial and harmful uses of a technology are rarely separable in advance. Managing that ambiguity is not a problem solved once at training time; it is an ongoing exercise of judgment, distributed across the people making decisions about how a system is used. The remedy for concentrated alignment is architectural: values should be encoded directly into model weights that are owned and adjustable by the organizations and individuals using the system, rather than relying solely on prompt-level customization, which leaves deeper model dispositions unchanged and creates a more exploitable surface.

Assessment and Open Questions

Framing questions usually treated as ethical or political, who should control AI's values, how much autonomy AI should have, as engineering questions with engineering answers is a productive move: build tools for weight-level customization, build richer interaction models, build evaluation frameworks tied to organizational outcomes rather than benchmark autonomy scores. But this technical program leaves open whether it can resolve tensions that are also economic and political in character. Smaller organizations may lack the resources to meaningfully customize model weights even if the tools exist. And it remains uncertain whether decentralized alignment produces genuine value diversity, as Hayek's account of markets would suggest, or simply fragments the same underlying problems across more actors without resolving them. The analogy to decentralized knowledge aggregation in economic markets is suggestive, but AI markets do not yet display the competitive dynamics that made Hayek's original argument compelling.

Conclusion

The long-term trajectory of AI development need not converge on either full human displacement or a small number of centrally governed systems. Grounding technical priorities in established accounts of tacit and distributed knowledge points toward specific, actionable commitments: richer interaction design, weight-level customization, and decentralized evaluation. Whether this alternative path succeeds will depend on evidence not yet available, on whether organizations given the tools to customize and align AI to their own knowledge and values actually do so in ways that preserve, rather than merely relocate, the concentration of power such a path is meant to avoid.