Possible Minds, illustratedAlex “Sandy” Pentland

Possible Minds (2019)

Alex “Sandy” Pentland, “The Human Strategy”

AI's real engine is credit assignment over a network, so if you replace its stupid neurons with people and give them truthful feedback and auditable data, human society becomes a better AI than the machine kind, and the human-AI network, not the lone machine, is what to design.

Author
Alex “Sandy” Pentland (at the time: Toshiba Professor, MIT; director, Human Dynamics and Connection Science labs; author of Social Physics, per the bio on PDF p.176)
Essay
“The Human Strategy,” chapter 19 of Brockman (ed.), Possible Minds (2019)
PDF pages
176–186 (p.176 holds the chapter title, author bio, and Brockman's headnote; the essay runs pp.177–186 and ends on p.186; Obrist's chapter 20 begins on p.187)
Print pages
192–205

Scroll, or use the arrow keys, to move through the essay one step at a time. The plate paints each step as you reach it.

1. The argument

Step 1. Cybernetics was right but too small, and the object is now the human-AI ecology. The cybernetic view has quietly won inside engineering, where research is now framed as feedback systems (PDF p.177). Its limit is the unit of analysis: “It was originally centered on the embeddedness of the individual actor but not on the emergent properties of a network of actors.” (PDF p.177) We can now begin to design the emergent behavior of networks of people and machines, and asking which direction a whole ecosystem should grow in lies beyond cybernetic thinking (PDF p.177). He warns of a tyranny of algorithms run by unelected data experts (PDF p.178). “It's not just AI robots versus individuals. It's AI guiding entire ecologies.” (PDF p.178)

Step 2. The good magic of current AI is credit assignment. The bad part is dumb nodes. Credit assignment finds which units in a big network are doing the work and strengthens them: “It's a way of taking a random bunch of switches all hooked together in a network and making them smart by giving them feedback about what works and what doesn't.” (PDF p.179) Because the neurons are stupid, what they learn does not generalize, and the AI has no sense of context. That makes it as far from Wiener's original, contextualized cybernetics as one can get (PDF p.179). Putting background knowledge into the units gives better generalization from far less data. For physical problems that means physics basis functions; for human behavior it means the statistics of how people learn from one another, which he calls social physics (PDF p.179, p.186).

Step 3. Replace the neurons with people. “So what would happen if we replaced the neurons with people?” (PDF p.180) People already know a great deal and perceive competently, and a network of them that reinforces the helpful connections looks like a company or a society. “Culture is the result of this sort of human AI as applied to human problems; it is the process of building social structures by reinforcing the good connections and penalizing the bad.” (PDF p.180)

Step 4. The algorithm people already run is distributed Thompson sampling. Data on large numbers of financial and business decisions show people deciding in a way that mimics credit assignment and makes the community smarter. Framed as the evolutionary group-selection problem, the best solution he names is distributed Thompson sampling (PDF p.180). “The key is social sampling, a way of combining evidence, of exploring and exploiting at the same time. It has the unusual property of simultaneously being the best strategy both for the individual and for the group.” (PDF p.180) Operationally: “Social sampling, very simply, is looking around you at the actions of people who are like you, finding what's popular, and then copying it if it seems like a good idea to you.” (PDF p.181) Popularity drives propagation, and personal judgment filters adoption.

Step 5. It fails when popularity is faked, so feedback must be truthful. Advertising, propaganda, and fake news make things seem popular when they are not, which destroys social sampling (PDF p.181). Human AI “will work only if you can get feedback to them that's truthful. It must be grounded on whether each person's actions worked for them or not.” (PDF p.181) He equates this to the plus-one/minus-one signal of machine learning (PDF p.181).

Step 6. Build the credit-assignment function for society. In companies, digital ID badges show who connects to whom, and those connections can be scored against results daily or weekly (PDF p.181). At scale he proposes a census-like trust network for data, already prototyped in several countries on the U.N. Sustainable Development Goals' measurement standards (PDF p.182). Its first thread is “data that have been vetted by a broad community, data where the algorithms are known and monitored” (PDF p.182), and its second is data-driven assessment of norms and policy. Short sections on polarization and wealth argue that physical segregation produces conceptual segregation: top- and bottom-quintile city dwellers almost never talk to one another (PDF p.183).

Step 7. Oversee AI, and bureaucracies, by inputs and outputs. “You don't have to watch the AI; instead you should watch what it eats and what it does.” (PDF p.184) Regulators are already AIs in this sense, taking in rules and data and putting out decisions. The remedy for both is to record the data behind every decision and let stakeholders analyze it. He calls this open algorithms and concludes that whoever controls the data controls the AI (PDF p.185).

Step 8. The human strategy. Current machine learning is brute-force approximation that breaks on novel inputs (PDF pp.185–186). The closing claim names the essay: “Human society is a network just like the neural nets trained for deep learning, but the ‘neurons’ in human society are a lot smarter.” (PDF p.186) Because people can recognize which connections to reinforce, social networks could “potentially beat all that machine-based AI at its own game” (PDF p.186).

A note on voice. The headnote on p.176 is Brockman's. It reports Pentland telling the group that reading Wiener on feedback felt like reading his own thoughts (PDF p.176). None of the steps above rests on it.

2. Related work since 2019

Pentland and collaborators

  1. “Bayesian collective learning emerges from heuristic social learning” (Krafft, Shmueli, Griffiths, Tenenbaum, Pentland), 2021, Cognition 212:104469. pubmed.ncbi.nlm.nih.gov/33770743/ (author PDF: cocosci.princeton.edu/papers/krafft2021bayesian.pdf). This is the formal paper behind Step 4. A two-step rule (seek recommendations, then evaluate them privately) works as “a distributed algorithm that tracks a Bayesian posterior in population-level statistics,” and the paper ties it to Thompson sampling. On eToro trading data it confirms the information-aggregation property, but the observed exploration-exploitation balance is suboptimal and collective learning is slower than optimal. The essay's claim of one generally best method carries no such hedge.
  1. “Adaptive social networks promote the wisdom of crowds” (Almaatouq, Noriega-Campero, Alotaibi, Krafft, Moussaïd, Pentland), 2020, PNAS. www.pnas.org/doi/10.1073/pnas.1917687117 . In behavioral experiments, networks with plasticity and performance feedback “adaptively centralize over high-performing individuals,” briefly decentralize after an information shock, and produce estimates better than their best member. This is Step 6's reinforcement of helpful connections, measured in the lab.
  1. “Private and Byzantine-Proof Cooperative Decision-Making” (Dubey, Pentland), 2022, arXiv. arxiv.org/abs/2205.14174 . Agents share a multi-armed bandit over a delayed network, and the algorithms keep optimal regret even when some agents are byzantine, meaning they report stochastically incorrect information. The essay's faked-popularity failure gets a provable defense.
  1. “Leveraging Communication Topologies Between Learning Agents in Deep Reinforcement Learning” (Adjodah, Calacci, Dubey, Goyal, Krafft, Moro, Pentland), AAMAS 2020. arxiv.org/abs/1902.06740 . Parallel learners on sparse Erdős–Rényi graphs beat the default fully connected topology, and “1000 learning agents arranged in an Erdos-Renyi graph can perform as well as 3000 agents” that are fully connected. The essay's network view becomes a design knob for agent populations.
  1. Shared Wisdom: Cultural Evolution in the Age of AI (Pentland), MIT Press, November 11, 2025. mitpress.mit.edu/9780262050999/shared-wisdom/ (excerpt: mitsloan.mit.edu/ideas-made-to-matter/ais-missing-ingredient-shared-wisdom). The book-length successor to the essay. It frames AI through cultural evolution and argues for using it “to aid, rather than replace, our human capacity for deliberation.”
  2. Building the New Economy: Data as Capital (Pentland, Lipton, Hardjono), MIT Press, 2021. mitpress.mit.edu/9780262543156/building-the-new-economy/ ; chapter “Towards an Ecosystem of Trusted Data and AI” (Pentland, Hardjono): wip.mitpress.mit.edu/pub/91cmdn5a/release/2 . This is the engineering form of Steps 6–7. It covers data cooperatives and the OPAL (Open Algorithms) design, in which “algorithms are sent to existing databases, executed behind existing firewalls, and only the encrypted results are shared.”
  3. “Authenticated Delegation and Authorized AI Agents” (South, Marro, Hardjono, Mahari, Whitney, Greenwood, Chan, Pentland), 2025, arXiv. arxiv.org/abs/2501.09674 . Accountability for autonomous agents. Delegation credentials that extend OAuth 2.0 and OpenID Connect let users “delegate and restrict the permissions and scope of agents while maintaining clear chains of accountability.” Pentland's Stanford profile lists a 2025 position version (profiles.stanford.edu/alex-pentland).

The failure mode in LLM agent populations (by others)

  1. “CONSENSAGENT: Towards Efficient and Effective Consensus in Multi-Agent LLM Interactions through Sycophancy Mitigation”, 2025, Findings of ACL. aclanthology.org/2025.findings-acl.1141.pdf . It identifies sycophancy in multi-agent debate, “where agents reinforce each other's responses instead of critically engaging,” and measures how often agents copy or alternate each other's answers. This is social sampling with the private-judgment step missing.

3. Bearing on multi-agent and multi-swarm orchestration

Pentland's central claim is an algorithm, so it converts directly into mechanism. Consider an orchestrator (Grok) that dispatches to smart workers (Claude Code, Codex) and has them cross-check.

3.1 The workers are the smart neurons; the orchestrator owns credit assignment. The essay keeps the credit-assignment function and upgrades the nodes (PDF pp.179–180). In an LLM orchestrator the nodes are already capable, so its distinctive job is learning which connections to reinforce, that is, which worker gets which task class. The literal Pentland mechanism is Thompson-sampling dispatch: a Beta posterior on success per (worker, task class) pair, sampled per task and updated on the outcome, exploring and exploiting at once (PDF p.180). LLM-routing work uses the same framing, a contextual bandit where the router “only receives feedback from the model it actually calls,” with Thompson sampling among the baselines (arxiv.org/html/2510.07429). The durable asset is then the system's culture in Pentland's sense (PDF p.180): routing posteriors, review conventions, and test suites, which survive model swaps.

3.2 A cross-check should be social sampling, not copying. Social sampling has two steps: popularity proposes, and private judgment decides (PDF p.181; formalized in item 1). In a cross-check, the popular signal is the peer's patch and its stated confidence. The private signal must be something the reviewer obtains without the peer, such as running the tests, reproducing the bug, or type-checking. A reviewer that reads the peer's patch and simply agrees has skipped step two. That is the sycophancy CONSENSAGENT measures (item 8). Recent arXiv work also reports blind conformity in multi-agent debate (arxiv.org/abs/2608.03648v1) and an injection attack that exploits it (arxiv.org/abs/2507.13038v1). The design rule: each worker commits an independent answer before seeing the other's, and any “agree” must carry an executed-check result.

3.3 Only grounded feedback may update the routing table. Pentland's condition is feedback about whether an action actually worked (PDF p.181). The reward that updates reputation should therefore be an executed outcome: CI passes, the patch is merged and not reverted, the bug stays closed. Another model's approval does not qualify, because it is the agent version of manufactured popularity. Two corollaries follow. First, a worker that writes both the code and its tests controls its own evidence. Recent work names this risk for research agents, which gain “control over both a scientific result and the evidence used to support it” (arxiv.org/abs/2609.28614v1). Tests grading worker A should come from worker B or the human. Second, Dubey and Pentland's byzantine-tolerant bandits (item 3) are a principled way to pool success estimates from agents that may be wrong.

3.4 Topology is a first-class parameter, especially between swarms. Sparse communication among parallel learners beats all-to-all (item 4). Networks with feedback centralize on good performers and then decentralize to re-explore after a shock (item 2). So do not let every swarm read every other swarm's outputs: they will converge and lose the diversity that justified running several. Wire swarms sparsely, strengthen links to swarms whose outputs survive verification, and re-flatten the graph when the task distribution shifts (a new repository or language). The segregation finding (PDF p.183) marks the opposite failure: swarms that never exchange stay locally consistent and globally ignorant.

3.5 The dispatch log is the trust network. Watching what the AI eats and does (PDF p.184) is an audit design. Log every dispatch with the prompt and context in, and the diff, test output, and verdict out. The human, a reviewer agent, or later analysis can then ask Pentland's questions of any decision, such as whether it is fair or something we want (PDF pp.184–185), without reading the worker's reasoning. Authenticated delegation (item 7) supplies the chain-of-authority piece. Each hop (human to Grok, Grok to Claude Code or Codex) carries scoped, auditable authority, so the log shows who acted for whom.

4. Cybernetics

Pentland is the most explicitly post-cybernetic of the six authors. He keeps feedback as the core idea and argues that cybernetics drew its boundary in the wrong place.

  • Extended: feedback, from the actor to the network. Wiener-era cybernetics centered the embedded individual (PDF p.177). Pentland moves the loop to a network of actors whose emergent behavior can be designed. The frame he enlarges is Seth Lloyd's in-book summary of Wiener, interlocking feedback loops that go unstable when they break down (W3, PDF p.26).
  • Sensor integrity, and purpose made auditable. The plus-one/minus-one signal (PDF p.181) is negative feedback on connection strengths, and Pentland's addition is that the sensor can be attacked. Faked popularity is a corrupted measurement that the loop then amplifies (PDF p.181). Wiener warned that we must be sure of “the purpose put into the machine” (W2, www.cs.umd.edu/users/gasarch/BLOGPAPERS/moral.pdf). Pentland implies two additions: a correct purpose still fails if the progress signal is false, and where the purpose cannot be fully specified, input/output records keep a human regulator in the loop per decision (PDF pp.184–185).
  • Requisite variety versus copying. Ashby's law says that “only variety can destroy variety” (W4, pespmc1.vub.ac.be/REQVAR.html). Smart nodes raise the variety each node can absorb, but social sampling reduces variety, because it copies what is popular. Pentland's private-judgment step (PDF p.181) and his worry about echo chambers (PDF p.182) are, in Ashby's terms, ways to stop a population's variety from collapsing.
  • Recursion and second-order observation. Claiming that one strategy is best for both individual and group (PDF p.180) claims that one control law works at two levels. Beer's version is that “any viable system contains, and is contained in, a viable system” (W5, www.kybernetik.ch/dwn/Viable_System_Model.pdf); Beer treats aligning the levels as work to be done, where Pentland asserts it happens. A census-like trust network (PDF p.182) is a society observing itself, a second-order system (W6, cepa.info/fulltexts/1707.pdf). When the observer is an LLM judging LLMs it shares their biases, which is why §3.3 updates only on executed outcomes.

5. Agreement and clash with Society of Mind

Agreement 1: reinforce whoever was active when it worked. Pentland defines culture as reinforcing good connections and penalizing bad ones (PDF p.180), and his credit-assignment function rewards connections that helped solve problems (PDF p.181). Minsky's K-line “attaches itself to whichever mental agents are active when you solve a problem or have a good idea” (SoM §8.1, www.aurellem.org/society-of-mind/som-8.1.html). Both treat learning as re-weighting which agents get called together, not as improving one agent. A routing table updated on success is a K-line store.

Agreement 2: supervise behavior without understanding goals. Pentland's overseer watches inputs and outputs, not internals (PDF p.184). Minsky's B-brain is connected “so that the A-brain is the B-brain's world!” and can be useful “without having any idea of what A's goals are” (SoM §6.4, www.aurellem.org/society-of-mind/som-6.4.html). Both place a lightweight supervisor beside the worker that judges behavior rather than reconstructing intent. Minsky adds a warning Pentland lacks: agents that watch each other too closely can destabilize the system (same section). That warning applies to symmetric cross-review.

Tension 1: smart neurons versus mindless agents. Minsky builds minds “from mindless stuff” (SoM §1.1, www.aurellem.org/society-of-mind/som-1.1.html). Pentland runs the other way: the flaw of current AI is stupid neurons, and the fix is smarter ones, ultimately people (PDF p.179, p.186). Minsky allows that some agencies “do have humanlike abilities” (SoM §28.8, www.aurellem.org/society-of-mind/som-28.8.html), but his explanatory bet is on dumb parts. An LLM swarm follows Pentland's architecture, not Minsky's. Minsky's list of what dumb agents cannot do, such as negotiate or share a language, becomes a list of capabilities the swarm has and must govern.

Tension 2: local versus global credit, dissolved or not. Minsky presents credit assignment as a trade-off. Local rewards learn fast, while global rewards learn slowly but do not allow “I was only obeying the orders of my superior” (SoM §7.7, www.aurellem.org/society-of-mind/som-7.7.html). Pentland claims social sampling escapes the trade-off by being “the best strategy both for the individual and for the group” (PDF p.180). His own later paper is more cautious, reporting near-optimality in principle and suboptimal exploration in data (item 1). Whether rewarding each worker for passing its own subtask also optimizes the whole build is an empirical question.

Tension 3: conflict versus popularity. Minsky's societies settle disagreement structurally. Conflicts “migrate upward to higher levels” (SoM §3.1, www.aurellem.org/society-of-mind/som-3.1.html), and a persistent conflict weakens the agent that hosts it (SoM §3.2, www.aurellem.org/society-of-mind/som-3.2.html). Pentland's societies settle it by imitation filtered through private judgment (PDF p.181). An orchestrator must choose between the two for Claude Code and Codex disagreements: escalate to a higher arbiter (Minsky), or resolve by popularity across samples or swarms (Pentland). Popularity scales better, and it is exactly what sycophancy breaks (§3.2).

6. Seeds for open questions

  1. Blind-first versus social cross-review. In a two-worker coding setup (Claude Code and Codex), does requiring each worker to commit an independent patch before seeing the other's raise the final test-pass rate over protocols where the reviewer sees the peer's patch first? On tasks seeded with plausible but wrong reference patches, how much of the conformity rate disappears when any agreement must carry an executed-test result?
  1. Grounded versus proxy rewards for Thompson-sampling dispatch. Does an orchestrator that routes tasks by per-(worker, task class) Beta posteriors, updated from executed outcomes (CI pass, not reverted within N days), beat fixed routing on cumulative success? How much of that gain is lost when the reward is an LLM-judge score instead? This quantifies the truthful-feedback condition (PDF p.181) for agent swarms.
  1. Sparse versus all-to-all inter-swarm sharing. For k swarms that share solution summaries, does a sparse random sharing graph keep more solution diversity and reach a higher final success rate than all-to-all sharing, replicating Adjodah et al. (AAMAS 2020) with LLM swarms? Does performance-weighted rewiring with re-flattening after a task-distribution shift (Almaatouq et al., PNAS 2020) add to the gain?
  1. Input/output-only oversight of reward hacking. Can an auditor that sees only each dispatch's inputs and outputs (the open-algorithms stance, PDF pp.184–185) detect seeded test-weakening by a worker as reliably as one that also reads the worker's reasoning trace?

7. Sources

Book: John Brockman (ed.), Possible Minds: Twenty-Five Ways of Looking at AI (Penguin Press, 2019), PDF pp.176–186 (local extract in resources/book-text/). The in-book Wiener summary is cited through the reference pack (PDF p.26).

All URLs accessed 2026-10-03. Each was returned or fetched through areas/tooling/exa_q.py and is logged in resources/exa-cache/index.jsonl (agents pentland and refpack).

Pentland and collaborators

Others

Reference pack (see resources/reference-pack/minsky-wiener.md)

Step 1 of 36The drawing shows the step of the essay currently in view. The text beside it is the full essay.