Questions

This event is a series of open discussions on the WIP work of exploring practical implementations of cooperative AI. We specifically focus our discussion on concrete, approachable ways to study and realize commitments for cooperation. And our goal is to bring together facilitate discussions between pragmatic commitment device researchers and cooperative AI researchers to better map out the directions of this collaboration.

On a high level, we try to answer those two questions:

1. How we can better formalize and study the properties of crypto commitment devices and the coordination games mediated by it via the lens of commitment games? Specifically, what kind of concrete simple games we can study to illustrate the power of crypto commitment devices.
2. How can we better design blockchains (a technology for implementing commmitments) for coordination games with algorithmic agent participants (cooperative AI)? What are some benchmark games that we can devise for boostrapping? e.g., are there any simple games where we can observe what kind of behavior emerges when AIs play games with the presence of a crypto commitment device.

To answer those two big questions, we might want to breakdown, and ask easier brainstorming questions to sparkle our creativity. So, for example:

1. Blockchain and AI: Synergy and Value Proposition
   • What, in your mind, is the most significant synergy between blockchain and AI?
   • What are the limitations around current studies and designs of the thesis of “blockchain as a commitment device”? Considering various approaches such as information design, game mining, principal-agent problem style payment, anti-mechanisms, darkdaos, and optimal credible auctions, how do we push things forward? What is lacking in crypto achieving this value proposition?
2. Commitment Devices: Role, Limitations, and Trade-offs
   • Both crypto and cooperative AI have focused significantly on commitment devices, a decentralized way of implementing coordination. How does this study relate to the traditional way of implementing coordination via mediators/mechanism design?
   • How should we navigate the tradeoff space between commitment devices and mechanism design? How do direct and indirect mechanisms come into play? Are we at a stage where we need more indirect mechanisms?
   • Commitments theoretically allow for optimal welfare in a game with a known structure. However, in practice, the game structure is often uncertain and dynamic. What do you think is the limit for commitments? How far can we push them, and should we push them?
3. Balancing Different Coordination Mechanisms
   • Negotiations and bargaining often come into play when there is abundant incomplete information. How far can we use negotiations to push the boundary of coordination?
   • Coordination can also be implemented via governance and policies. How do we balance this approach with the commitment devices approach? To what extent is human coordination insufficient?
   • What role can policies and regulations play in complementing the use of blockchain technology for AI coordination and alignment?
   • Are there any other approaches to consider, such as intervening in the training process? What, in your opinion, is the best way to achieve cooperative AI or AI safety, and why?
4. Practical Implementation of Cooperative AI
   • What are some robust ways to implement resolution of program equilibria? How could these solutions be realistically implemented for complex, economically meaningful applications?
   • Are there concrete coordination games with respect to Learning and Long-term Models (LLMs) and/or generative AIs that we can implement? For example, something in relation to recommender systems and private data?
   • Are there any benchmark games that we can implement to illustrate the idea of cooperative AI, such as Cicero, colored tiles, or any simple games where we can observe the emergent behavior of AIs playing games with the presence of a crypto commitment device?
   • How can we best leverage the "crypto sandbox" for testing the limits of AI coordination and for understanding the practical implications of different commitment devices?
   • What are the potential design variations for commitment devices in the context of AI coordination, and how might these different designs impact the behavior of AI agents?
5. Future Direction and Challenges
   • How can we ensure that cooperative AI solutions are robust in the face of future changes and market morphisms? Assuming we use a blockchain, will the blockchain mechanisms keep up with the speed at which the games are played? Will we witness more AI mechanism designers?
   • How can the field of AI leverage the benefits of crypto-economic commitments to address coordination and alignment challenges? What might be the potential hurdles in implementing this approach? What are your expected biggest challenges? Are there any fundamental limitations you see in this approach?
   • How might the evolving market structure impact the alignment and coordination of existing games?

---

Vincent Conitzer
Director, Foundations of Cooperative AI Lab (FOCAL)
Professor of Computer Science
(with affiliate/courtesy appointments in Machine Learning, Philosophy, Tepper School of Business)
Carnegie Mellon University

Head of Technical AI Engagement, Institute for Ethics in AI
Professor of Computer Science and Philosophy
Visiting Fellow at Pembroke College
Oxford University

2a. Commitment Devices: Role, Limitations, and Trade-offs
Both crypto and cooperative AI have focused significantly on commitment devices, a decentralized way of implementing coordination. How does this study relate to the traditional way of implementing coordination via mediators/mechanism design

Rephrasing: What can blockchain and AI say that is new about coordination? Can you put them somewhere on the map of previous literature, or is there a difference in kind between what they enable and what was there before? Do they trivially solve problems that were considered "hard" previously?

1b. Blockchain and AI: Synergy and Value Proposition
What are the limitations around current studies and designs of the thesis of “blockchain as a commitment device”? Considering various approaches such as information design, game mining, principal-agent problem style payment, anti-mechanisms, darkdaos, and optimal credible auctions, how do we push things forward? What is lacking in crypto achieving this value proposition?

What are tangible steps to map insights from the literature onto "crypto-native" problems?

• Bottom-up approach: Find small examples, try to solve them using previous ideas/models
• Top-down approach: Survey of results/taxonomy and mapping each category to crypto-specific problems

What are we missing? What is a critical learning from the theory of commitments that is "folklore" in the field but has very practical applications that we should think about? (Im)possibility results, underexplored avenues…

4a. Practical Implementation of Cooperative AI
What are some robust ways to implement resolution of program equilibria? How could these solutions be realistically implemented for complex, economically meaningful applications?

The eBay "autobid" function allowed passive bidders to participate meaningfully, a bit like CFMMs with passive liquidity allow for liquidity providers to be "always on". The ability to deploy a program to act on your behalf seems to enable new economic activities. But do they scale well? Are program equilibria useful beyond the smaller examples of solving prisoner's dilemma? New results look at how they compose with one another, is there more to be said about composition? Is this the next problem to solve for PE?

2c. Commitment Devices: Role, Limitations, and Trade-offs
Commitments theoretically allow for optimal welfare in a game with a known structure. However, in practice, the game structure is often uncertain and dynamic. What do you think is the limit for commitments? How far can we push them, and should we push them?

Can commitments become stale? Are there meaningful ways to commit in a complex environment, where conditional strategies need to condition on many unseen things? Theory of commitment when you have incomplete information, e.g., you don't know the whole strategy set/you are learning?

3b. Balancing Different Coordination Mechanisms
Coordination can also be implemented via governance and policies. How do we balance this approach with the commitment devices approach? To what extent is human coordination insufficient?

Interesting! People do not care about the role that policies and regulations play for blockchain and AI (3c), but they care about how "softer" institutions such as governance and policies enable human coordination. My question: What do you see as coming first? Do commit devices extend softer institutions, or do they enable them in the first place?

How can the field of AI leverage the benefits of crypto-economic commitments to address coordination and alignment challenges? What might be the potential hurdles in implementing this approach? What are your expected biggest challenges? Are there any fundamental limitations you see in this approach?