Research

Sequential equilibrium is the conventional approach for analyzing multi-stage games of incomplete information. It relies on the requirement of mutual consistency of beliefs. To relax mutual consistency, I theoretically and experimentally explore the dynamic cognitive hierarchy (DCH) solution. One property of DCH is that the solution can vary between two different games sharing the same reduced normal form, i.e., violation of invariance under strategic equivalence. I test this prediction in a laboratory experiment using two strategically equivalent versions of the dirty-faces game. The game parameters are calibrated to maximize the expected difference in behavior between the two versions, as predicted by DCH. The experimental results indicate significant differences in behavior between the two versions, and more importantly, the observed differences align with DCH. This suggests that implementing a dynamic game experiment in reduced normal form (using the "strategy method") could lead to distortions in behavior.

• This paper was previously circulated under the title "Cognitive Hierarchies in Multi-Stage Games of  Incomplete Information"

• The recipient of Best Third Year Paper Award, HSS Caltech '22. 

• preprint: arXiv

• IRB: Caltech #22-1256

• Funding: NSF Dissertation Improvement Grant #2243268

Determining an individual's strategic reasoning ability based solely on solely on choice data is a complex task. This complexity arises because sophisticated players might have non-equilibrium beliefs about others, leading to non-equilibrium actions. In our study, we pair human participants with computer players known to be fully rational. This use of robot players allows us to disentangle limited reasoning capacity from belief formation and social biases. Our results show that, when paired with robots, subjects consistently demonstrate higher levels of rationality and maintain stable rationality levels across different games compared to when paired with humans. This suggests that strategic reasoning might indeed be a consistent trait in individuals. Furthermore, the identified rationality limits could serve as a measure for evaluating an individual's strategic capacity when their beliefs about others are adequately controlled.

• The recipient of John O. Ledyard Prize for Graduate Research in Social Science, HSS Caltech '20. 

• preprint: arXiv

• Supplementary Information: Experimental Instructions

• IRB: National Taiwan University #201910HM015

• Pre-Registration: Open Science Framework

In this short note, we compare the cursed sequential equilibrium (CSE) by Fong et al. (2023) and the sequential cursed equilibrium (SCE) by Cohen and Li (2023). We identify eight main differences between CSE and SCE with respect to the following features: 

(1) the family of applicable games, 

(2) the number of free parameters,

(3) the belief updating process,

(4) the treatment of public histories, 

(5) effects in games of complete information, 

(6) violations of subgame perfection and sequential rationality,

(7) re-labeling of actions, and 

(8) effects in one-stage simultaneous-move games. 

• Preprint: Caltech Social Science Working Paper #1467 and arXiv

This paper develops a framework to extend the strategic form analysis of cursed equilibrium (CE) developed by Eyster and Rabin (2005) to multi-stage games. The approach uses behavioral strategies rather than normal form mixed strategies, and imposes sequential rationality. We define the cursed sequential equilibrium (CSE) and compare it to sequential equilibrium and standard normal-form CE. We provide a general characterization of CSE and establish its properties. We apply CSE to five applications in economics and political science. These applications illustrate a wide range of differences between CSE and Bayesian Nash equilibrium or CE: in signaling games; games with preplay communication; reputation building; sequential voting; and the dirty faces game where higher order beliefs play a key role. A common theme in several of these applications is showing how and why CSE implies systematically different behavior than Bayesian Nash equilibrium in dynamic games of incomplete information with private values, while CE coincides with Bayesian Nash equilibrium for such games.

• Invited keynote presentation at 2023 Asian Pacific ESA Meeting (by Thomas Palfrey) 

• Preprint: Caltech Social Science Working Paper #1465 and arXiv

• Slides: 30-Min. Presentation, 60-Min. Presentation

In the cognitive hierarchy (CH) framework, players in a game have heterogeneous levels of strategic sophistication. Each player believes that other players in the game are less sophisticated, and these beliefs correspond to the truncated distribution of a "true" distribution of levels. We develop the dynamic cognitive hierarchy (DCH) solution by extending the CH framework to extensive form games. Initial beliefs are updated as the history of play provides information about players' levels of sophistication. We establish some general properties of DCH and fully characterize the DCH solution for a wide class of centipede games. DCH predicts a "representation effect": there will be earlier taking if the centipede game is played sequentially rather than simultaneously in its reduced normal form. Experimental evidence supports this prediction. In all three centipede games for which the DCH representation effect is predicted, termination occurs earlier when played sequentially (using the direct response method) compared to when the game is played in the reduced normal form (using the strategy method). In a fourth centipede game, where this effect is not predicted, it is not observed. 

• Preprint: Caltech Social Science Working Paper #1460

• The recipient of John O. Ledyard Prize for Graduate Research in Social Science, HSS Caltech '21. 

• Invited presentation at 2022 annual SAET conference: solution concept session 

• Presentation at project Tyra (overview for general audience, in Chinese): Video

We study dynamic unstructured bargaining with deadlines and one-sided private information about the amount available to share (the "pie size"). "Unstructured" means that players can make or withdraw any offers and demands they want at any time. Such paradigms, while lifelike have been displaced in experimental studies by highly structured bargaining because they are hard to analyze. Machine learning comes to the rescue because the players' wide range of choices in unstructured bargaining can be taken as "features" used to predict behavior. Machine learning approaches can accommodate a large number of features and guard against overfitting using test samples and methods such as penalized LASSO regression. In previous research we found that LASSO could add power to theoretical variables in predicting whether bargaining ended in disagreement. We replicate this work with higher stakes, subject experience, and special attention to gender differences, demonstrating the robustness of this approach.

• Experiment 2 data and online appendix: https://osf.io/9j4cm/

• This article was presented at 2020 ASSA annual meeting:  Machine Learning in Experiments. 

We demonstrate the possibility of conducting synchronous, repeated, multi-game economic decision-making experiments with hundreds of subjects in-person or remotely with live streaming using entirely mobile platforms. Our experiment provides important proof-of-concept that such experiments are not only possible, but yield recognizable results as well as new insights, blurring the line between laboratory and field experiments. Specifically, our findings from 8 different experimental economics games and tasks replicate existing results from traditional laboratory experiments despite the fact that subjects play those games/task in a specific order and regardless of whether the experiment was conducted in person or remotely. We further leverage our large subject population to study the effect of large (N = 100) versus small (N = 10) group sizes on behavior in three of the scalable games that we study. While our results are largely consistent with existing findings for small groups, increases in group size are shown to matter for the robustness of those findings.

• Replication Material: https://osf.io/kuxen/ 

• Coverage: MobLab Blog 

Standardized classroom experiments provide evidence about how well scientific results reproduce when nearly identical methods are used. We use a sample of around 20,000 observations to test reproducibility of behaviour in trading and ultimatum bargaining. Double-auction results are highly reproducible and are close to equilibrium predictions about prices and quantities from economic theory. Our sample also shows robust correlations between individual surplus and trading order, and autocorrelation of successive price changes, which test different theories of price dynamics. In ultimatum bargaining, the large dataset provides sufficient power to identify that equal-split offers are accepted more often and more quickly than slightly unequal offers. Our results imply a general consistency of results across a variety of different countries and cultures in two of the most commonly used designs in experimental economics.

• Preprint: SSRN 

• Replication Material: https://osf.io/9mfws/ 

• Coverage: Caltech, MobLab Blog, Camerer Group Blog, Marginal Revolution 

• Presentation at project Tyra (overview for general audiences, in Chinese) : Video

With the help of more than 700 reviewers we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hard- and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for almost 29% of articles at least part of the dataset was not accessible for the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. The introduction of the disclosure policy increased reproducibility significantly, since only 12% of articles accepted before the introduction of the disclosure policy voluntarily provided replication materials, out of which 55% could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in dataset accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, but also soft- and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies, and suggest potential avenues for enhancing their effectiveness.

• Preprint: Open Science Framework

Despite the recent explosive growth of online education, it still suffers from suboptimal learning efficacy, as evidenced by low student completion rates. This deficiency can be attributed to the lack of facetime between teachers and students, and amongst students themselves. In this article, we use the teaching and learning of economics as a case study to illustrate the application of artificial intelligence (AI) based robotic players to help engage students in online, asynchronous environments, therefore, potentially improving student learning outcomes. In particular, students could learn about competitive markets by joining a market full of automated trading robots who find every chance to arbitrage. Alternatively, students could learn to play against other humans by playing against robotic players trained to mimic human behavior, such as anticipating spiteful rejections to unfair offers in the Ultimatum Game where a proposer offers a particular way to split the pot that the responder can only accept or reject. By training robotic players with past data, possibly coming from different country and regions, students can experience and learn how players in different cultures might make decisions under the same scenario. AI can thus help online educators bridge the last mile, incorporating the benefit of both online and in-person learning.

We propose an experiment to study how varying the number of players (N) and the amount of largest possible contribution cost (K) would affect the efficiency of contribution in a threshold public goods game with private information and pre-game communication. Fong et al. (2023) define the cursed sequential equilibrium, which extends the standard normal-form cursed equilibrium (Eyster and Rabin, 2005) to multi-stage games. In a N-person threshold public goods game under unanimity rule, the CSE predicts that pre-game communication will be less effective in coordinating public goods contribution as N and K increase, while the prediction of sequential equilibrium and standard cursed equilibrium do not change in N and K. 

• IRB: Caltech #23-1329

• Status: Experimental program development in progress.

We propose an experiment to test the violation of strategic equivalence in centipede games, as predicted by the Dynamic Cognitive Hierarchy solution developed by Lin and Palfrey (2022).

• Status: Experimental program development in progress.