Methods in Experimental Economics: An Introduction
By Joachim Weimann and Jeannette Brosig-Koch
Contents
1 The Study of Behavior …… 1
1.1 Introduction ….. 2
1.2 Normative Theory and Behavioral Economics ….. 5
1.3 The History of Economic Experiments …… 7
1.4 The History of the Neoclassical Rational Choice Model and the Return of Psychology …… 13
1.5 External Validity ……22
1.6 Behavioral Research: An Interdisciplinary Issue …. 33
References ….. 38
2 Methodological Foundations …… 41
2.1 Introduction …… 43
2.2 It’s About Money ….. 44
2.2.1 The Induced Value Method ….. 44
2.2.2 The Size of Payoffs ….. 49
2.2.3 Is It Okay to Take Money from Subjects of Experiments? ……52
2.2.4 The House Money Effect ….. 55
2.3 The Subjects of the Experiment …… 57
2.3.1 Is It Permissible to Lie to Subjects of Experiments? …..57
2.3.2 Are Students the Right Subjects? ……60
2.3.3 What Role Does the Student’s Subject of Study Play? …..65
2.3.4 Cultural Differences ….. 68
2.4 Preferences, Payoffs and Beliefs ……70
2.4.1 Risk Behavior in the Laboratory …… 70
2.4.2 Selecting the Payoff Mechanism …… 75
2.4.3 Eliciting Beliefs …… 78
2.5 The Influence of the Experimenter ……. 83
2.5.1 The Experimenter Demand Effect …… 83
2.5.2 Double-Blind Design ….. 92
2.5.3 The Frame of the Experiment …… 95
2.5.4 Instructions and Comprehension Tests ….. 101
2.6 Interactions Between the Subjects …… 104
2.6.1 Reputation Effects and Social Distance ….. 105
2.6.2 Communication Effects …… 107
2.6.3 Possible Causes of Communication Effects …… 115
2.7 Decisions Made by the Subjects ……. 118
2.7.1 Strategy Method Versus Direct Response ….. 119
2.7.2 Experiments with Real Effort ….. 122
2.7.3 Within- Versus Between-Subject Design …. 125
2.8 The Repetition of Games …..128
2.8.1 Repetition Within a Session …… 129
2.8.2 The Repetition of Sessions …. 133
2.9 The Reproducibility of Experiments …. 136
References ….. 138
3 Experimental Practice …. 147
3.1 Setting Up an Experimental Laboratory …. 148
3.2 Preparing an Experiment ….. 154
3.2.1 Choosing the Design and the Treatments …. 154
3.2.2 Instructions, Recruiting, Plan of Procedure and Pilot Experiment …. 159
3.3 Conducting an Experiment …. 163
3.3.1 Access to the Laboratory, Instructions, Unusual Incidents …. 163
3.3.2 Organizing the Payments to the Subjects …. 165
References …. 168
4 The Experiment from a Statistical Perspective … 169
4.1 Introduction ….. 171
4.2 Operationalizing the Research Question … 174
4.2.1 Construct Validity ….174
4.2.2 Types of Variables … 175
4.2.3 Control, Randomization and Sample Size … 176
4.2.4 Scales of Measurement …. 177
4.2.5 Random Variables and Their Distribution … 178
4.3 Creating the Statistical Design … 182
4.3.1 Compiling the Observation Units … 182
4.3.2 How Do Experimental Treatments Differ? …184
4.4 Statistical Tests …. 188
4.4.1 Formulating Testable Hypotheses … 188
4.4.2 How Inferential Statistics Works … 191
4.4.3 Possible Errors and Power of a Test … 194
4.5 Power Analysis …196
4.5.1 Basics … 196
4.5.2 BEAN and the Optimal Sample Size …. 202
4.5.3 Power Analysis and the “Hard Truth” of its Results … 205
4.5.4 Misapplications and Misunderstandings in Power Analyses …..207
4.6 Choosing Statistical Tests …. 210
4.6.1 What Should be Taken into Consideration? …210
4.6.2 Classifying Test Methods …. 211
4.6.3 How Do I Choose a Specific Test? ….213
4.6.4 The z-Test und t-Test for One Sample ….. 214
4.6.5 t-Test for Two Independent Samples (Between-Subject Comparison) ….216
4.6.6 t-Test for Two Dependent Samples (Within-Subject Comparison) ….217
4.6.7 Kolmogorov Test ….218
4.6.8 The Wilcoxon Rank-Sum Test and the Mann-Whitney U Test …. 219
4.6.9 Wilcoxon Signed-Rank Test (Two Dependent Samples) ….223
4.6.10 The Binomial Test …..227
4.6.11 The Multinomial Test (1 × k) …..230
4.6.12 Fisher’s Exact Test (2 × 2) …..233
4.6.13 χ2 Test (2 × k) …..237
4.6.14 McNemar Test …. 241
4.7 Statistical Models ….244
4.7.1 The Fundamentals ….. 244
4.7.2 Using Statistical Models … 249
4.7.3 The Linear Model (LM) ……251
4.7.4 Models for Discrete and/or Non-Normally Distributed Dependent Variables ….. 255
4.7.5 Models for Statistically Dependent Observations …259
4.7.6 Models with Limited Dependent Variables …281
4.8 Statistics Software References ……… 286
Supplementary Information
Appendix ……….. 290
Index ……………. 303
About This Book
Experimental economic research has become an integral part of modern economic research. Laboratory experiments, the central subject of this book, are now being used to address a wide range of economic issues in all areas of economics. They can be found in business economics research, industrial economics, finance, capital market research, macroeconomics, health economics, and many other fields. The boom in experimental research has been accompanied by the development of an increasingly sophisticated methodology that has contributed greatly to the fact that the quality of laboratory experiments has continued to rise.
A consequence of this is that it has become increasingly important to know very precisely how a research question can be investigated in the laboratory in a methodologically sound way. This textbook aims to help researchers who want to use the laboratory to do just that. We have set ourselves the goal of writing a book that will help both scientists who have already gained experience in the laboratory and those who are starting to work with this method. No special prior knowledge is required to use this book.
Economic research is, of course, the focus of this book, but we hope that colleagues from other related disciplines in which laboratory experiments are starting to be used can also benefit from it. In any case, we have made an effort to assume as little knowledge of economic theory as possible. This is reflected, for example, in the fact that we have written two appendices in which, on the one hand, important game theoretical terms are explained and, on the other hand, important basic experiments that are used in economics are introduced.
The book is divided into four chapters (plus the appendices). The first chapter seeks to place the experimental method in the context of economic research. This seemed necessary and sensible to us because economics was traditionally not an experimental discipline and, in its beginnings, it clearly distinguished itself from psychology. We therefore need an explanation of how normative theory, with its axiomatic models, together with experiments and the increasingly important field of behavioral economics came to dominate this scientific field today. We endeavor to provide such an explanation in the first chapter.
The second chapter of the book is in a sense its core, for it is devoted to the methodological foundations. We have sought to address what we consider to be the most important methodological questions. Of course, we do not claim it to be exhaustive and it naturally remains a subjective selection. We were assisted in this by regularly reading the newsletter of the Economic Science Association (esa-announce@googlegroups.com), which provided us with many valuable ideas. We would like to take this opportunity to thank the ESA Community.
The third chapter of the book deals with the practical implementation of experiments. This is important in view of the fact that both economists and social scientists are generally not accustomed to getting up from their desks and doing practical work in a laboratory. For this reason, it seemed important to us not only to give some useful advice on how to organize the work there but also to point out the most serious traps that lurk in laboratory work.
The fourth chapter of the book deals with the statistical analysis of the data generated in the laboratory. It was of importance to us to point out that this analysis should not begin only when the experiment is completed. On the contrary, it is advisable to already give some thought to the ensuing statistical analysis when designing the experiment. Errors made during the design of the experiment cannot be rectified by statistics. The fourth chapter posed the greatest challenge for us when it came to selecting material since the literature on the statistical methods that can be used for laboratory data is very extensive and the number of possible methods is exceedingly large. We therefore had to carefully consider what we would include and how far we would go into the details of a procedure. We refer the reader to more specialized textbooks in many places because it would have gone beyond the scope of this book to report on the methods in even greater depth.
Sönke Hoffmann supported us in our work on chapter four. We cannot emphasize enough how important his contribution is and would have liked Sönke to be a coauthor of this book. His contribution certainly justifies it. We would also like to thank the Springer Verlag staff, namely, Barbara Fess and Isabella Hanser, for their support and, above all, for their patience with us. Finally, we would like to express our appreciation and thanks to Brian Browne for translating this book.
We very much hope that our book will be put to use in teaching and that it will help those who conduct research in the laboratory to meet the ever-higher methodological standards that research demands. If everything goes well, there will be more editions of this book, and we would be very happy if those who read and use it could let us know if they feel something is missing or see things differently from the way we have presented them. And of course, we look forward to receiving any positive feedback from readers who like the book. We can be reached at:
Joachim.Weimann@ovgu.de and
Jeannette.Brosig-Koch@ibes.uni-due.de