Do markets, as already argued by Adam Smith, have a civilising effect and thus make market participants more moral? Or are thinkers like Karl Marx and Thorstein Veblen closer to the truth and is moral behaviour subordinated to profit interests in market economies? The question of the influence of competition on morality can be traced back to the beginning of modern social science research, but a clear answer is still lacking: there are both empirical studies that find a positive effect of competition on moral behaviour and those that show the opposite. Researchers led by Felix Holzmeister, Michael Kirchler and Jürgen Huber from the University of Innsbruck, together with colleagues from Vienna, Stockholm and Amsterdam, used this open question as the starting point for a meta-study recently published in PNAS: “We wanted to use this question to investigate how great the variability of experimental research results can be when the same question is addressed with different study designs,” explains Felix Holzmeister from the Department of Economics at the University of Innsbruck, co-author of the study.
45 study designs
Usually, experimental studies are conducted with only one study design; researchers decide how morality and competition should be represented in an experiment. “It seems obvious that the study design directly determines the outcome of a study, as does the sample of respondents and the method of analysis. However, the extent of the influence of the study design on the result is still largely unclear due to a lack of empirical evidence,” says Holzmeister. In order to investigate the effect of study design on study results, the researchers launched a call to colleagues to submit study designs on the deliberately open question of the influence of competition on moral behaviour – in the end they ended up with 45 different experiments, submitted by 88 researchers from 75 different institutions in 18 countries. In order to exclude other effects as far as possible, the subjects were recruited from the same pool of participants and the data from the 45 individual study designs were analysed using the same statistical tests. In total, more than 18,000 people participated in one of the 45 studies via an online portal.
A meta-analysis of the 45 individual study results suggests a weak negative effect of competition on moral behaviour. However, the variability of the results is substantial: while many of the studies provide insignificant results, seven studies indicate a significant negative effect, and two studies indicate a significant positive effect. “The answer to the question of the influence of competition on moral behaviour thus depends crucially on which study design is chosen,” explains Felix Holzmeister.
Unnoticed uncertainty
Each of the 45 experiments could potentially have been conducted and published as an independent scientific study. Accordingly, drawing conclusions from a single result – based on a single study design – to a generalised statement involves considerable pitfalls. The uncertainty as to whether an alternative study design would have led to the same conclusion has so far mostly remained unconsidered. "Facing so large variation in effects across designs emphasizes that researchers should be careful about drawing generalized conclusions from a single study design," says Christoph Huber from the Institute for Markets and Strategy at the Vienna University of Economics and Business (WU Wien), one of the co-authors of the study. The approach of the Innsbruck authors shows a possible way out of this dilemma: Instead of many independent studies, the “team science” approach allows researchers to conduct much larger data collections in which different defensible study designs are systematically implemented in order to draw generalisable conclusions. In this way, uncertainty due to variability of results from different study designs can not only be measured, but also specifically used to accelerate the scientific knowledge process.
The raw data of the study are available online: https://osf.io/r6anc/
Publication:
Huber, Dreber, Huber, Johannesson, Kirchler, Weitzel, …, Holzmeister: Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs, PNAS 2023, Vol. 120, DOI: 10.1073/pnas.2215572120, https://doi.org/10.1073/pnas.2215572120