Serwe, Prings - Who will Win Wimbledon The Recognition Heuristic.pdf

(115 KB) Pobierz
241242381 UNPDF
Journal of Behavioral Decision Making
J. Behav. Dec. Making, 19: 321–332 (2006)
Published online in Wiley InterScience
( DOI: 10.1002/bdm.530
Who will Win Wimbledon? The Recognition
Heuristic in Predicting Sports Events
1 Giessen University, Germany
2 Saarland University, Germany
Goldstein and Gigerenzer (2002) described the recognition heuristic as a fast, frugal,
and effective decision strategy. However, most studies concerning the recognition
heuristic have been conducted in static domains, that is, in domains where it could
plausibly be argued that relevant variables stay relatively constant. Yet the question
is whether the heuristic would also work in dynamic environments where the quality
of the actors rises and falls, such as in sports. We tested performance of the recognition
heuristic in a dynamic environment and used it to predict the outcomes of tennis
matches in Wimbledon 2003. Recognition data of amateur tennis players and
laypeople was used to build recognition rankings. These rankings correlated with
official rankings and led to at least as good predictions. Simulations of individual
choices showed high recognition validities of both amateurs (0.73) and laypeople
(0.67). In a second study the recognition heuristic correctly predicted 90% of actual
individual choices. Overall, the recognition heuristic may be effectively generalized
to dynamic environments. Copyright # 2006 John Wiley & Sons, Ltd.
key words
decision making; recognition heuristic
In research on judgment and decision making, the term ‘‘heuristics’’ is frequently used to describe simple
and naive decision strategies. However, two different views on the effectiveness of heuristics are discussed.
On the one hand, heuristics can be seen as suboptimal replicas of statistical computations that lead to
systematic biases and decision failures (e.g., Kahneman, Slovic, & Tversky, 1982; Kahneman & Tversky,
1996; Pohl, 2004). On the other hand, Goldstein and Gigerenzer (2002) emphasize that heuristics can be
‘‘ecologically rational’’, that is, they achieve accurate inferences by exploiting patterns of information in
the environment with the help of evolved psychological capacities. Moreover, due to their simple application
* Correspondence to: Sascha Serwe, Giessen University, Department of Psychology, Otto-Behaghel-Str. 10F, 35394 Giessen, Germany.
Copyright # 2006 John Wiley & Sons, Ltd.
322 Journal of Behavioral Decision Making
they lead to fast, frugal, and therefore efficient results, when time, knowledge, and computational power
are limited. Gigerenzer and colleagues present several fast and frugal heuristics, for example, Take The Last,
QuickEst, etc. (Gigerenzer & Goldstein, 1996; Gigerenzer & Selten, 2001; Gigerenzer, Todd, & the ABC
Research Group, 1999; Todd & Gigerenzer, 2000). The simplest of all heuristics in their research program
is the recognition heuristic (Goldstein & Gigerenzer, 1999, 2002). The aim of this article is to test the effec-
tiveness and descriptive value of the recognition heuristic in a dynamic real world environment. In particular,
we predicted the outcomes of tennis matches in Wimbledon 2003 using the recognition heuristic and com-
pared its performance with predictions of two official rankings and the betting market.
Using the concept of ‘‘recognition’’ the world can be divided into two categories: the novel, unrecognized
objects (never heard or seen before) versus the previously experienced and recognized objects. If one has to
decide which of two objects has a higher criterion value (say for example, which city has a larger popula-
tion), this binary recognition information can result in predictions better than chance. Suppose there is an
environment where the probability of object recognition is positively correlated with the criterion (e.g., the
larger a city the higher is the probability that you know it). Then the recognition heuristic will perform well:
If one object is recognized and the other is not, infer that the recognized object has the higher value. Thus,
one crucial precondition that determines performance of the recognition heuristic is the correlation between
recognition and criterion. The strength of this relationship is measured by the recognition validity , which is
the number of correct predictions divided by the number of all discriminating pairs (i.e., excluding cases
where the recognition heuristic does not make a prediction).
Another precondition for the recognition heuristic is that there are at least some unrecognized objects.
More precisely, the recognition heuristic can be applied most often in a reference class of objects if half
of the objects are recognized. If you know more (or less) than half of the objects, the recognition heuristic
is applicable less often. Thus, if the recognition heuristic is the only decision rule and if > 0.5, the counter-
intuitive less-is-more effect (Goldstein and Gigerenzer, 1999, 2002) can be predicted. Suppose there are 100
cities and you have to decide, for all possible pairs, which city is bigger. Furthermore, there is a strong cor-
relation between recognition and population of a city. If you do not recognize any city, your only possibility
is to guess. If you recognize all cities, you cannot use the recognition heuristic either. However, if you recog-
nize half of the cities, the theoretical maximum of all pairs — for infinite samples it converges to 50% —
consist of one recognized and one unrecognized city. Due to the strong correlation many decisions in these
situations will be correct. If the recognition heuristic is not the only decision rule, further rules might
improve decisions in situations where both objects are recognized. The proportion of correct decisions in
these situations is defined as the knowledge validity . Yet as long as > , the less-is-more effect persists.
Empirical evidence
In the city population task described above usually about 90% of participants decide in accordance with the
recognition heuristic and achieve good results by using it (Goldstein & Gigerenzer, 1999, 2002). Moreover,
in group decisions with respect to city population, members who can use the recognition heuristic are more
influential than members who recognize both alternatives (Reimer & Katsikopoulos, 2004). However, the
recognition heuristic is domain specific and works only in environments in which the recognition validity
is high. Respectively the theory of the adaptive toolbox (Gigerenzer & Selten, 2001) predicts people to do
not use it mindlessly but only in situation where the subjective validity of the recognition cue is high.
For instance in the city population task people decide against recognized cities if they are tiny neighboring
towns or if they are (such as Chernobyl) recognized for reasons clearly unconnected to population size
Copyright # 2006 John Wiley & Sons, Ltd.
Journal of Behavioral Decision Making, 19, 321–332 (2006)
S. Serwe and C. Frings
The Recognition Heuristic 323
(Oppenheimer, 2003). Besides the city population task, Goldstein and Gigerenzer (1999, 2002) present
objective recognition validities above 0.6 for highest mountains, largest Italian provinces, largest deserts,
tallest buildings, largest islands, longest rivers, largest U.S. banks and largest seas.
All these domains have in common that the criterion stays relatively constant. Thus, the correct answer to
questions such as ‘‘Which island is larger— Borneo or Ellesmere Island?’’ is the same today as it was a few
years back and will be in the future. This might be essential for the recognition heuristic to work because the
correlation between recognition and the unknown criterion usually arises by mediators. This process, how-
ever, takes time. This is well documented for the city population task. Goldstein and Gigerenzer (2002)
counted in how many newspaper articles between 1985 and 1997 the name of the 83 largest German cities
appeared. There is a high correlation between the number of times a German city is mentioned during this
period and its population (ecological correlation) on the one hand. On the other hand, there is a strong cor-
relation between the number of times a German city is mentioned in American newspaper articles and its
probability of recognition by Americans (surrogate correlation). As a result, for Americans, recognizing a
German city allows inferring that in most cases this city has a larger population than an unrecognized
German city (recognition validity). Since the criterion (relative city population) stays constant, this inference
is facilitated. However, if the criterion itself varies, continuous and rapid adjustment of recognition memory
is needed. This might impair performance of the recognition heuristic.
Dynamic environments
Many everyday decisions have to deal with swiftly changing criterion values. How does the recognition heur-
istic perform in such dynamic real world environments? Besides the described static domains, so far only one
dynamic domain has been analyzed: There have been several attempts to use the recognition heuristic on the
stock market. On the one hand depots of recognized stocks seem to outperform depots of unrecognized
stocks, the market, chance depots and even mutual fonds (Borges, Goldstein, Ortmann, & Gigerenzer,
1999). On the other hand this does not hold in a bear market (Boyd, 2001) and the performance varies a
lot over time (Frings, Holling, & Serwe, 2003). However, the strategy used in these studies differed from
the city population task. In fact, aggregated recognition data has been used to build rankings, and several
portfolios have been derived from these rankings (e.g., one portfolio consisted of stocks from all companies
that have been recognized by more than 90% of the participants). This procedure is neither fast nor frugal and
has little in common with the original application of the recognition heuristic to two-alternative forced-
choice tasks. If you want to use solely your own recognition information for application of the recognition
heuristic to the stock market, you have to invest in stocks of all recognized companies and ignore stocks of all
unrecognized companies. This is again different from the standard application of the recognition heuristic.
Besides these methodical differences, it seems questionable whether performance of a strategy on the stock
market could be a valid test at all. Some researchers argue that no strategy can perform above-average on the
stock market — at least in the long run (see e.g., Cootner, 1967; Lucas, 1980). Even in the short run perfor-
mance of any strategy fluctuates to a large extent due to high volatility of the stock market and success in one
time period can hardly be generalized to other periods. Therefore, the inconsistent results of the recognition
heuristic on the stock market can not be interpreted safely. A further test in another dynamic environment,
which is perhaps better understood and which allows the application of the recognition heuristic in its ori-
ginal form, might be useful.
The prediction of tennis matches is such a test-situation: If you have to predict the winner of a tennis
match, you have to decide between two options and there are at least some pairs where you know one player
but do not know the other. In contrast to the stock market examples, it is a ‘‘natural two-alternative-forced-
choice task’’ and recognition heuristic can be applied fast and frugally in its original form. Additionally,
predictions for the same match can be derived from other strategies like official rankings or the betting mar-
ket which are known for their accurate predictions of sports events (Boulier & Stekler, 1999, 2003).
Copyright # 2006 John Wiley & Sons, Ltd.
Journal of Behavioral Decision Making, 19, 321–332 (2006)
324 Journal of Behavioral Decision Making
Will the recognition heuristic successfully predict tennis matches in Wimbledon? On the one hand, the
chances of success seem to be good: It seems plausible, that there is an ecological correlation between the
success of a tennis player and the number of newspaper articles about him, analog to the city population
example. The more the media report on a tennis player, the higher is the chance to be recognized. Thus,
at least moderate recognition validities should be found, and people should infer that recognized tennis
players win against unrecognized players. Moreover, predictions of English soccer matches by foreign
(Turkish) students were nearly as accurate as the predictions by domestic (English) students (Ayton & ¨ nkal,
1997). This is often interpreted in favor of the recognition heuristic. However, the students in this study seem
to use a familiarity cue rather than a recognition cue. Familiarity (Griggs & Cox, 1982) as well as availability
(Tversky & Kahneman, 1973), however, are graded distinctions among recognized items, and are therefore
more elaborated than the binary recognition heuristic (for a more detailed comparison between predictions
based on binary recognition information and predictions based on graded fluency of retrieval see Schooler &
Hertwig, 2005).
On the other hand, it seems questionable whether the recognition heuristic will work in a dynamic envir-
onment like tennis, because recognition may hardly be flexible enough: It takes time to become a reliably
successful tennis player who appears regularly in the media. Even then it might take time until the chance of
being recognized by laypeople becomes high. Thus, currently successful players might stay unrecognized
until they are established. Moreover, a successful tennis player who is recognized once might stay recog-
nized over a long period of time even if his success declines. To cope with declining strength systematic
forgetting is needed. Schooler and Hertwig (2005) demonstrate the benefits of forgetting for the recognition
heuristic. However, systematic rather than unsystematic forgetting is needed. These factors might impair
performance of the recognition heuristic.
To test the performance of the recognition heuristic in a domain with a dynamic criterion we predicted the
results of the most prominent tennis tournament of the world, the Lawn Tennis Championships at Wimble-
don. The aim of the study was twofold. Study 1 analyzes the performance of the recognition heuristic and
compared it to predictions of two official rankings and the betting market. We collected recognition data of
two groups with different experience in the domain of tennis (amateurs and laypeople). To measure the per-
formance of the recognition heuristic, an aggregated recognition ranking and the individual recognition data
was used to simulate predictions that would have been made if the recognition heuristic had been used con-
sistently. In Study 2 the descriptive value of the recognition heuristic was analyzed. We again collected
recognition data of amateurs and laypeople and assessed whether they actually used the recognition heuristic
for predicting tennis matches of the round of sixteen and quarter-final matches in Wimbledon 2003.
Twenty-nine tennis players from Neukirchen-Vluyn (amateurs, AM) and 96 students from Jena who did not
play tennis (laypeople, LAY) were surveyed. The amateurs (20 men, 9 women) had an average age of
M ¼ 46.5 years (range from 14 to 72 years), and had been playing tennis for M ¼ 17.2 years (range from
1 to 50 years). The laypeople (12 men, 84 women) had an average age of M ¼ 21.8 years (range from 19
Copyright # 2006 John Wiley & Sons, Ltd.
Journal of Behavioral Decision Making, 19, 321–332 (2006)
S. Serwe and C. Frings
The Recognition Heuristic 325
to 30). The amateurs watched more tennis than the laypeople (M AM ¼ 3.69, M LAY ¼ 1.80, rated on a scale
from 1 to 5, where ‘‘1’’ indicates ‘‘never’’ and ‘‘5’’ means ‘‘very often’’). Data of 3 laypeople were excluded
because they did not recognize a single player.
Participants were surveyed just a few days before the first match of Wimbledon 2003. They had to decide
whether they recognized the names (‘‘heard of before’’) of professional male tennis players or not. The list
comprised the 112 players who started in the Wimbledon Men’s single tournament 2003 without qualifica-
tion (another 16 starters had to qualify first). First, two recognition rankings were calculated by counting the
number of amateurs (REC-AM) and laypeople (REC-LAY) who recognized each player. These recognition
rankings were compared with each other and with two official rankings: the ATP Champions Race (ATP-CR)
and the ATP Entry Ranking (ATP-ER). The ATP-CR is the official worldwide ranking of male tennis players
for one calendar year. The ATP-ER is another official worldwide ranking procedure and results from success
in the last 52 weeks (with some minor exceptions, for more details on both rankings see Association of
Tennis Professionals (ATP), 2003). All rankings were used to predict the outcomes of each match played
in Wimbledon Men’s single tournament 2003 with an easy decision rule: ‘‘The player with the better rank
will win’’.
It is quite unusual to use recognition rankings to evaluate the performance of the recognition heuristic since
the recognition heuristic originally deals with individual decision behavior and its outcomes. Thus, for each
tennis match 1 played in Wimbledon 2003 a prediction by each participant was simulated using the individual
recognition data. Consistent with the rules of the recognition heuristic, if one out of two players was recog-
nized, this player was predicted to win. Each match with either both players recognized or both players not
recognized was ignored since no prediction can be derived from the recognition heuristic. If the recognition
heuristic was applied, two further predictions were simulated for the same match, namely predictions relying
on the actual rank of both players in the two official ranking procedures ATP-CR and ATP-ER. Again ‘‘The
player with the better rank will win’’ was used as simple decision rule for ATP-CR and ATP-ER. Moreover, we
also included another prediction relying on the betting market (BET) and we used as a decision rule for each
single match ‘‘The player with the lower odds will win’’. However, note that BET in contrast to the other pre-
dictors does not evaluate the absolute strength of a player but his chance to win against a given opponent in a
specific match. Additionally, BET has another advantage: It can change its evaluation of a player in the course
of the tournament. It is exactly for these reasons that it could be questioned whether BET is a comparable (and
fair) competitor. 2 Evaluation about the performance of the recognition heuristic is therefore focused on the
comparison between the recognition heuristic and the ATP rankings.
Unless otherwise noted, all effects referred to as statistically significant throughout the text are associated
with p values less than 0.05, two-tailed.
Since for practical reasons no data was collected for qualificants, here and in the following text each match does mean ‘‘each match for
which data was collected’’ (96 out of 127 matches played at all).
We thank an anonymous reviewer for drawing our attention to this point. Please note that it could principally be argued, that adding
BET to the analysis destroys comparability and therefore BET should be excluded from analysis. However, we think that the comparison
to the betting market predictions is despite this problem a useful information: Even if it is unfair to compare the recognition heuristic or
ATP-rankings with BET (from a theoretical point of view), BET is practically one possible cue in a real world scenario. This cue can be
discovered as easy as the ATP rankings and analog to our simulation, the betting odds for a player change during the Wimbledon
tournament whereas the ATP rankings are updated afterwards and thus stay constant. Thus, in the real world, using betting odds can
prove to be a fast and frugal strategy.
Copyright # 2006 John Wiley & Sons, Ltd.
Journal of Behavioral Decision Making, 19, 321–332 (2006)
Zgłoś jeśli naruszono regulamin