This article appeared in Mind & Language, v. 10, no. 4, 1995, 437-445.

Archived at Website for the Rutgers University Research Group on Evolution and Higher Cognition.


Choice Effects and the Ineffectiveness of Simulation

Shaun Nichols
Department of Philosophy
College of Charleston
Charleston, SC 29424
nichols@cofc.edu

Stephen Stich
Department of Philosophy and Center for Cognitive Science
Rutgers University
New Brunswick, NJ 08901
stich@ruccs.rutgers.edu

and

Alan Leslie
Department of Psychology &
Center for Cognitive Science
Rutgers University
New Brunswick, NJ 08903


Contents

  1. Introduction
  2. Actor Effects and Observer Noneffects
  3. Further Evidence against Simulation Theory
  4. Conclusion



Abstract

Kuhberger et al. show that producing the Langer effect is considerably more difficult than has been assumed. Although their results clearly demonstrate a need for further exploration of the Langer effect, none of their arguments undermines the evidence against simulation theory that we presented in Nichols et al. (1995). In our study, the actor subjects did show an effect, but the prediction subjects did not predict it, despite the fact that they were provided with all the details of the actor's situation. Further, we report additional evidence that corroborates our empirical case against simulation theory.


1. Introduction

The paper by Kuhberger, Perner, Schulte, and Leingruber is a welcome addition to what we hope will be a growing body of empirical work concerned with simulation theory. In attempting to replicate Langer's results and ours they produced results that are both surprising and interesting. These results raise the important possibility that there is more going on in Langer's lottery situation than has been suggested in the literature. Following Langer, we had assumed that the group difference in selling prices was a fairly straightforward result of subjects either having or not having a choice of tickets. Kuhberger et al. show that this assumption is problematic. Although their results clearly demonstrate a need for further exploration of the Langer effect, their findings don't threaten our argument against simulation theory. In section 2 we will argue that none of the arguments offered by Kuhberger et al. undermines the evidence against simulation theory that we presented in Nichols et al. (1995). In section 3 we will present some additional empirical evidence that corroborates our case against simulation theory. We conclude that current empirical evidence still weighs heavily against simulation.

2. Actor Effects and Observer Noneffects

Kuhberger et al. offer several different lines of argument against the evidence in Nichols et al. We'll consider each of these arguments separately.

2.1. Replicability

Kuhberger et al. suggest that their failure at replicating the effect might indicate that there really is no Langer effect. That would, of course, undermine our argument in Nichols et al. If there really is no Langer effect, then the fact that our observer subjects failed to make predictions conforming to the effect can't count as evidence against simulation. However, as we'll explain, Kuhberger et al. don't have much of an argument against the existence of the Langer effect.

In considering the possibility that the Langer effect may not exist, Kuhberger et al. suggest that the reports of the effect in the literature may merely represent a reporting bias. They write, 'The two positive reports in the literature (Langer, 1975, and Nichols et al., 1995) may reflect a reporting bias of positive findings in a sea of unreported failures to replicate.' (Section 6.1). It is, of course, possible that there's a reporting bias; it's possible that there are several unpublished failures to replicate the Langer effect. But it seems just as likely that there have been several unpublished successful replications of the Langer effect. One might even argue that an experimentally sound failure to replicate Langer's effect would be more worthy of publication than a successful replication. Witness Kuhberger et al.

As far as we know, Kuhberger et al. are right that there are no published studies that are exact replications of Langer's study 2. However, if we view the Langer effect as part of the broader phenomenon of 'illusion of control', then there have been many replications and extensions (e.g., Langer & Roth, 1975; Ayeroff & Abelson, 1976; Bouts & Van Avermaet, 1992). Moreover, there are other studies in which subjects' perceived control in chance situations is affected by a choice/no-choice variable. For instance, experiment 3 in Langer (1975) is explicitly presented as a replication of the effects of choice on perceived control, and Wortman (1975) independently produced similar results. Wortman told subjects that a red marble and a blue marble would be used to represent two different prizes, one of which was attractive to the subject and one of which was unattractive. She told the subjects which marble corresponded to which prize and had subjects put the marbles into a coffee can. In the no-choice condition, the experimenter reached into the can without looking and picked one of the marbles for the subject. In the choice condition, the subject was allowed to reach into the can without looking and pick one of the marbles. Subjects in the choice condition felt that they could influence the color of the marble they received significantly more than did subjects in the no-choice condition. There is, then, at least some confirmation that the perception of control in chance situations is affected by the choice/no-choice variable.

If there really is a Langer effect, though, we might wonder why Kuhberger et al. couldn't produce it. The experiments in Kuhberger et al. differ from our experiment and from Langer's experiments in many potentially significant ways. Perhaps the most obvious differences that they fail to consider are differences in the subject populations. One particularly salient difference between the subjects in Kuhberger et al. and the subjects in both Langer and Nichols et al. is cultural. Kuhberger et al.'s subjects were presumably Austrian, and the subjects in Langer and our experiments were American. It's entirely possible that some of the cultural differences between the US and Austria are relevant to producing the Langer effect. Although it doesn't address the illusion of control directly, there is some evidence that Americans are more likely than members of other cultures to attribute responsibility for outcomes to individuals rather than to situations (e.g., Miller, 1984).

In light of these considerations, surely it's hasty to conclude on the basis of Kuhberger et al. that the Langer effect doesn't exist. Kuhberger et al. do show that there is more involved in obtaining the Langer effect than was previously thought. Indeed, the bulk of Kuhberger et al. is aimed at showing that it's difficult to produce the Langer effect. But it must be emphasized that this in itself is no evidence against our argument. Our argument against simulation in no way depends upon the Langer effect being easily replicated or cross-culturally uniform. For however things turn out in the first part of the experiment, simulation theory predicts that someone watching the videotape of that part will correctly predict (simulate) the outcome, whatever that outcome was. Since the subjects in our Langer replication did show an effect, and the observer subjects failed to predict that effect, this counts as evidence against simulation regardless of the difficulty and complexity involved in replicating the Langer effect.

2.2. Critical factors

The possibility that the Langer effect doesn't exist isn't the only argument that Kuhberger et al. offer against us. Indeed, they concede that there probably is a Langer effect. But they maintain that there is still a problem for us, since it's not clear which factors are critical in producing the effect. Kuhberger et al. suggest several critical factors that may have contributed to the effect. They suggest that the tendency to rely on anchors may be different for choice and no-choice conditions, the level of personal interaction may affect selling prices, and the nature of the filler task may affect the personal value attached to the ticket. They then argue that given the apparent complexity of the critical factors, it is hard to know whether the right combination of factors is presented to the observer subjects.

We agree with Kuhberger et al. that the Langer effect may depend on some of the critical factors they suggest. And this might be quite important in directing future work in social psychology. However, we don't see how it is relevant to the issue over simulation. It is, of course, crucial to our case that the observer subjects are presented with the critical factors. However, in our experiment, all of the critical factors suggested by Kuhberger et al. were present in the videotape, so the observers presumably saw all of the relevant information.

Perhaps a bit more detail about the observer condition is in order. The observer subjects were shown a videotape that included the distribution of the tickets, the filler task, and the asking for a selling price (which the actor subject noted on his filler task answer sheet). The observer subjects who saw the tape were provided with the same answer sheets that were used in the Langer-effect replication, and they were asked to predict both the answers that the (actor) subject would give to the grammar questions and the buy back price that he noted on his answer sheet. In short, the subjects saw all the details of the situation on the videotape. As a result, whatever subtle features of the situation triggered the difference in selling price, those features were presumably there for the observer subjects to see. Kuhberger et al. might claim that there is some unknown critical factor that wasn't presented to the observer subjects, but that seems altogether ad hoc.

Perhaps Kuhberger et al. want to suggest that because the Langer effect may depend on a subtle combination of factors, we can't be sure that the observer subjects noticed all of the critical features. This would mean that our results don't threaten simulation since our experimental design doesn't ensure 'imaginative adequacy'. However, it's imperative to keep in mind that simulation only needs to be tested under approximately typical everyday conditions. The everyday conditions under which people typically predict behavior are far from ideal, and yet these predictions are generally successful. That's what simulation theory has to account for. In other words, we know that people are generally very good at predicting other people's behavior, even when the cues they are using are very subtle and they are not consciously aware of what cues they are using. Consider, for example, the cues we use in predicting whether a driver will let us merge in traffic, or the cues we use in predicting whether someone wants to end a conversation. In our experiment, we provided conditions that were at least as good as such typical everyday conditions, so there can be no serious worry that our experiment didn't ensure imaginative adequacy.

2.3. Individual differences

Kuhberger et al. seem to think that the problem posed to us by the potential complexity of the critical factors is exacerbated by the fact that there are apparently great individual differences in selling prices. They write,

Since not every target participant's situation may have contained the critical combination of factors and since (due to the large individual differences) not every simulator participant is one who would show the effect even when confronted with the right combination of factors, it becomes very difficult to make sure that enough of the right simulators are shown the right combination of factors (6.3).

The first thing to note is that the problem that critical factors are supposed to pose is entirely independent from any problem emerging from individual differences in selling prices. Since in each condition all of the observer subjects saw the same videotape, the same information was available to them all. It's a completely different question whether there are 'enough of the right simulators' in the observer study.1

We've already explained why it's implausible to suppose that the observer subjects weren't shown the right combination of factors. To suggest that successful simulation requires more than the information that was available to our observer subjects is to admit that simulation is a marginal ability that would fail in most real life situations. On the quite different point concerning how we ensure that a sample of observers accurately reflects the range of individual differences that may be expected in a sample of actors, we have to say simply that from a statistical point of view, this is a bizarre criticism. The observer subjects in our experiment were drawn randomly from the same pool as our actor subjects. We are therefore entitled to make the standard assumption that the two groups will have the same statistical properties within the margins of sampling error.

2.4. Behavioral Distractions

The final argument against Nichols et al. is that it may in fact be the very thoroughness of our videotape that undermines our evidence against simulation. Kuhberger et al. suggest that by actually showing the target person on the videotape, we may have suppressed the use of simulation in our observer subjects. The idea is that the presence of the target person may have made behavioral cues more prominent, leading subjects to make their predictions on the basis of the behavioral cues rather than on the basis of simulation.

This argument is seriously problematic in a couple of ways. First, there is no more reason to think that behavioral cues would distract a subject from using simulation than there is to think that behavioral cues would prompt a subject to use simulation. Simulation theory is entirely consistent with either possibility. As a result, the suggestion that behavioral cues suppress simulation is thoroughly unconvincing. But even without this difficulty simulation theorists should not welcome the suggestion that behavioral cues suppress the use of simulation. For a huge portion of our everyday prediction of behavior occurs in situations in which we can see the person whose behavior we're trying to predict. As a result, if these cases suppress the use of simulation, then a good chunk of our folk psychological prediction, and, crucially, a good chunk of successful folk psychological prediction, doesn't depend on simulation. Again, this argument marginalizes simulation. Since the only live alternative to how we predict behavior is the theory theory, it would follow that a large portion of successful folk psychology depends on theory. Furthermore, if we have enough folk psychological theory to successfully predict behavior in the presence of the target, then presumably we have enough theory to predict behavior in the absence of the target.

Although the studies reported by Kuhberger et al. raise a number of interesting questions about the Langer effect, none of these questions pose a serious threat to the argument that we offered in Nichols et al. As a result, we would still maintain that the results of Nichols et al. count heavily against simulation theory. Further, as we'll explain in the next section, there is now additional evidence that further undermines simulation theory.

3. Further Evidence against Simulation Theory

After completing Nichols et al., we discovered some evidence that points in the same direction as the evidence we reported. The experiment reported in Nichols et al. focused on the prediction of other people's behavior, when that behavior is strongly influenced by an 'effect' whose existence comes as a surprise to most people. In another recent study, Loewenstein and Adler (1995) looked at the ability of subjects to predict their own behavior, when that behavior is influenced by a little known effect. What they found was that, in this case too, people make the wrong predictions. Ironically, the effect that Loewenstein & Adler exploit is one that Kuhberger et al. suggest as an important factor in the lottery experiments -- the 'endowment effect' (Thaler, 1980). The endowment effect is a robust and rapidly appearing tendency for people to set a significantly higher value for an object if they actually own it than they would if they do not own it. Here is how Loewenstein & Adler describe the phenomenon.

In the typical demonstration of the endowment effect (see, e.g., Kahneman, Knetsch and Thaler 1990), one group of subjects (sellers) are endowed with an object and are given the option of trading it for various amounts of cash; another group (choosers) are not given the object but are given a series of choices between getting the object or getting various amounts of cash. Although the objective wealth position of the two groups is identical, as are the choices they face, endowed subjects hold out for significantly more money than those who are not endowed (Loewenstein and Adler, 1995, pp. 929-930).

In an experiment designed to test whether 'unendowed' subjects could predict the value they would set if they were actually to own the object in question, the experimenter first allowed subjects (who were members of a university class) to examine a mug engraved with the school logo. A form was then distributed to approximately half of the subjects, chosen at random, on which they were asked 'to imagine that they possessed the mug on display and to predict whether they would be willing to exchange the mug for various amounts of money.'2 When all the subjects who received the form were finished filling it out, all the subjects were presented with a mug and given a second form with instructions analogous to those on the prediction form. But on the second form it was made clear that they actually could exchange the mug for cash, and that the choices they made on this second form would determine how much money they might get. 'Subjects were told that they would receive the option that they had circled on one of the lines -- which line had been determined in advance by the experimenter.' (Loewenstein and Adler, 1995, p. 931). The results showed that subjects who had completed the first form substantially underpredicted the amount of money for which they would be willing to exchange the mug. In one group of subjects, the mean predicted exchange price was $3.73, while the mean actual exchange price for subjects (the same ones who made the prediction) was $5.40! There did indeed seem to be an 'anchoring effect' in this experiment, but it seems to have depressed the actual exchange price: the mean actual exchange price for subjects who did not make a prediction about their own selling price was even higher at $6.46.

In this experiment, as in the Nichols et al. experiment, it is hard to see how an advocate of simulation theory can offer a principled explanation of the results. Obviously, the mistaken prediction cannot be the result of a difference between the psychological mechanisms of the predictor and the target since in this experiment the subjects are predicting their own decisions. Nor is it plausible for the simulation theorist to suppose that the Pretend Belief and Desire Generator would have a problem generating the pretend beliefs and desires that result from being told that one now actually owns the mug. For if the pretense mechanism that the simulation theorist posits can't handle simple cases like pretending that one has been given a mug one has had ample opportunity to examine, then the pretense mechanism must be very fallible indeed. And if that is the case, then it is a mystery why our predictions turn out to be correct as often as they do. For the theory theorist, by contrast, there is no puzzle about how the results are to be explained. Folk psychology just doesn't include any information about the endowment effect. So when the effect plays a major role in determining behavior, predictions based on folk psychology get it wrong.

4. Conclusion

We've argued that Kuhberger et al. don't undermine the evidence against simulation presented in Nichols et al. The crucial feature of our experiment was that our replication subjects showed an effect and our observer subjects failed to make predictions that conformed to the effect. The fact that it's difficult to produce the Langer effect may undermine prevailing interpretations of the Langer effect, but it doesn't undermine our case against simulation. Further, there is now additional empirical evidence that the prediction of behavior doesn't derive from simulation. As a result, the empirical case against simulation is actually stronger now than it was when we completed Nichols et al.

Although we find Kuhberger et al.'s arguments against us unconvincing, there is a more fundamental point of greater importance on which we evidently agree. Like Kuhberger et al., we think that much of the work in deciding between simulation theory and theory theory needs to be done experimentally. We hope that future empirical work on simulation will be carried out with the seriousness and ingenuity of Kuhberger et al.



References

Ayeroff, F. & Abelson, R. 1976: ESP and ESB: Belief in Personal Success at Mental Telepathy. Journal of Personality and Social Psychology, 34, 240-247.

Bouts, P. & Van Avermaet, E. 1992: Drawing Familiar or Unfamiliar Cards: Stimulus Familiarity, Chance Orientation, and the Illusion of Control. Personality & Social Psychology Bulletin, 18, 331-335.

Kahneman, D., Knetsch, J., and Thaler, R. 1990: Experimental Tests of the Endowment Effect and the Coase Theorem. Journal of Political Economy, 98, 1325-1348.

Langer, E. 1975: The Illusion of Control. Journal of Personality and Social Psychology, 32, 311-328.

Langer, E. & Roth, J. 1975: Heads I Win, Tails It's Chance: The Illusion of Control as a Function of the Sequence of Outcomes in a Purely Chance Task. Journal of Personality and Social Psychology, 32, 951-955.

Loewenstein, G. and Adler, D. 1995: A Bias in the Prediction of Tastes. The Economic Journal: The Quarterly Journal of the Royal Economic Society, 105, 929-937.

Miller, J. 1984: Culture and the Development of Everyday Social Explanation. Journal of Personality and Social Psychology, 46, 961-978.

Nichols, S., Stich, S., Leslie A., and Klein, D. 1995: Varieties of Off-Line Simulation. In P. Carruthers and P. Smith (eds.), Theories of Theories of Mind. Cambridge: Cambridge University Press, 39-74.

Thaler, R. 1980: Toward a Positive Theory of Consumer Choice. Journal of Economic Behavior and Organization, 1, 39-60.

Wortman, C. 1975: Some Determinants of Perceived Control. Journal of Personality and Social Psychology, 31, 282-294.



NOTES


Address for correspondence: Shaun Nichols, Department of Philosophy, College of Charleston, Charleston, SC 29424.

1. We use the terms 'observer subjects' or 'predictors' rather than Kuhberger et al.'s 'simulators', since that term suggests that the subjects are using simulation. And that is the very point at issue.

2. Loewenstein and Adler, 1995, p. 931. The exact wording of the form was as follows:

We are interested in your opinion about the mug displayed at the front of the room. Imagine that we gave you a mug exactly like the one you can see, and that we gave you the opportunity to keep it or trade it for some money. Below are a series of lines marked "Keep mug___ Trade it for $amount___." On each line check whether you would think that you would prefer to keep the mug or to trade it in for the amount of money written on the line. Check one or the other on every line.

The remainder of the page consisted of 40 lines in which the amount of money for which the mug might be traded increased from 25 cents to 10 dollars, in 25 cent increments.