Is it science?
Carl Sagan once noted that the best antidote to pseudoscience is real science, and it’s certainly a fact that the findings of robust scientific studies can be far more shocking and difficult to believe than those that have undeservedly received a mainstream ‘common sense’ welcome. I have written previously about one prominent neuromyth – that of ‘learning styles’ – but I’ve recently been prompted to reflect in more detail on the moment that every chessplayer experiences with every move they make – that moment when our neurons shift from a period of invisible and infinitely reflexive discussion (however brief or lengthy) to the decisive and irreversible instruction to one of our limbs actually to take hold of a piece and move it to a new square. (We’ll ignore for now those embarrassingly betwixt-and-between episodes of hand-hovering or reach-and-withdraw, which reveal too much about either our state of mental confusion or our tendency towards egregious gamesmanship.) How reliable is that instinct? What drives that moment, whether it’s a quixotic impulse or, channelling W.B. Yeats, that more lumbering cousin: that rough beast, its hour come round at last, slouching towards Bethlehem to be born? Is that moment founded on a rational process, or is it affected by a pseudoscientific belief that in reality might be doing more harm to our game than good? And if it’s the latter, can we eliminate it from our repertoire of chessplaying habits?
Reasonable parameters
Since I lack both the space and the chess strength to attempt to say anything useful about the process of evaluating candidate moves (Mikhail Shereshevsky, Erik Kislik, Mark Dvoretsky and Artur Yusupov are just some of the more credible chessplayers and trainers who have written authoritatively in this area), I’d like to limit this blogpost to some reflections inspired by the person who first got us thinking about this area at all. I am referring of course to Alexander Kotov’s highly influential book from the early 70s, Think Like a Grandmaster. In introducing his notion of the tree of analysis, Kotov recalls an episode where he is lecturing to a roomful of strong chessplayers. He asks them, ‘Do you know how to analyse variations?’ and proceeds to outline a scenario in which a player spends 30 minutes deliberating about the relative merits of two moves before being suddenly struck by the possibility of a third and, alarmed by the prospect of impending time trouble, “… without any more ado, without any analysis …”, plays it. “Just like that, with hardly any consideration at all”. Kotov recalls that he drew a round of spontaneous applause and laughter from his audience – a round that stand-up comedians know to derive from recognition: “So accurate was my picture of their trials and tribulations” (pp. 15-16).
Kotov is clearly using this incident as an illustration for his broader argument, developed in the chapters that follow, that we need to base our decisions on rational and considered processes rather than whim, ‘intuition’ or impulse. But there is a risk that the reader might infer an unintended meaning – that the moves first considered are more likely to be ‘correct’, and that the Johnny-come-lately idea should be put firmly in its box. And it is this notion, specifically and more generally, that I’d like to commit to a short scientific interrogation.
The availability heuristic
Consider the possible future course of the scenario described by Kotov. What if that late-arriving candidate-cum-actual move turns out to be a mistake, and the game turns on it? It is possible that the player will lambast herself for her impulsivity, and console herself with the belief that if only she’d stuck to her initial hunch and played one of the moves she’d invested a half-hour in analysing, the game might have turned out very differently. But it is certain that she’d remember that moment as a critical point in the game, because the pain of poor decisions lingers longer than the memories of good ones (cf the concept of loss aversion). This plays into a phenomenon known as the availability heuristic, where we estimate the likelihood of something happening using the metric of how readily it springs to our mind. So just as students who change correct answers in exams to incorrect ones recall those decisions far better than students who change incorrect answers to correct ones (Kruger et al, 2005), the availability heuristic suggests that as chessplayers we’re far more likely to remember those moments when we spurned our initial hunches in favour of later-arriving ones and went on to regret the change, rather than those happier occasions when we decided against our early candidates in favour of a late-arriving idea, and went on to win.
Trust-your-first-hunch is a fallacy
Of course, the availability heuristic might explain why we recall bad decisions more readily than good ones, but it doesn’t say anything about the objective merits of initial hunches relative to later ideas. But the wider scientific literature does, and two rather surprising conclusions emerge: 1. There is a pervasive piece of test-taking folklore that suggests you should stick with your original answer, especially if you’re in some doubt about it. 2. This advice, and the people who offer it, are wrong.
The ‘first-instinct fallacy’ is just that, a fallacy, and Lilienfeld et al even include it as one of their 50 Great Myths of Popular Psychology. They cite an impressive literature base, with over 60 studies revealing that when students make changes to their multiple choice test answers they’re more likely to change from a wrong to a right answer than the other way around. And for every point lost when changing from a right to a wrong answer, they gain from two to three points on average in changing from a wrong to a right answer. Moreover, students who change more answers tend to receive higher test scores (though these findings are correlational rather than causative and may reflect the fact that those who change their answers a lot could be more invested in the test and possibly higher performers to begin with).
Why should chessplayers be much different from students taking multiple choice examination papers? After all, a chess game is the ultimate multiple choice test on every move and though not every move can be evaluated as ‘correct’ or ‘incorrect’ in some neat multiple-choice binary, in very many instances on the board decisions are certainly more good than bad, or vice versa. Bilalic et al have investigated and discussed the so-called Einstellung (set) effect on chessplayers and others in some detail: this effect describes a phenomenon in which the first idea that comes to mind – triggered by familiar features of a problem (pattern recognition) – prevents a better solution being found. This risk is of course intuitively captured in Emanuel Lasker’s famous advice to sit on your hands when you find a good move, and seek a better one. So should we be routinely discounting the trust-your-first-hunch imperative given the evidence that we’re more likely to benefit from than be punished by a late-change? Probably, but with some reservations.
A couple of caveats
There are at least two important qualifications to the ‘when in doubt, change your answer’ strategy. The first relates to the evidence that we shouldn’t change our minds if we’re just guessing that an answer might be wrong. We should have a good reason to suspect that our answer (or intended move) is a mistake (Shatz & Best, 1987). This should make both intuitive and rational sense to we chessplayers: there is no substitute for the hard grind of checking our analyses for veracity, and if our calculations suggest a line probably doesn’t work, we should be looking for better alternatives. The second caveat relates to the evidence that students who routinely do poorly on multiple choice tests may benefit less from changing their answers than other students (Best, 1979). So if the extension to the chess domain holds, we amateur players at least should probably change our answers only when we’re fairly sure that we’d be making a poor move by acting on our initial hunch. Perhaps titled players deserve greater freedom to play fast and loose with their first and later ideas? It’s an hypothesis that might merit empirical examination.
Last thoughts, before pressing ‘send’
If the overall message of this post is to have the confidence to put two fingers up to the dominant orthodoxy of ‘trust your first instinct’ – at least when your analyses justify any reservations you might have about it/them – it might be worth returning to Kotov’s original story: that third, late-arriving idea might well be an error, or it may be an improvement over the first two candidate moves. Even in the absence of a definitive judgment we’d do well to trust our processes: if that half hour spent considering the first two moves revealed their flaws, we’d do well to consider an alternative. But that third move isn’t necessarily an improvement simply because it’s ‘none of the above’ – it deserves the same rigorous scrutiny. But ideas arriving late often do so for a reason – they’re the fruits of a deep strategic and tactical immersion, where the demands and possibilities reveal themselves only slowly, in a tantalising positional burlesque.
The last words go to those ravagers of common-sense pseudoscience, Lilienfeld et al, p.80:
“When in doubt, we’re usually best not trusting our instincts. After all, our first hunches are just that – hunches. If we have a good reason to believe we’re wrong, we should go with our head, not our gut, and turn that pencil upside-down.”
Contact: [email protected]
References
Best, J.B. (1979); Item difficulty and answer changing. Teaching of Psychology, 8, 228-230
Bilalic´, M., McLeod, P. & Gobet, F. (2008); Inflexibility of experts—Reality or myth? Quantifying the Einstellung effect in chess masters. Cognitive Psychology, 56, 73–102
Kotov, A. (1971); Think Like a Grandmaster. London: Batsford
Kruger, J., Wirtz, D. & Miller, D. (2005); Counter-factual thinking and the first-instinct fallacy. Journal of Personality and Social Psychology, 88, 725-735
Lilienfeld, S.O., Lynn, S.J., Ruscio, J. & Beyerstein, B.L. (2010); 50 Great Myths of Popular Psychology – Shattering widespread misconceptions about human behavior. Chichester UK: John Wiley & Sons
Shatz, M.A. & Best, J.B. (1987); Students’ reasons for changing answers on objective tests. Teaching of Psychology, 14, 241-242
