PSYC0146 Coursework 2020-2021 Background Since the publication of Kahneman’s Thinking, fast and slow (Kahneman, 2011), main ideas in so-called “dual systems” or dual-process" theories have entered public discourse on psychology and decision making. In his bestseller, Kahneman proposes a distinction between two modes of thought, “System 1” being fast, instinctive and emotional, and “System 2” slower, more deliberative, and more logical. This distinction is not Kahneman’s sole invention; dual systems and dual process theories have a long history in psychology (e.g. Evans and Stanovich, 2013; Sloman, 1996; Stanovich and West, 1999), although the usefulness of such dualisms is not universally accepted (e.g. Keren and Schul, 2009; Shanks and St. John, 1994). Table 1 lists some of the purported characteristics of the two systems. Table 1: General characterisctics of System 1 and System 2 pro- cesses. System 1 System 2 unconscious conscious implicit explicit automatic controlled rapid slow low effort high effort large capacity small capacity default process inhibitory associative rule-based contextualized abstract domain specific domain general evolutionary old evolutionary recent nonverbal verbal independent of working memory limited by working memory non-logical logical parallel serial Biases in judgement and reasoning are often attributed to System 1 thought processes. Perhaps if people slowed down and allowed System 2 processes to guide their reasoning and judgements, cognitive biases could be avoided. This idea has gained some traction, and advice is often given to slow down, reflect, and base decisions on rational deliberation rather than intuition. Although seemingly plausible, there is not all that much experimental evidence to show that such advice is actually beneficial. One could also argue that speeding up is sound advice, as unconscious and intuitive reasoning may be highly accurate as well (e.g. Gladwell, 2005). Consider the well known “Linda problem” (Tversky and Kahneman, 1983), which goes as follows: Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? 1. Linda is a bank teller. 2. Linda is a bank teller and is active in the feminist movement. 1 When asked this question, the majority of people tends to choose answer 2. However, according to the rules of probability, the probability of a conjunction (e.g., bank teller and feminist) can never be larger than the probability of one of it constituents (e.g., a bank teller): P (A and B) ≤ P (A). The normatively incorrect answer is ascribed to the “representativeness heuristic”, as the description of Linda above seems representative of someone active in the feminist movement. The representativeness heuristic is assumed to be an example of a System 1 type thought process. Whilst reasoning on the basis of representativeness and similarity often provides a quick and accurate answer, there are clearly cases where it fails. The question is whether slowing down would help people to give the correct answer. If people don’t know the rules of probability, or otherwise have access to the conjunction rule, would they be able to work out the correct answer through slow and deliberative reasoning? And what if they have somehow acquired the wrong rule for solving these problems? Might relying on System 2 processes then not make them more likely to fail in providing the correct answer? Whether “slowing down” or “speeding up” is beneficial may thus rely on a variety of factors. Firstly, some people may be more naturally inclined to rely on reflection and deliberation (i.e. System 2) than others. Secondly, some people may have access to the correct rules to solve particular judgement and decision problems, whilst others don’t. Finally, some of the problems seem to require a reasonable level of statistical numeracy. Individual differences in this ability may also affect to what extent slowing down or speeding up is beneficial. Experiment An experiment was conducted to determine whether it is possible to improve people’s reasoning by asking them to think slow or fast, and the role in this of individual differences in reflection, numerical competence, and access to abstract rules. The experiment was conducted in two separate sessions (completed at least 24 hours apart). In the first session, participants completed a number of tasks aimed to measure their ability for cognitive reflection, numerical reasoning, and the application of abstract rules. In the second session, participants were randomly assigned to one of five conditions, with instructions on how they should solve the subsequent judgement problems (e.g. fast or slow). They then completed a set of 12 judgement tasks covering a range of classic problems where cognitive biases have found. The five conditions were: ? Control: Participants completed the judgement tasks at their own pace, without further instruction. ? Slow: Participants were instructed to provide their answers after extensively reflecting on each problem. ? Fast: Participants were instructed to provide the first intuitive answer that came to their mind. ? Fast-Slow: Participants were asked to provide two answers. First an initial intuitive answer. They were instructed to then extensively reflect on the problem and their initial answer, before submitting their final answer. The data contains the final answers only. ? Incentive: In this condition, participants were not instructed to answer in a particular way. However, they were told they would be able to receive a monetary bonus payment for correct answers. Incentives are often found to increase motivation and performance. This condition is then also a useful control to assess the effectiveness of instructions to “slow down” or “speed up”. In the data you have been assigned, there are n = 200 participants in each condition. Tasks Cognitive reflection To measure participants’ natural inclination for cognitive reflection, they completed the Cognitive Reflection Test (Frederick, 2005) as well as an alternative version which was designed to be less dependent on numerical ability (Thomson and Oppenheimer, 2016). The original Cognitive Reflection Test consists of three questions such as A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? _____ cents. The alternative version consists of four questions. 2 Numerical ability The Berlin Numeracy Test (Cokely et al., 2012) was used to measure participants’ statistical numeracy. It consists of four questions such as Imagine we are throwing a five-sided die 50 times. On average, out of these 50 throws how many times would this five-sided die show an odd number (1, 3 or 5)? _____ out of 50 throws. Application of rules To assess whether participants would be able to solve the judgement problems in principle (i.e. they have access to the abstract rules to solve them), problems such as the Linda problem were stated in alternative “decontextualized” ways. For instance, the Linda problem might be stated as Imagine John owns a laptop. Rank the following from most likely to least likely: 1. The laptop has a 14 inch screen. 2. The operating system is Linux. 3. The laptop has a 14 inch screen and the operating system is Linux. There were two such decontextualized versions of each of the six classic judgement problems in the next section. Judgement tasks Participants were presented with two versions of each of six classic judgement problems. These were: ? Conjunction problem (Tversky and Kahneman, 1983). This problem taps into people’s tendency to ignore that the probability of a conjunction can never be larger than the probability of one if its constituent events. The Linda Problem is an example. ? Probability maximising problem (Stanovich and West, 2008). If one outcome has a slightly higher probability than another, to maximise your probability of winning, you should always bet on the option with the highest probability (maximising) rather than also sometimes betting on the option with the lower probability (probability matching). ? Omission problem (Ritov and Baron, 1990). This problem taps into people’s tendency to choose a default option, rather than an alternative option which is superior. ? Base rate problem (Kahneman and Tversky, 1973). This problem taps into people’s tendency to ignore the base rate (e.g. probability of a disease) when judging posterior probabilities (e.g. the probability of a disease after a positive test result). ? Denominator neglect problem (Kirkpatrick and Epstein, 1992). This problem taps into people’s tendency to choose based on absolute, rather than relative frequencies. For instance, one bowl has 4 out of 25 “wins”, and another 2 out of 10 “wins”, people often choose the first, even though the probability of winning is larger in the second option. ? Covariation problem (Wasserman et al., 1990). This problem taps into people’s tendency to not consider all relevant information when making judgements about causality. Procedure Participants were recruited via an online platform and completed all the tasks in the study over two days. In the first session, they completed the measures of reflection, numeracy, and access to abstract rules. In the second session, they were randomly assigned to one of the five conditions. Upon reading the corresponding instructions, they were then asked to rate, on a scale of 0 (not at all) to 100 (completely), how willing they were to comply with these instructions. They then completed the 12 judgements tasks in two blocks, with the order randomized within each block. 3 Assignment You should analyse the dataset you have been assigned (with either R or JASP) and write a short report (up to 3000 words1) detailing your analyses and results. It is up to you to decide how you analyse the data. You should at least test the following hypotheses: 1. Does instructing participants to “think slow” increase or decrease their overall performance in the judgement tasks (compared to no instruction)? 2. Does instructing participants to “think fast” increase or decrease their overall performance in the judgement tasks (compared to no instruction)? 3. Are these effects different for those with a different “natural tendency” to cognitively reflect on problems? In addition, there are numerous other hypotheses that are of potential interest. You can use your own intuition and imagination to explore this. If these additional hypotheses and analyses are inspired from the background information, you might classify these analyses as “confirmatory”. If they are post-hoc and the result from prior explorative analyses and tests, you should classify them as “explorative”. Your write-up should roughly take the form of a results section of journal article reporting on an empirical psychology study. You can include a (very brief) introduction outlining the main objectives of the study, as well as a very brief summary of the methods. This probably should take no more than 10% of the total word count, and is not mandatory. But if one of your hypotheses is supported by prior research, this may be useful to mention. Before conducting your confirmatory tests, you should explore the data through informative plots and descriptive statistics. You do not have to report on this part of the analysis, unless the results impact on your choice of analysis. Yuu should consider what the most appropriate analysis is to test a given hypothesis. In your write-up, you should briefly justify your choice of statistical model. For instance, if you were to include covariates in a model, you should briefly explain your choice of covariates. When reporting results, please follow the APA guidelines and include all relevant statistics in a succinct way. Don’t just focus on the significance of the tests, but also interpret what they mean (e.g., is the outcome higher or lower in one condition compared to another?). Support the test results with informative figures or tables (figures tend to be better at conveying complex results, but sometimes a table may be needed). You should end the report with a discussion of your findings, and what they imply in light of the objectives of the experiment. You should submit, as separate files, your write-up, as well as a copy of the R script (or Rmarkdown file, if you use this), or the JASP output file (if you use JASP). When possible, please submit your files in pdf format. At the end of both documents, please indicate the number of the dataset you analysed (e.g.: “Dataset analysed: data_130.csv”). Your mark will be based on the write-up, and not the R/JASP file. However, where the report is unclear, it will be very useful to assess how you obtained your results. All submissions will be marked anonymously, and we will only use the data identifier to use the appropriate dataset if we need to check your results (not to link against your name). The data You should download the unique data set assigned to you. Each data file is available as a comma separated value (csv) file, and named e.g. data_130.csv if you were assigned dataset number 130. All datasets contain real data from a real experiment, but each data set is different, and hence your results will differ from those of other students. All datasets contain the following variables: ? id: unique numeric id for each participant ? gender: stated gender (“male”, “female”, or “other”). ? age_group: numeric variable indicating age group. 1 = “Under 18”, 2 = “18-24”, 3 = “25-30”, 4 = “31-40”, 5 = “41-50”, 6 = “51-64”, 7 = “65 or over”. ? education: numeric variable indicating the education level. 1 = “elementary school”, 2 = “Middle school/junior high”, 3 = “Some high school”, 4 = “High school diploma”, 5 “Associate degree”, 6 = “Bachelor’s degree”, 7 = “Master’s degree”, 8 = “Doctorate (or more)”. 1The word count does not including tables, figures, or e.g. references if you need those. 4 ? condition: character variable indicating condition: control, slow, fast, fast-slow, or incentive. ? follow_instructions: rated willingness (between 0 and 100) to follow the instructions given. ? reflection: numeric variable indicating score on a combined cognitive reflection test (between 0 and 7). ? numeracy: numeric variable indicating score on the Berlin Numeracy Test. ? rules: numeric variable indicating total score on all rule application items (between 0 and 12). ? rules_conjuction to rules_covariation: score on the specific types of the rule application items (each between 0 and 2). The rules variable is the sum of these 6 variables. ? performance: numeric variable indicating total score on all judgement items (between 0 and 12) ? perf_conjunction to perf_covariation: score on the specific types of the judgement items (each between 0 and 2). The performance variable is the sum of these 6 variables. ? ave_rt: numeric variable indicating the average response time (in seconds) to the judgement items. ? rt_conjuction to rt_covariation: response time for the specific types of the judgement items. this is the sum of the response times to each of the two items of a specific type. The ave_rt variable is the sum of these 6 variables divided by 12. ? total_duration: overall time taken to complete the second part of the study. References Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., and Garcia-Retamero, R. (2012). Measuring risk literacy: The berlin numeracy test. Judgment and Decision making. Evans, J. S. B. and Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on psychological science, 8(3):223–241. Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic perspectives, 19(4):25–42. Gladwell, M. (2005). Blink: The Power of Thinking Without Thinking. Back Bay Books. Kahneman, D. (2011). Thinking, fast and slow. Macmillan. Kahneman, D. and Tversky, A. (1973). On the psychology of prediction. Psychological review, 80(4):237. Keren, G. and Schul, Y. (2009). Two is not always better than one: A critical evaluation of two-system theories. Perspectives on psychological science, 4(6):533–550. Kirkpatrick, L. A. and Epstein, S. (1992). Cognitive-experiential self-theory and subjective probability: further evidence for two conceptual systems. Journal of personality and social psychology, 63(4):534. Ritov, I. and Baron, J. (1990). Reluctance to vaccinate: Omission bias and ambiguity. Journal of behavioral decision making, 3(4):263–277. Shanks, D. R. and St. John, M. F. (1994). Characteristics of dissociable human learning-systems. Behav Brain Sci, 17(3):367–395. Sloman, S. A. (1996). The empirical case for two systems of reasoning. Psychological bulletin, 119(1):3. Stanovich, K. E. and West, R. F. (1999). Discrepancies between normative and descriptive models of decision making and the understanding/acceptance principle. Cognitive psychology, 38(3):349–385. Stanovich, K. E. and West, R. F. (2008). On the relative independence of thinking biases and cognitive ability. Journal of personality and social psychology, 94(4):672. Thomson, K. S. and Oppenheimer, D. M. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision making, 11(1):99. Tversky, A. and Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological review, 90(4):293. Wasserman, E. A., Dorner, W., and Kao, S. (1990). Contributions of specific cell information to judgments of interevent contingency. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(3):509. 5 學霸聯盟:"http://aessay.org"