Faking the Implicit Association Test (IAT): Predictors, Processes, and Detection

Transkript

1 Faking the Implicit Association Test (IAT): Predictors, Processes, and Detection Dissertation zur Erlangung des akademischen Grades doctor rerum naturalium (Dr. rer. nat.) vorgelegt der Fakultät für Human- und Sozialwissenschaften der Technischen Universität Chemnitz am 05. Juli 2013 von Diplom-Psychologin Jessica Röhner, geboren am in Meerane, Technische Universität Chemnitz Gutachter dieser Arbeit: Prof. Dr. Astrid Schütz, Otto-Friedrich-Universität Bamberg Prof. Dr. Josef Krems, Technische Universität Chemnitz

2 Danksagung Ich möchte mich bei allen Personen von ganzem Herzen bedanken, die geholfen haben, diese Dissertation zu ermöglichen. Ich danke meiner Doktormutter Prof. Dr. Astrid Schütz, die mich von Anfang an engagiert unterstützte und mir den nötigen Freiraum zur Realisierung meiner Ideen gab. Sie hatte immer ein offenes Ohr für meine Ideen und inspirierte mich durch ihre konstruktive Kritik bei der Erforschung im Bereich dieses vergleichsweise jungen, spannenden, aber auch stark polarisierenden Gebietes. In diesem Zusammenhang möchte ich mich auch für die vielen anregenden und interessanten Diskussionen in einer angenehmen Atmosphäre bedanken. Sie ermöglichte es mir, auf vielfältige Art und Weise mit anderen Wissenschaftlern und Wissenschaftlerinnen in Kontakt zu treten und Kenntnisse sowie Ideen auszutauschen. Auch hierfür möchte ich mich herzlich bei Frau Prof. Dr. Schütz bedanken. Weiterhin danke ich Prof. Dr. Josef Krems für seine Bereitschaft, diese Arbeit zu begutachten. Ich danke auch meinen (Ex-)KollegInnen Detlef Bergmann, Olena Klavina, Sophia Nizielski, Antje Rätzer-Schramm, Jasmin Reinhard, Dr. Katrin Rentzsch, Dr. Almut Rudolph, Martin Rüdiger und Dr. Michela Schröder-Abé für ihre vielfältigen Anregungen und ihr konstruktives Feedback. Ganz besonders möchte ich hierbei meine Kollegin und inzwischen auch gute Freundin Luise Bartholdt hervorheben. Luise, ich danke dir für die produktive Zusammenarbeit, die vielen Ratschläge und dafür, dass du mir immer mit Rat und Tat zur Seite standest. Nicht vergessen möchte ich PD Dr. Stefan Engeser, Dr. Jens Eisermann und Jun.-Prof. Dr. Anja Strobel, die mir während ihrer Vertretungszeit an der Professur viele hilfreiche Tipps und konstruktives Feedback gaben. Ein Dank geht auch an die Probanden und Probandinnen für ihre Teilnahme an meinen Untersuchungen. Zudem möchte ich mich bei den Teilnehmenden meiner Seminare zum vorliegenden Thema bedanken. Ich freue mich, dass ich bei vielen von euch das Interesse für dieses spannende und spannungsgeladene Thema wecken konnte. Nicht vergessen möchte ich auch die Studierenden, die mit ihren Projekt-, Bachelor- und Masterarbeiten (inzwischen unter meiner Betreuung) an meine Arbeit angeknüpft haben.

3 Ganz besonders möchte ich mich hiermit bei den Menschen bedanken, die mich schon seit Jahren treu auf meinem Weg begleiten: Silvia und Tanja Augustin, Janine Hoffmann, Mario Heinig, Sebastian Hoppe, Stefan Krüger, Matthias Lorenz, Manuela Mätzold, Marco Ranger und Patrick Saller. Jede und jeder Einzelne von euch hat auf seine/ihre ganz eigene Art und Weise dazu beigetragen, dass meine Dissertation in dieser Form entstehen konnte, und dafür danke ich euch aus tiefstem Herzen. Ich danke auch meiner Oma Ingrid Klotz, die immer ein offenes Ohr für mich hatte, und meinem Vater Günter Röhner, der mich bei Bedarf unterstützte. Mein ganz besonderer Dank geht in diesem Zusammenhang an meinen Bruder Christian Röhner, der mir durch seine intelligenten Rückfragen und seine Fähigkeit, Dinge aus sehr verschiedenen Perspektiven zu betrachten, eine besonders starke Quelle der Inspiration war. Vergessen möchte ich auch nicht die vielen Angehörigen, die den Abschluss meiner Promotion leider nicht mehr miterleben durften: Joachim Klotz, Walther Röhner, Charlotte Röhner (geb. Zschörnich) und Linda Schmidt (geb. Rosenbusch). Nicht zuletzt möchte ich mich bei Philipp Thoss bedanken, der mir seit Beginn meiner Promotion immer zur Seite stand und als ruhiger Fels in der Brandung meine oft überschäumenden Gedanken- und Ideenstrudel mit viel Geduld und Nachsicht ertrug. Philipp, ich danke dir aus tiefstem Herzen für das Erdulden meiner oft nächtlichen Einfälle verbunden mit der Aufforderung zu anschließender Diskussion, für das Tolerieren von Mindmapping und Planung neuer Untersuchungen während des Urlaubs sowie für deinen bedingungslosen Beistand bei dem sehr spannenden, aber nicht immer einfachen Unterfangen, Wissen zu schaffen. Deine Toleranz gegenüber nicht enden wollenden Ausführungen über Vor- und Nachteile Impliziter Assoziationstests (IATs) haben mich erheblich bestärkt und mir den Mut gegeben, für meine Ideen und deren Implikationen auch nach teils polemischer Kritik weiter zu kämpfen und einzustehen.

4 Zusammenfassung Unverfälschbarkeit stellt ein wichtiges Gütekriterium psychologischer Testverfahren dar. Dieses Kriterium gilt dann als erfüllt, wenn das Testverfahren auf Grund seiner Konstruktion keine Steuerung oder Verzerrung der Ausprägung von Testwerten seitens der Versuchspersonen ermöglicht (vgl. Moosbrugger & Kelava, 2012). Im Gegensatz zu direkten Verfahren (z.b. Fragebogen und Interviews), bei welchen die Ausprägung hinsichtlich eines Merkmales durch Selbstbeschreibung der Versuchspersonen erfragt wird und eine Verfälschung (z.b. durch sozial erwünschtes Antwortverhalten) nicht ausgeschlossen werden kann, wurde indirekten Verfahren (z.b. dem Impliziten Assoziationstest; IAT; Greenwald, McGhee, & Schwartz, 1998) lange Zeit Immunität gegen Fälschungsversuche unterstellt. Diese begründet sich unter anderem durch die Annahme, dass mittels indirekter Verfahren implizite Merkmale gemessen werden. Implizite Merkmale unterscheiden sich von den eher klassischen expliziten Merkmalen, welche vorwiegend mittels direkter Verfahren gemessen werden. Ein wesentlicher Unterschied besteht darin, dass Versuchspersonen nicht notwendigerweise um die Ausprägung hinsichtlich ihrer impliziten Merkmale wissen und dass sie diese Ausprägung auch nicht kontrollieren können (vgl. De Houwer, 2006; De Houwer & Moors, 2007, in press). Die theoretischen Annahmen bezüglich der Eigenschaften impliziter Merkmale bzw. Messergebnisse legen zwei Implikationen nahe. Erstens: Wir können implizite Merkmale ausschließlich über indirekte Zugänge erfassen, da diese nicht notwendigerweise bewusst sind und so eine Selbstauskunft nicht möglich erscheint. Zweitens: Personen können ihre impliziten Messergebnisse nicht kontrollieren und folglich auch nicht verfälschen. Vermutlich gab es auch aus diesem Grund vor wenigen Jahren einen regelrechten Boom, der zu der Entwicklung einer Vielzahl indirekter Verfahren zur Erfassung impliziter Merkmale geführt hat. Ob jedoch die Messergebnisse dieser Verfahren tatsächlich implizit und damit nicht verfälschbar sind, darf nicht nur theoretisch unterstellt, sondern muss empirisch überprüft werden (vgl. De Houwer, 2006). Der IAT gilt als das bekannteste, reliabelste und valideste indirekte Verfahren (Bosson, Swan, & Pennebaker, 2000; Rudolph, Schröder-Abé, Schütz, Gregg, & Sedikides, 2008). In meiner

5 Dissertation habe ich mich aus diesem Grund der empirischen Überprüfung auf Verfälschbarkeit des IATs gewidmet. Die vorliegende Dissertation besteht aus insgesamt fünf Kapiteln. Das 1. Kapitel bildet eine theoretische Einführung zu den Themen Fälschung im diagnostischen Kontext und zum IAT. Grundlegende Befunde und Fragen zur Verfälschbarkeit des IATs werden dargestellt. Kapitel 2 bis 4 bilden empirische Beiträge meiner Forschung, die sich jeweils schwerpunktmäßig mit unterschiedlichen Aspekten der Verfälschbarkeit des IATs beschäftigen. In Kapitel 2 wird der Frage nachgegangen, unter welchen Bedingungen der IAT verfälschbar ist. Bis dato haben die wenigen existierenden Studien ein sehr widersprüchliches Bild bezüglich der Verfälschbarkeit des IATs aufgezeigt. Ein Grund hierfür könnte sein, dass potentiell relevante Faktoren, welche die Verfälschbarkeit des Verfahrens beeinflussen können, noch nie gemeinsam in einer Studie untersucht wurden. Die vorliegende Studie wurde genau mit diesem Ziel konstruiert und durchgeführt. Die Ergebnisse verweisen auf ein komplexes Zusammenspiel verschiedener Faktoren und zeigen auf, unter welchen Bedingungen der IAT verfälschbar ist. Implikationen dieser Ergebnisse werden kritisch diskutiert. In Kapitel 3 werden die Fragen beantwortet, wie Personen den IAT verfälschen und ob Fälschung im IAT detektierbar ist. Die Forschung hat sich bislang nur bedingt damit beschäftigt, was fälschende Personen tun, um ihre Messergebnisse wie gewünscht zu beeinflussen. Es wurde auch noch nicht untersucht, ob Versuchspersonen unter verschiedenen Bedingungen (z.b. Fälschungsziel: hohe vs. niedrige Testwerte) unterschiedliche Strategien anwenden. Dennoch wurden Indices vorgeschlagen, welche in der Lage sein sollen, Fälschung im IAT zu detektieren (Agosta, Ghirardi, Zogmaister, Castiello, & Sartori, 2011; Cvencek, Greenwald, Brown, Gray, & Snowden, 2010). In der vorgestellten Studie habe ich einerseits untersucht, welche Strategien fälschende Personen anwenden und ob sie, je nach Bedingung, zu unterschiedlichen Strategien greifen. Andererseits habe ich untersucht, welche dieser Strategien tatsächlich mit erfolgreicher Fälschung des IATs einhergehen. Schließlich habe ich untersucht, ob die in der Vergangenheit vorgeschlagenen Indices tatsächlich in der Lage sind, erfolgreiche FälscherInnen zu detektieren. Meine Ergebnisse zeigen, dass fälschende Personen unterschiedliche Strategien anwenden, um ihr Ziel zu erreichen. Damit verbunden zeigte sich auch, dass es schwerer ist als bislang angenommen, erfolgreiche FälscherInnen im IAT zu detektieren. Implikationen dieser Ergebnisse werden kritisch diskutiert. Kapitel 4 beschäftigt sich mit der Frage, ob kognitive Fähigkeiten ein erfolgreiches Fälschen im IAT erleichtern. Bisher wurden diese Fähigkeiten nur mit Fälschungserfolg in direkten Verfahren in

6 Verbindung gebracht (vgl. Hartshorne & May, 1928; Nguyen, Biderman, & McDaniel, 2005; Ones, Viswesvaran, & Reiss, 1996; Pauls & Crost, 2005; Snell, Sydell, & Lueke, 1999; Tett, Freund, Christiansen, Fox, & Coaster, 2012; Weiner & Gibson, 2000). In der vorgestellten Studie habe ich untersucht, ob sie auch beim Fälschen des IATs eine Rolle spielen. Besonders habe ich mich dabei für die Rolle des g Faktors der Intelligenz, der Verarbeitungsgeschwindigkeit und der Konzentrationsfähigkeit interessiert. Die Ergebnisse meiner Studie zeigen auf, dass einige dieser Prädiktoren tatsächlich einen Einfluss auf den Fälschungserfolg im IAT haben. Implikationen dieser Ergebnisse werden kritisch diskutiert. Das 5. Kapitel bildet eine Zusammenführung und Integration der Befunde meiner Forschung in die bestehende Theorie. Zudem werden ein Ausblick für die weitere Forschung sowie Empfehlungen für die Praxis gegeben.

7 Table of contents Chapter 1: Introduction The concept of faking Faking and related concepts: Many names - one meaning? Why faking is worthy of empirical investigation Do people fake in psychological contexts? Is faking a problem for psychological assessment? Can faking be detected or even prevented? Indirect measures as a solution for fakeability? What are indirect measures? What is the IAT? Can one prevent faking by using the IAT? The Questions in this dissertation Under which conditions is the IAT fakeable? How do people fake the IAT and is faking on the IAT detectable? Do cognitive abilities facilitate faking on the IAT? Chapter 2: Under which conditions is the IAT fakeable? Status quo of the research on the fakeability of the IAT Faking success after general instructions or the use of persuasive material Faking success after specific information about the desired impression Faking success after teaching faking strategies Boundary conditions of faking success A critical review of past research on faking success The present study Method Participants Procedure Faking instructions Measures... 25

8 2.4 Results Reliability Manipulation check Faking of the self-esteem IAT Predicting faking success by individual differences in baseline implicit self-esteem Discussion and conclusion Which factors influence the fakeability of the IAT? Limitations and further directions Theoretical and practical implications Chapter 3: How do people fake the IAT and is faking on the IAT detectable? Replicating the IAT procedure The IAT procedure using the extraversion IAT as an example The computation of the IAT effect Which faking strategies are conceptually possible? Conceptually possible strategies involving the manipulation of reaction times Conceptually possible strategies involving the manipulation of errors A critical review of past research on faking strategies Why people might use strategies beyond slowing down Why faking direction matters Why pre-existing knowledge matters Why faking success matters The present study Method Participants Procedure Measures Analytic strategy... 44

9 3.6 Results Reliability Manipulation check Distribution of error rates across the different faking conditions What fakers do at naïve faking and what leads to faking success What fakers do at informed faking and what leads to faking success Discussion and conclusion What fakers do to fake the IAT What leads to faking success? Is it sufficient to look at slowing down to reveal successful faking? Limitations Theoretical and practical implications Chapter 4: Do cognitive abilities facilitate faking on the IAT? Do cognitive abilities predict faking success on the IAT? Why intelligence (the g factor) might predict faking success Why processing speed might predict faking success Why concentration performance might predict faking success The present study Method Participants Procedure Faking instructions Measures Results Reliability Manipulation check Predicting faking success by individual differences in cognitive abilities Discussion and conclusion Do cognitive abilities predict faking success on the IAT? Theoretical and practical implications Limitations and further directions Conclusion... 74

10 Chapter 5: Summary and concluding comments Are people able to fake the IAT? Does faking on the IAT represent a problem for psychological assessment? Effects of faking on IAT scores and rank ordering Effects of faking on the validity of IATs Can faking be detected on the IAT? Theoretical implications Existing faking models should be extended Integrating fakeability of the IAT into (dual-)process models Contamination of IAT scores A general limitation Future directions Robustness against faking of other indirect measures? Is it possible to prevent people from faking on IATs? Detection of faked scores Implications for practical settings Interpret IAT scores with caution! Consider that participants may have IAT experience! Consider that participants may know how the IAT works! Do not simply rely on faking indices! References 90 Appendix 109 Eidesstattliche Erklärung Lebenslauf Jessica Röhner

11 Chapter 1: Introduction 1 CHAPTER 1: INTRODUCTION 1 Lüge: Ein Double, das die Wahrheit in gefährlichen Situationen vertritt [Lie: A double for the truth in risky situations]. Wieslaw Brudzinski ( ) This statement by Wieslaw Brudzinski illustrates that there can be situations in life in which it is not advantageous or might even be unprofitable to state the truth exactly as it is. One of those situations might be when a person is subjected to a psychological assessment. Why? Psychological tests are implemented in situations where there is a lot at stake, for example, in the areas of selection and clinical diagnostics (Ziegler, MacCann, & Roberts, 2012). In such high-stakes situations, people may have a vested interest in making a certain impression. Therefore, they may sometimes try to guide their test scores in a direction that does not necessarily represent the truth. In selection processes, people may make an effort to represent themselves as very conscientious because they think conscientiousness is an important trait of the future employee. In clinical diagnostics, people may have an interest in simulating depression or another mental disorder in order to obtain an early retirement pension by fraud. Although the reasons and goals that people have for faking psychological tests may seem obvious, faking is a very controversial topic. The fakeability of indirect measures, in particular, has been long neglected in scientific research. How can this be explained? Dualprocess models (Back, Schmukle, & Egloff, 2009; Strack & Deutsch, 2004) have suggested 1 The major aspects of formatting (e.g., alignment, font, layout, and positioning of figures and tables, reference list, style of the language) of this dissertation follow the guidelines of the American Psychological Association (i.e., APA, 6th ed.). For a better structure and in order to make my work more accessible to the reader, I deviated from the guidelines by numbering the headings at Levels 1, 2, and 3.

12 CHAPTER 1: INTRODUCTION 2 that direct and indirect measures 2 assess different representations of the self. Whereas answers on direct measures (e.g., self-description questionnaires) are the result of a person s conscious decision process, the behavior measured by indirect measures (e.g., Implicit Association Tests; Greenwald, McGhee, & Schwartz, 1998) is assumed to result from associative links and motivational orientations (i.e., approach and avoidance tendencies) that are not necessarily conscious. Accordingly, indirect measures are different from the more traditional direct measures because they do not rely on self-reports (i.e., the result of a conscious decision process) but instead use reaction times or errors (i.e., an indication of the associative links) to assess the construct of interest. Indirect measures have been assumed to be robust against faking as researchers have suggested that the behavior measured by them is elicited by unconscious and uncontrollable associations. By contrast, it seems evident that the results of direct measures, as they stem from the participant s decision process, may involve a process of weighting potential answers against each other (e.g., by assessing which answer will make a good impression). Most likely as a consequence of these assumptions, the fakeability of direct measures has been researched for about 60 years. By contrast, indirect measures have for a long time and without any empirical research been assumed to be immune to faking. This is probably one of the reasons why empirical studies in this controversial and less attended area of research are lacking. My dissertation is aimed at answering questions and closing some of the gaps regarding the fakeability of the most popular indirect measure, namely, the Implicit Association Test (IAT; Greenwald et al., 1998). Prior to that, this chapter is aimed at 2 Sometimes the term implicit measures is used incorrectly as a synonym for indirect measures, and the term explicit measures is used instead of direct measures by some researchers. The terms explicit and implicit measures, however, do not refer to the measurement procedures but rather to the measurement outcomes themselves (De Houwer, 2006; De Houwer & Moors, 2007, in press). In the following, I will thus use the terms direct and indirect when referring to measurement procedures. Accordingly, I will use the terms explicit and implicit only when referring to the outcomes of these measurement procedures and to the underlying constructs.

13 CHAPTER 1: INTRODUCTION 3 providing an overview of the concept of faking, of indirect measures, and of the IAT. In the following, faking and related concepts will be explained and distinguished from each other. 1.1 The Concept of Faking How can we define and describe faking behavior? Faking can be defined as a response set aimed at providing a portrayal of the self that helps a person to achieve personal goals. Faking occurs when this response set is activated by situational demands and person characteristics to produce systematic differences in test scores that are not caused by the attribute of interest (Ziegler et al., 2012, p. 8). Faking includes the following three key features: intentionality (i.e., it is a deliberate and conscious process), other-orientation (i.e., it is a question of deceiving others rather than oneself), and deception (i.e., it is a matter of not being honest; Holden & Book, 2012). Different models have been suggested for describing faking behavior (e.g., McFarland & Ryan, 2000; Mueller-Hanson, Heggestad, & Thornton, 2006; Snell, Sydell, & Lueke, 1999; Ziegler, 2011). One aspect that they all have in common is the assumption that faking requires at least two components. First, a participant must be motivated to fake his or her test outcome. Second, the participant must be able to fake his or her test outcome. The latter was suggested to be related to the general mental ability of the participant (e.g., Hartshorne & May, 1928; Nguyen, Biderman, & McDaniel, 2005; Ones, Viswesvaran, & Reiss, 1996; Pauls & Crost, 2005; Snell et al., 1999; Tett, Freund, Christiansen, Fox, & Coaster, 2012; Weiner & Gibson, 2000). Faking models, however, are largely untested (see Mueller-Hanson et al., 2006), and variables that influence faking success have predominantly been merely proposed instead of being investigated empirically. Furthermore, research on variables that influence faking success have focused on faking with regard to direct measures or applicant samples only. However, we can summarize that faking models assume that people are able to fake when they are motivated to do so. The ability to fake might additionally depend on several variables and processes that still remain to be investigated.

14 CHAPTER 1: INTRODUCTION Faking and related concepts: Many names - one meaning? There are a lot of concepts that have often been used synonymously with the term faking. These concepts are, for example, response distortion (e.g., Rosse, Stecher, Miller, & Levin, 1998), socially desirable responding (e.g., Marcus, 2003, 2004; Musch, Brockhaus, & Bröder, 2002; Paulhus 1984, 1986, 1994, 1998, 2002), self-deception (e.g., Paulhus, 1986; Paulhus, Friedhandler, & Hayes, 1997), impression management (e.g., Mueller-Hanson et al., 2006; Paulhus, 1994), faking good (e.g., McFarland & Ryan, 2000), faking bad (e.g., Boon, Gozna, & Hall, 2008), and simulation or dissimulation (e.g., Schmidt-Atzert, Bühner, Rischen, & Warkstein, 2004). All of these concepts imply the assumption that with respect to psychological measurements, the test scores are not influenced only by the attribute of interest that should be measured by the certain measurement procedure. Additionally, other sources of variance that may also influence the test results are being measured at the same time (see Ziegler et al., 2012). Nevertheless, a synonymous application of these concepts is not always admissible as I will explain below. Faking and response distortion. The concept of response distortion is sometimes used interchangeably with the term faking. However, the distortion of responses can include deliberate and conscious as well as unconscious processes (Rosse et al., 1998). As faking is always an intentional process, response distortion might include faking but the two concepts cannot be regarded as isomorphic. Faking and socially desirable responding. Responses might also be distorted by a phenomenon called socially desirable responding. Scales measuring the tendency of people to respond in a socially desirable manner incorporate items such as: I don t gossip about other people s business or My first impressions of people usually turn out to be right (BIDR; Paulhus, 1998). These items illustrate that socially desirable responding is linked to people s motivation to present themselves in a favorable light. Socially desirable responding can be divided into two subfacets called self-deceptive enhancement and impression management

15 CHAPTER 1: INTRODUCTION 5 (e.g., Musch et al., 2002; Paulhus, 1986, 1994, 1998). The subfacets determine the goal of the favorable self-presentation (i.e., looking good to oneself vs. looking good to others). Paulhus (1986) defined self-deceptive enhancement as a less conscious attempt to look good to oneself (p. 146), whereby a certain level of self-deceptive enhancement is regarded as typical of mentally healthy individuals (e.g., to protect self-esteem; see Paulhus, 1986). In contrast to self-deceptive enhancement, impression management is described as a conscious response distortion aimed at conveying a certain impression of oneself to others (Paulhus, 1994). It is, however, arguable whether the adaption to social norms and the endeavor to impress the public necessarily require deliberate distortion (see Mummendey, 1990; Musch et al., 2002; Tedeschi, 1981). In contrast to socially desirable responding, faking is always an intentional form of response distortion that is always oriented toward others (Holden & Book, 2012). Consequently, faking might be distinguishable from both subfacets of socially desirable responding. Faking and self-deceptive enhancement might be distinguishable concepts because faking is always oriented toward others, whereas self-deceptive enhancement is not. Faking might also be distinguishable from impression management because impression management is not necessarily a conscious process, which is, however, a key definitional feature of faking. Supporting these assumptions, Wrensen and Biderman (2005) determined that socially desirable responding is negatively connected to faking success (r = -.24). However, results by Tett and Christiansen (2007) as well as Griffith and Peterson (2008) indicated that there is a moderate effect regarding the correlation between socially desirable responding and faking (i.e., about r =.32). Thus, it is most likely that impression management and faking share some common features, namely, when impression management passes as an intentional and conscious process. Faking, faking good/dissimulation, and faking bad/simulation. The concepts faking good/dissimulation and faking bad/simulation are faking subfacets that specify the direction

16 CHAPTER 1: INTRODUCTION 6 of faking, although the expressions dissimulation and simulation are more commonly used in clinical settings. The concepts faking good and dissimulation describe cases in which participants fake high scores on a desired outcome variable (e.g., Jonkisz, Moosbrugger, & Brandt, 2012). They might try to hide an illness, impairment, or weakness from others (e.g., Schmidt-Atzert et al., 2004). Dissimulating or faking good in situations such as job applications may be used for the purpose of maximizing their chances of being hired. The concepts simulation and faking bad describe cases in which participants purposely pretend to have low scores on a desired outcome variable (e.g., Jonkisz et al., 2012), for example, if they fake having the symptoms of an illness or impairment (e.g., Boon et al., 2008; Schmidt-Atzert et al., 2004). They can be motivated to simulate or fake bad because of compensation claims for psychological and physical injuries or in the case of criminal offenders to reduce a sentence or even to escape sentencing. Integrating faking into related concepts. To summarize, response distortion can be viewed as a generic term comprising the concepts socially desirable responding and faking. Probably only one facet of socially desirable responding (i.e., impression management) has an overlap with the concept of faking. Faking can be further divided into faking good, faking bad, simulation, and dissimulation in order to describe faking directions (e.g., high or low scores on the construct of interest). During the course of completing my doctoral thesis, I developed an overview of the conceptual distinctions between these faking-related concepts (see Figure 1). In the following section, I will try to highlight why faking, although not wellresearched, is such an important topic Why faking is worthy of empirical investigation. Unfakeability represents an important psychometric property of psychological measurement procedures. This psychometric property is said to be achieved if the measurement procedure is designed in such a way that it is impossible for the participants to control or distort their test scores (see Moosbrugger & Kelava, 2012).

17 CHAPTER 1: INTRODUCTION 7 From a logical and very critical standpoint, one could argue that faking would be largely ignorable if at least one of three conditions was fulfilled: first, if people did not fake at all (e.g., because they were not able to); second, if faking didn t represent a problem of psychological assessment; and third, if faking could be detected or even prevented. Three questions can be posed relating to those relevant conditions: first, Do people fake in psychological contexts? ; second, Is faking a problem regarding psychological assessment? ; and third, Can one detect or even prevent faking?. Thus, I will highlight the importance of faking as a research topic by summarizing empirical research and consecutively answering those questions in the following. Naturally, the presented results stem from research that has investigated faking on direct measures as faking on indirect measures has been highly neglected so far. However, the results will point out the problems linked to and caused by faking. The results may also illustrate why the IAT has caused so much excitement as researchers have previously assumed it to be a less fakeable or unfakeable measure Do people fake in psychological contexts? Faking would not be an important topic in psychological assessment if no one faked. So the question is: Do people actually fake their test scores to convey a certain impression of themselves to others? Some empirical studies have attempted to research this issue. The proportion of fakers in applied settings. Griffith, Chmielowski, and Yoshita (2007) tried to answer this question by using a within-subjects design in a real applicant setting. As part of the application process, applicants were required to complete a personality test (i.e., a direct measure to assess conscientiousness). Later, the researchers again contacted the people who had applied for a job and asked them to complete the same personality test twice again. First, they were instructed to answer in the most honest way possible. After this, they were asked to complete the personality test in such a way that it would evoke a very good impression of themselves (i.e., faking good). The test scores of the participants differed depending on the situation they were in (i.e., application, honest, or faking good condition).

18 CHAPTER 1: INTRODUCTION 8 Comparing the test scores of the participants across the several conditions revealed that faking was not only possible, but also that up to 50% of the applicants had faked their scores in the real application situation in order to improve their chances in the selection process. The systematicness of faked responses in applied settings. Meta-analyses by Birkeland, Manson, Kisamore, Brannik, and Smith (2006) as well as Viswesvaran and Ones (1999) have shown that applicants do fake direct measures that are used to assess the Big Five personality variables. Birkeland et al. (2006) additionally demonstrated that faking was not executed in an unsystematic manner. Applicants rather faked their answers depending on the job they applied for and modified their answers in a well-directed way on the dimension they regarded as being job-relevant. Thus, we can summarize that faking on psychological measurements is actually possible and that a lot of people do fake, at least on direct measures. More dramatically, they fake in a systematic manner and modify their answers in a well-directed way. These results would be largely ignorable if faking had no impact on the validity of tests or on our diagnostic decisions. Does it? Is faking a problem for psychological assessment? Results regarding the effects of faking are very sparse. Nevertheless, I would like to carefully describe some of the results. Effects of faking on scale means and rank ordering. Studies have shown that faking does influence scale means (Rosse et al., 1998; Stark, Chernyshenko, Chan, Lee, & Drasgow, 2001; Viswesvaran & Ones, 1999). The reported effects are at least moderate (see Holden & Book, 2012). The shift in mean scores toward higher or lower scores would not be a big problem if all test participants results showed the same shift toward higher or lower scores. This would leave the rank order of test scores unaffected. However, studies have shown that the rank order changes due to faking (Christiansen, Goffin, Johnston, & Rothstein, 1994; Rosse et al., 1998). Therefore, in selection processes, there is the potential that the decision-

19 CHAPTER 1: INTRODUCTION 9 maker will make a decision that is not in favor of the participant with the highest true score (e.g., on conscientiousness) but is rather in favor of the participant who has faked the highest score. Effects of faking on the correlational structures of tests. Moreover, faking can have an impact on the correlational structure of tests. Results by Schmit and Ryan (1993) indicated NEO-FFI factor differences between job applicants and nonapplicants as they found an additional factor called the ideal employee among the job applicants. Thus, the correlational structure of tests can be changed via faking. Effects of faking on the validity of tests. In this line of research, Bäckström, Björklund, and Larsson (2009) showed that for three different five-factor inventories, the variance could be attributed to only one factor. This factor was highly associated with social desirability. Accordingly, the validity of tests can be impaired via faking. How can this be explained? In psychological assessments, the focus is on the systematic variance caused by the attribute of interest. The test scores of fakers, however, additionally incorporate a second systematic source of variance that influences the test results: the variance caused by faking. Thus, it is not only the attribute of interest that is assessed, but also the ability to fake (Barrick & Mount, 1996; Stark et al., 2001). This phenomenon has consequences for construct validity. Consequences regarding criterion validity, however, have also been discussed. Depending on the proportions of variance that are accounted for by the attribute of interest and by faking, either faking is assessed instead of the attribute of interest, or vice versa. This fact can have consequences for criterion validity (e.g., Douglas, McDaniel, & Snell, 1996; Holden, Wood, & Tomashewski, 2001). To summarize, people do fake, and their faking is likely to affect scale means, rank orders of test scores, correlational structures, and test validity. For these reasons, it would be beneficial to be able to detect faked scores and to correct them, or better yet, to prevent faking completely. If this was possible, faking as a factor would be negligible. So is it possible?

20 CHAPTER 1: INTRODUCTION Can faking be detected or even prevented? One can differentiate between proactive approaches to faking and reactive approaches against faking. Proactive approaches are implemented with the goal of preventing faking, whereas reactive approaches are implemented in order to detect and optionally, as a second step, to correct faking. Approaches to detect faking. The strategies that have been suggested to detect faking (i.e., reactive approaches) range from identifying fakers by their scores on SDR scales (e.g., BIDR; Paulhus, 1994, 1998; Musch et al., 2002) to recent approaches such as identifying fakers through word use (Ventura, 2012) or response latencies (Robie et al., 2000). After detecting fakers, one can try to correct the dataset by either removing the fakers data, eliminating the faked responses, or retesting. However, there is a lot of criticism regarding procedures that try to detect and correct faking (e.g., as already explained, social desirability does not predict faking). Additionally, not many studies have supported the utility of methods that are used to detect faking. Again, most of the studies have focused on socially desirable responding rather than faking. Thus, we cannot detect and correct faking at present because a lot more research is needed. Approaches to prevent faking. Therefore, researchers have tried to develop strategies to prevent faking (i.e., proactive approaches); for example, neutralizing items (Bäckström et al., 2009) or using different types of warnings (e.g., Dilchert & Ones 2012). However, results are contradictory and disappointing. Although some studies have found no significant differences in faking behavior between warned and unwarned participants (Sullivan & Richter, 2002), others have indicated more inventive faking techniques among warned participants (Youngjohn, Lees-Haley, & Binder, 1999) or even negative effects on the response behavior of honest test takers in the case of lump-sum warnings (Dwight & Donovan, 2003). Thus, in the worst case scenario, warning the participants not to fake might even turn out to be disadvantageous.

21 CHAPTER 1: INTRODUCTION 11 As stated earlier, faking would be largely ignorable if at least one of three conditions was fulfilled: first, if people did not fake at all (e.g., because they were not able to); second, if faking did not represent a problem regarding psychological assessment; and third, if faking could be detected or even prevented. The empirical research I presented earlier clearly highlights that faking cannot be ignored because: (a) people do fake; (b) faking represents a problem in psychological assessment; and (c) until now, we have not been able to correct faking or to prevent people from faking. So would it not be great to have measures that simply cannot be faked? 1.2 Indirect Measures as a Solution for Fakeability? As mentioned before, indirect measures are assumed to be more immune to faking because they do not rely on self-reports but rather use reaction times or errors to assess, for example, personality, attitudes, or preferences. This might be one of the reasons why they have become widely used in marketing, personality psychology, social psychology, clinical psychology, and health psychology to name only a few areas (e.g., Asendorpf, Banse, & Mücke, 2002; Brunel, Tietje, & Greenwald, 2004; Greenwald et al., 1998; Teachmann, Gregg, & Woody, 2001; Wiers, Van Woerden, Smulders, & De Jong, 2002) What are indirect measures? As is also true for the term direct measure, the term indirect measure refers to a measurement procedure (De Houwer, 2006; De Houwer & Moors, 2007, in press). Indirect measures differ from direct measures as they aim to draw inferences about the construct of interest by measuring reaction times or errors instead of asking participants to self-assess the extent to which they embody, for example, a certain personality trait (De Houwer, 2006). Additionally, most indirect measures are assumed to assess implicit measures (i.e., the measurement outcome), whereas most direct measures are assumed to assess explicit measures. Properties of the measurement outcome. De Houwer (2006) and De Houwer and Moors (2007, in press) have claimed that implicit measures should be a result of a virtue of

22 CHAPTER 1: INTRODUCTION 12 processes that are, amongst others, uncontrolled, unintentional, goal-independent, purely stimulus-driven, autonomous, unconscious, efficient, and fast. Thus, by definition, the measurement outcome of indirect measures should be immune to faking as it must fulfill the requirement of being uncontrollable and hence unfakeable. The popularity of indirect measures. In less than a decade, assessing implicit measures via indirect measurement procedures has become very popular due to their assumed advantages. There was a boom and sudden interest in indirect measures (assumed to assess implicit measures) that resulted in the development of various indirect measures such as the IAT (Greenwald & Farnham, 2000; Greenwald et al., 1998) with the modified forms of the Single-Category IAT (SC-IAT; Karpinski & Steinman, 2006), the Single Block IAT (SB- IAT; Teige-Mocigemba, Klauer, & Rothermund, 2008), the Recoding-Free IAT (IAT-RF; Rothermund, Teige-Mocigemba, Gast, & Wentura, 2009), the Go/No-Go Association Task (GNAT; Nosek & Banaji, 2001), and the Extrinsic Affective Simon Task (EAST; De Houwer, 2003). Among these procedures, the IAT is the most popular and one of the most reliable and valid measures (Bosson, Swan, & Pennebaker, 2000; Rudolph, Schröder-Abé, Schütz, Gregg, & Sedikides, 2008). In the following section, I will explain the structure and procedure of the IAT What is the IAT? The IAT (Greenwald et al., 1998) is a computerized procedure that is used to assess the construct of interest in an indirect manner. Therefore, the IAT is introduced to participants as a sorting task, and they are asked to assign stimuli (i.e., words or pictures) that appear consecutively the middle of the display screen to distinct categories presented on the left and right sides of the upper half of the display screen. Altogether, within each IAT, there are four categories that belong to one of two target categories (Target Category A vs. Target Category B), building the so-called target dimension, or that represent one of two attributes (Attribute Category A vs. Attribute Category B), building the so-called attribute dimension. Participants are requested to assign

23 CHAPTER 1: INTRODUCTION 13 the stimuli to those categories by pressing a key located on the side of the keyboard that corresponds to the side of the display screen on which the category appears (e.g., the letter d might be used for the word in the upper left corner, and the letter k might be used for the word in the upper right corner). They are asked to make their assignments as quickly as possible but to simultaneously avoid errors as well. Error feedback is given if a stimulus is assigned incorrectly. Participants have to complete different blocks of sorting tasks to complete the IAT. Within the so-called single dimension practice blocks, participants have to sort stimuli into either only the two target categories or into only the two attribute categories. In other words, each key is related to only one category in this case. These blocks are used to introduce the sorting task to the participants. The crux of the matter is that within the so-called combined or critical blocks, participants are asked to sort stimuli into all four categories by still using only the two keys. In other words, in this case, each key is related to two categories: one of the target categories and one of the attribute categories. These blocks are used to measure the strength of the associations between the targets and the attributes. The IAT procedure using the self-esteem IAT as an example. To give an example, to measure implicit self-esteem with the IAT (Greenwald & Farnham, 2000; Rudolph, Schröder, & Schütz, 2006), the strength of the associations between the target dimension self-relevant (e.g., I, my, me) versus non-self-relevant (e.g., your, your, others) and the attribute dimension positive (e.g., holiday, peace, luck) versus negative (e.g., war, grief, harm) is measured (see Figure 2). Participants are instructed to respond to exemplars of those categories (appearing in the middle of the screen) by pressing a key located on the side of the keyboard that corresponds to the side of the display screen on which the category appears (i.e., the letter d is used for the word in the upper left corner, and the letter k is used for the word in the upper right corner). The IAT consists of seven blocks, whereby Blocks 1, 2, and 5 are practice blocks to introduce the target or attribute discrimination. Block 1 introduces the target

24 CHAPTER 1: INTRODUCTION 14 discrimination between self-relevant versus non-self-relevant words. Words of both categories (i.e., self-relevant and non-self-relevant) appear one after another on the computer screen. The participants identify the words as either self-relevant words or non-self-relevant words by pressing one of the two response keys. Block 2 introduces the attribute discrimination. Again, words from both categories (i.e., positive and negative) appear consecutively on the computer screen and have to be assigned by participants via the two response keys. In Block 5, the participants are asked to use the opposite assignment of the target discrimination from Block 1 and again have to assign the words appearing on the computer screen. Blocks 3, 4, 6, and 7 are the so-called critical blocks in which attribute and target discriminations are combined (i.e., participants have to assign words from all four categories in these blocks). In Blocks 3 and 4, positive and self-related words have to be responded to with one key, whereas negative and non-self-related words have to be responded to with the other key (i.e., the congruent IAT blocks). In Blocks 6 and 7, negative and selfrelated words have to be responded to with one key, whereas positive and non-self-related words have to be responded to with the other key (i.e., the incongruent IAT blocks). 3 The IAT effect. The idea behind the IAT is as simple as it is brilliant: Mapping two categories onto only one key should be easier when the two categories are associated with each other than when the two categories are not associated with each other. Accordingly, as the target categories are differentially associated with the attribute categories, participants should find one set of the combined blocks easier and therefore should react more quickly to the words in this set of the combined blocks. The IAT effect (i.e., the D measure) indicates the difference in reaction times between the two sets of the combined blocks and is assumed to provide a measure of implicit attitudinal difference between the two categories. 3 The presentation of the combined blocks can be counterbalanced in IATs. Thus, as a researcher, one can decide whether the participant will work on the congruent IAT blocks first and afterwards on the incongruent ones (i.e., the sequence of the IAT as explained above) or if the blocks should be presented the other way around. In order to avoid unnecessarily complicating the description of the IAT, only the most common block order is presented above. In my studies, I always counterbalanced the block presentation. Thus, I will explain the counterbalancing in the following chapters when necessary.

25 CHAPTER 1: INTRODUCTION 15 In the example of a self-esteem IAT, a person with high implicit self-esteem should find it easier to assign words in the first set of combined blocks (i.e., Blocks 3 and 4) because he or she has to respond to positive words with the same key as to self-relevant words (and to negative words with the same key as to non-self-relevant words), and this mapping corresponds to the person s associations in memory. Comparably, this person should find it more difficult to assign the stimuli in the second set of combined blocks (i.e., Blocks 6 and 7) because he or she has to respond to negative words and to self-relevant words with the same key (and to positive words and to non-self-relevant words with the other key), and this mapping does not correspond to the person s associations in memory. Conversely, a person with low implicit self-esteem should find it easier to assign the words in the second set of combined blocks (i.e., Blocks 6 and 7) as this corresponds to his or her associations in memory than in the first set of combined blocks (i.e., Blocks 3 and 4) as this does not correspond to his or her associations in memory Can one prevent faking by using the IAT? In contrast to direct measures, the IAT is assumed to assess implicit associations inferred from reaction times on the critical blocks. As explained before, implicit associations have to meet the requirement of being uncontrollable (e.g., unfakeable). It was assumed that IAT results constitute such implicit associations. However, De Houwer (2006) and De Houwer and Moors (2007, in press) claimed that the implicitness has to be empirically demonstrated instead of merely being assumed. Until now, results regarding the fakeability of IATs have been sparse and even contradictory. Thus, it is more than questionable whether one can prevent faking by using the IAT. 1.3 The Questions in this Dissertation Summing up, there is a large knowledge gap regarding the fakeability of psychological measures as the fakeability of indirect measures has hardly been researched so far. On account of its frequent use, reliability, and validity (Bosson et al., 2000; Rudolph et al., 2008)