Figures
Abstract
During social interactions, we make inferences about people’s personal characteristics based on their appearance. These inferences form a potential prejudice that can positively or negatively bias our interaction with them. Not much is known about the effects of negative bias on face perception and the ability to recognize people faces. This ability was investigated by recording event-related potentials (ERPs) from 128 sites in 16 volunteers. In the first session (encoding), they viewed 200 faces associated with a short fictional story that described anecdotal positive or negative characteristics about each person. In the second session (recognition), they underwent an old/new memory test, in which they had to distinguish 100 new faces from the previously shown faces. ERP data relative to the encoding phase showed a larger anterior negativity in response to negatively (vs. positively) biased faces, indicating an additional processing of faces with unpleasant social traits. In the recognition task, ERPs recorded in response to new faces elicited a larger FN400 than to old faces, and to positive than negative faces. Additionally, old faces elicited a larger Old-New parietal response than new faces, in the form of an enlarged late positive (LPC) component. An inverse solution SwLORETA (450–550 ms) indicated that remembering old faces was associated with the activation of right superior frontal gyrus (SFG), left medial temporal gyrus, and right fusiform gyrus. Only negatively connoted faces strongly activated the limbic and parahippocampal areas and the left SFG. A dissociation was found between familiarity (modulated by negative bias) and recollection (distinguishing old from new faces).
Citation: Proverbio AM, La Mastra F, Zani A (2016) How Negative Social Bias Affects Memory for Faces: An Electrical Neuroimaging Study. PLoS ONE 11(9): e0162671. https://doi.org/10.1371/journal.pone.0162671
Editor: Peter James Hills, Bournemouth University, UNITED KINGDOM
Received: February 27, 2015; Accepted: August 28, 2016; Published: September 21, 2016
Copyright: © 2016 Proverbio et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The authors confirm that all data underlying the findings are fully available without restriction. However, data containing participant information cannot be deposited to a public repository and shared, according to our ethical approval granted by the Ethics Committee of University of Milano-Bicocca. Data are confidential and will be available upon request. Interested researchers and readers can contact the corresponding author (mado.proverbio@unimib.it) of the manuscript to request access to the data.
Funding: This work was supported by the 9928 2013-ATE-0037 grant from University of Milano-Bicocca.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Prejudice is a type of "premature judgment" of subjects of which we have no direct, complete and sufficient knowledge. Allport [1] described the prejudice as a feeling, positive or negative, relative to a person or a thing before direct experience or that is not based on the person or thing. Prejudice formation is considered to be an adaptive cognitive process that helps the human mind to process information through the help of categories. After these categories have been created, prejudice inevitably arises. We know that people’s social traits are inferred on the basis of their appearance and that this judgment activates a potential bias able to influence social interactions. At this regard a social context network model (SCNM), a fronto-insular-temporal network has been proposed by Ibáñez and Manes [2], as being responsible for processing social contextual effects, able to update the context and use it to make predictions and consolidate context-target associative learning. Indeed many studies have extensively shown the role of contextual and social factors in in stimulus recognition (e.g., [3]). However, not much is known about the impact of these biases on specific aspects of human cognition, such as memory processes. While walking through the streets of our city, we encounter hundreds of people, and although we will easily forget some of them, others remain etched in our memory. Why are some faces remembered better than others?
A strong indicator of how and when a face is remembered is face typicality [4]. Unusual faces with atypical features are better remembered than more typical faces [5], even if less attractive. Another component that affects facial memory is perceived trustworthiness. Some studies suggest that faces perceived as the most untrustworthy are remembered significantly more than those perceived as reliable [6]. However, in these experiments, characters were perceived as unreliable only on the basis of perceptual appearance. In a study manipulating social traits, Mealey and colleagues [7] associated a face with some fictional personal traits (irrelevant history, history of unreliability, history of cheating) of the person depicted in the photograph. A week later, the experimental subjects were presented the old as well as new faces, and the task was to recognize which one had been previously encountered. The results showed a higher percentage of recognition of persons described as crooks in the learning phase. Generally, negative emotionally-valenced stimuli are remembered better than positive or neutral stimuli. For example, Ochsner [8] reported better performance in an old/new task for faces with a negative facial expression vs. a positive or a neutral face, whereas another study [9], in which both faces and scenes were used as experimental stimuli, showed that greater recognition of stimuli with negative valence occurred compared to those with neutral valence. The reason for this advantage might be due to stronger memory traces for aversive stimuli that might involve associations with amygdala and limbic structures. Indeed, investigations of emotion’s impact on memory have shown that emotion enhances memory [10–12] and that this enhancement is associated with increased engagement of amygdala, hippocampus, parahippocampus, and medial temporal lobe regions [13]. The enhanced activity in these areas has been observed during both encoding [14–17] and retrieval [18–20].
However, a dissociation has been shown between the sensation of knowing something/someone (familiarity) and recall, which is associated with the ability to retrieve the details of the learning situation (where and when that person has been met, for example). This dissociation was first demonstrated by a “remember-know” (R/K) paradigm introduced by Tulving [21]. This paradigm involves a learning phase of a given stimulus material, followed by a memory session in which participants decide whether they "remember" the material and are able to evoke specific qualitative information associated with it or just "know" the stimulus that is familiar to them without the ability to recall specific details. The idea behind this dissociation is that "remembering" reflects the process of recollection, characterized by the conscious recovery of details about the spatiotemporal context associated with stimulus learning. However, "knowing" might be considered an index of stimulus familiarity because it is characterized by the simple sensation of having seen/encountered an item, with no further recovery information related to it [22]. Many ERP studies have provided evidence of a dissociation between familiarity and recollection [23–28]. Research has shown that recollection is associated with an increased positivity at parietal electrode sites in between 600–900 ms (also known as late positive complex, LPC, or parietal old/new effect). The parietal old-new effect is more substantial for the items recognized as old and correctly associated to their context of study rather than old stimuli that are studied but not properly assigned to their original context of study [29–33]. In contrast, stimulus familiarity is associated with an increased frontal FN400 response (300–600 ms) that would originate in one or more regions of the prefrontal cortex. Its amplitude is reduced or attenuated in response to stimuli already seen or studied; therefore, it is inversely related to stimulus familiarity. The response most likely indicates the coding of the prefrontal cortex for new material.
In an interesting behavioral study, Bell and Buchner [34] showed that 72 faces were accompanied by a description of neutral, negative (disgusting) or positive traits. The study phase was followed by a face recognition task. The results showed that although the old/new discrimination task was not influenced by the type of descriptive information provided with the face (positive, neutral or negative), source memory for faces associated with disgusting behavior (i.e., memory for the disgusting context in which the face was encountered) was consistently better than source memory for other types of faces. The authors concluded that there is a dissociation between recollection and source memory with regard to memory for faces associated with negative or threatening contexts that may be instrumental in avoiding the negative consequences of encounters with persons associated with negative or threatening behaviors. A similar pattern was found a few years earlier in another study by Buchner and coworkers [35].
The aim of the present study was to investigate the effect of a negative social bias on memory for faces. A short fictional story was presented to participants in association with a male or female face in an encoding session; memory recollection of faces was immediately tested in an old/new face recognition paradigm. The effect of memory encoding per se (old vs. new discrimination) was observed by considering ERPs to positively biased faces, whereas the effect of negative prejudice was ascertained by comparing the effects of positive vs. negative bias on memory recognition. A validation pre-test on an independent group of judges showed that both positive and negative biases significantly affected faces trustworthiness perception. Our assumption was that although both positive and negative biases actively modulated face processing, brain response to negative faces would differ from positive faces. We also expected to find a dissociation between the conscious effect of recollection indexed by LP parietal effect (known as old/new parietal effect) and an implicit familiarity effect, dependent on the strength of memory traces and indexed by frontal FN400 modulation.
Materials and Methods
Participants
Seventeen right-handed people (6 males and 11 females) aged 22 to 28 years old participated in the study. All participants were right-handed with normal or corrected vision. The handedness was assessed through the administration of the Italian version of the Edinburgh Inventory Questionnaire. We also assessed the lack of any brain damage, previous history of epilepsy or current medications that could affect the electrical activity of the brain. The experiment was conducted with approval from the Ethical Committee of University of Milano-Bicocca and in compliance with APA ethical standards for the treatment of human volunteers (1992, American Psychological Association). Informed written consent was obtained from all subjects. All participants received academic credit for their participation. All experiments were performed in accordance with relevant guidelines and regulations.
Stimuli
The material consisted of 300 photographs of close-ups of unfamiliar Caucasian-like faces of various ages (150 men and 150 women), collected from various free-photo web sites. Each image had a size of 350 x 438 pixels and was colored and at high resolution. 200 photographs (100 men and 100 women) were used in the encoding task (therefore named “old” faces), whereas the other 100 faces (“new” faces) were used in a subsequent recognition task. In the encoding task, faces were presented in association with a short story (1 sentence) that provided fictional information on the character, such as an anecdote or personal information. As seen from the examples provided in Table 1, the biographic information could be positive, thereby inducing a positive prejudice toward the depicted character or vice versa, a negative prejudice could induce a negative bias. More precisely, 100 photographs (50 men and 50 women) were matched with a positive description relative to personality traits or specific biographical events (POS), and the remaining 100 (50 men and 50 women) were characterized negatively (NEG). The other 100 faces, 50 men and 50 women, were not accompanied by verbal descriptions and were used as novel faces for the memory recognition task (NEW). As displayed in Table 2, faces for the encoding task were carefully balanced by sex, age, facial expression valence across the two categories (negative and positive prejudice). The possible effect of face age, sex or expression on ERPs was not investigated, because it was annulled across classes. Indeed, the aim of the study was to understand the effect of prejudicial bias on face processing, regardless of these variables accurately controlled for.
The degree of positivity or negativity of verbal description was balanced across the 4 stimulus categories based on the scores obtained in a stimulus validation procedure. Stimuli were validated by administering a social trust test to 20 independent judges (10 women and 10 men ranging in age from 25 to 60 years). Faces were shown to judges for a few seconds, one by one, using a PowerPoint presentation and accompanied by a short verbal description appearing below. Judges were asked by the experimenter to evaluate the degree of trust inspired by each face by considering the following question: would you trust this person enough to leave your baby nephew or your home keys in their hands? The scale ranged from 1 to 3, from 1 = No, I do not trust this person at all; 2 = I do not know; 3 = Yes, this person inspires confidence.
Using this procedure, 45 faces or descriptions were changed because they were neutral according to at least 75% of judges and were re-tested. The overall scores obtained by each stimulus underwent a repeated measure ANOVA, which demonstrated the efficacy of verbal description in inducing a significantly negative (1.46, SE = 0.04) or positive (2.63, SE = 0.03) bias (F [1,98] = 136, p<0.00001) in both male and female characters (face gender effect was not significant).
Stimulus luminance was individually measured by using a Minolta CS 100 Luminance meter, and a repeated measures ANOVA proved their equiluminance across categories (men POS = 20.56 (SE = 1.19), Men NEG = 21.90 (SE = 1.19), Men NEW = 20.20 (SE = 1.19), Women POS = 21.49 (SE = 1.19), Women NEG = 22.3 (SE = 1.19), Women NEW = 19 (SE = 1.19) Foot-lamberts).
The length (# of words) and number of letters of verbal description of POS and NEG faces was also balanced across classes, as proved by a lack of a statistically significant result (# of words: Men POS = 8.1 (SE = 0.25), Men NEG = 8.08 (SE = 0.25), Women POS = 8.60 (SE = 0.25), Women NEG = 9.14 (SE = 0.25); # of letters: Men POS = 39.56 (SE = 1), Men NEG = 40.44 (SE = 1), Women POS = 43.6 (SE = 1), Women NEG = 44.49 (SE = 1).
Procedure
The participants sat in a dark test cubicle that was acoustically and electrically shielded (Faraday cage) in front of a computer screen that was placed at 114 cm from the participants' eyes. The subjects were instructed to gaze at the center of the screen (where a small permanent green cross served as a fixation point) and avoid any eye or body movement during the recording session. The stimuli were presented randomly and mixed (as age, sex, emotional valance, prejudice valance) at the center of the screen in eight different blocks lasting approximately 5 min and preceded by 3 warning signals ‘‘ready”, ‘‘set”, ‘‘go” presented for 500 ms. In the encoding task, sentences were typed in lower case Arial Narrow font and were white on a dark grey background. Each sentence was presented for 3000 ms, and it was followed by a 500 ms interstimulus interval (ISI). Then, the associated face was presented, which lasted for 2000 ms. Each pair of descriptions/face stimuli was followed by the “NEXT” screen that lasted 1 sec (please see Fig 1 for a description of the paradigm) that clearly signaled the presentation of a new pair. In the recognition task, each face was presented for 800 ms with a random ISI varying from 1300–1400 ms. The task consisted of pressing a button with the index finger in response to old faces and with the middle finger in response to new faces as accurately and fast as possible. The left and right hands were used alternately throughout the recording session, and the order of the hand and task conditions were counterbalanced across the subjects. The experiment was preceded by a training phase in which 10 faces and fictional stories that were not included in the experiment were presented. Participants were explicitly told that they had to pay attention to the faces to recall them in the recognition task.
EEG Recording and Analysis
EEG data were recorded continuously by using the EEProbe system (ANT Software, Enschede, The Netherlands) from 128 scalp sites according to the 10–5 international system [36] at a sampling rate of 512 Hz. Averaged mastoids served as the reference lead. Horizontal and vertical eye movements were also recorded. The EEG and electrooculogram (EOG) were amplified using a half-amplitude band pass of 0.016–100 Hz. Electrode impedance was maintained below 5 kΩ. EEG epochs were synchronized with the onset of the stimulus presentation. Computerized artifact rejection was performed to discard epochs in which eye movements, blinks, excessive muscle potentials, or amplifier blocking occurred. The artifact rejection criterion was a peak-to-peak amplitude that exceeded 50 μV, which resulted in a rejection rate of ~5%. The -100-0 ms time range served as reference for baseline correction. Event-related potentials (ERPs) from 100 ms before (−100 ms) to 1000 ms after face stimuli onset were averaged. ERP epochs associated with behaviorally incorrect responses or with discarded RTs were also rejected. ERP components were identified and measured if they reached their maximum amplitude (and in agreement with previous ERP literature) with respect to the average baseline voltage. The mean area amplitude of ERP components of interest was quantified at the anterior frontal and prefrontal sites (AF7, AF8, FP1, FP2) in the time window 600–800 ms, corresponding to anterior negativity (AN) in the Encoding task. The anterior and inferior frontal sites (AF3, AF4, F5, F6) between 400–600 ms and the parietal sites (P1, P2) between 450–550 ms corresponded to the FN400 and “Old-New” LPC parietal response, respectively, in the recognition task. Multifactorial repeated-measures ANOVAs were applied to the amplitude data. The factors were prejudice valence (POS, NEG) and electrode (anterior frontal, prefrontal) hemisphere (left or right) in the encoding task and face type (POS, NEG, NEW), electrode (varying according to the ERP component, see above) and hemisphere (left, right) in the recognition task. Multiple post-hoc mean comparisons were performed using Tukey's test.
swLORETA Source Localization
Low-resolution electromagnetic tomography (LORETA) was applied to ERPs recorded during the recognition task for the processing of the 3 types of faces (NEG; POS, NEW) in the 450–550 ms time window. LORETA [37] is a discrete linear solution to the inverse EEG problem and corresponds to the 3D distribution of neuronal electrical activity that has maximally similar (i.e., maximally synchronized) orientation and strength between neighboring neuronal populations (represented by adjacent voxels). In this study, an improved version of the standardized weighted LORETA was used [38]. This version, called swLORETA, incorporates a singular value decomposition-based lead field-weighting method. The source space properties included a grid spacing (the distance between two calculation points) of five points (mm) and an estimated signal-to-noise ratio (which defines the regularization of three; a higher value indicates less regularization and therefore less blurred results) of three. Using a value of 3–4 for the computation of SNR in the Tikhonov's regularization produces superior accuracy of the solutions for any inverse problems. swLORETA was performed on the group data (grand-averaged data) to identify statistically significant electromagnetic dipoles (p<0.05) in which larger magnitudes correlated with more significant activation. The data were automatically re-referenced to the average reference as part of the LORETA analysis. A realistic boundary element model (BEM) was derived from a T1-weighted 3D MRI dataset through segmentation of the brain tissue. This BEM model consisted of one homogeneous compartment comprising 3446 vertices and 6888 triangles. Advanced Source Analysis (ASA) employs a realistic head model of three layers (scalp, skull, and brain) created using the BEM. This realistic head model comprises a set of irregularly shaped boundaries and the conductivity values for the compartments between them. Each boundary is approximated by a number of points, which are interconnected by plane triangles. The triangulation leads to a more or less evenly distributed mesh of triangles as a function of the chosen grid value. A smaller value for the grid spacing results in finer meshes and vice versa. With the aforementioned realistic head model of three layers, the segmentation is assumed to include current generators of brain volume, including both gray and white matter. Scalp, skull, and brain region conductivities were assumed to be 0.33, 0.0042, and 0.33, respectively. The source reconstruction solutions were projected onto the 3D MRI of the Collins brain, which was provided by the Montreal Neurological Institute. At this regard, it should be noted that the realistic head model in LORETA may affect source localization results due to individual differences in head and brain shape. The probabilities of source activation based on Fisher's F-test were provided for each independent EEG source, and their values are indicated by a “unit” scale (the larger, the more significant). Both the segmentation and generation of the head model were performed using the ASA software program Advanced Neuro Technology (ANT, Enschede, Netherlands).
Results
Behavioral Results
Reaction times (RTs) that exceeded the mean value ±2 standard deviations were discarded, which resulted in a rejection rate of 2%. Both RTs and hit percentages were subjected to separate multifactorial repeated-measures ANOVAs with one factor of variability: face type (POS, NEG, NEW). The ANOVA performed on RTs yielded the significance of face type (F [2, 22] = 3.949; p<0.03; ε = 0.66; Greenhouse Geyser corrected p = 0.05; η2 = 0.354). Post-hoc comparisons indicated faster responses to OLD than NEW faces (POS: 716 ms, SE = 15; NEG: 723 ms, SE = 17.6; NEW: 750 ms, SE = 17.18), with no effect of bias. The ANOVA performed on hit percentages (old faces correctly recognized as old, or vice versa) also yielded the significance of face type (F [2, 22] = 12.905; p<0.000196; ε = 0.61; Greenhouse Geyser corrected p = 0.00217; η2 = 0.53). Post-hoc comparisons indicated a higher recognition rate (p<0.002) for new than old faces (NEW: 86.55 S.E. = 3.13%; POS = 68.79%, SE = 2.12; NEG: 68.67, SE = 2.15), with no effect of bias. D-prime values (d’) were = 1.923 for new faces, 0.911 and 0.908 for positively and negatively biased faces, respectively. Overall prejudice did not significantly affect recollection accuracy.
Electrophysiological Data
The ERPs recorded from high density montage (128 channels) during the encoding task are displayed in Fig 2. It is important to mention that any difference between the two class of ERPs to faces depended on having just read a positive or negative comment on the person depicted.
The effect of prejudice affected face processing over prefrontal areas during coding.
Encoding task.
Fig 3 shows grand average ERPs recorded at various anterior, central, parietal and posterior sites during the encoding session. Faces belonging to the two classes (positive and negative prejudice) elicited identical ERPs at occipito/temporal sites where sensory and configuration analysis was conducted because of careful balancing of their sensory and semantic characteristics (age, sex, emotional valence of facial expression). However, an increased late negativity during processing of negatively biased faces was observed and quantified at anterior scalp sites.
A long lasting, late anterior negativity during the processing of negatively biased faces was observed, and no difference at P1 or N170 processing level, due to careful interclass balancing was noted.
Anterior negativity (600–800 ms). The ANOVA results showed an effect of prejudice valence (F [1, 11] = 5.482, p<0.05; η2 = 0.33) with larger responses to faces belonging to the NEG (0.64 μV, SE = 0.57) than POS (1.52 μV, SE = 0.78) category.
Recognition task.
Fig 4 show the ERPs recorded from 128 scalp sites during the recognition task. Large memory effects are visible at anterior (FN400) and parietal sites (Parietal Old/New effect). The effect of prejudice can be appreciated at anterior sites, with a clear gradient in FN400 amplitude from the less familiar to the most familiar items.
The effect of prejudice affected face processing over prefrontal areas during recognition.
FN400 (400–600 ms). The ANOVA performed on FN400 responses recorded at anterior sites revealed a strong effect of face type, with larger potentials to new than old items (F [2, 22] = 7.7973, p<0.005; η2 = 0.41). Post-hoc comparisons indicated greater FN400 responses to NEW (-0.92 μV, SE = 1.74) than POS (0.20 μV, SE = 1.67) faces (p<0.01), and to POS than NEG faces (0.83 μV, SE = 1.59; p<0.01), as evidenced by the ERP waveforms displayed in Fig 5. Furthermore, the significant interaction of face type x hemisphere (F [2, 22] = 4.9964, p<0.01; η2 = 0.31) showed larger FN400s to new faces over the right than left hemisphere (p<0.01), with no other hemispheric asymmetry (NEG LH: 0.70 μV, SE = 1.52; RH: 0.96 μV, SE = 1.67. POS LH: 0.24 μV SE = 1.62; RH: 0.15 μV SE = 1.73; NEW LH: -0.55 μV, SE = 1.68; RH: -1.29 μV, SE = 1.82 μV).
A gradient of familiarity is visible at anterior sites, indicating an enhanced processing of new vs. old, and of positive vs. negative faces, possibly negatively correlated to the strength of face memory traces. Recognition of familiar faces (OLD) resulted in the typical parietal old/new effect, in the form of an enhanced late positivity.
Parietal Old/New effect (LPC 450–550 ms). Face type also significantly affected the parietal LPC which was greater in response to old than new items (F[2, 22] = 5.02, p<0.01; η2 = 0.31), especially over the RH, with no effect of social prejudice on face recollection (NEG LH: 3.12 μV, SE 0.96, RH: 3.16 μV, SE = 0.90; POS LH: 3.12 μV, SE = 1.13; RH: 3.28 μV, SE = 1.14; NEW LH: 2.63μV, SE = 1.07; RH: 2.37 μV, SE = 0.99).
LORETA source reconstruction. To identify the intracranial neural generators of surface potential, an inverse solution swLORETA was applied to ERPs recorded during the recognition task for the processing of the 3 types of faces (NEG; POS, NEW) in the time window corresponding to the recollection (LPC) and familiarity (FN400) effects (450–550 ms). Table 3 shows the Talairach coordinates (in mm) corresponding to the active sources, in addition to their magnitude (in nAm). Recall of faces (old vs. new) was associated with activity in the right superior frontal cortex, left medial temporal gyrus (BA 39) and right fusiform gyrus (FG, BA37, also called “fusiform face area”). However, recall of negative faces was uniquely associated with activations in the left superior frontal gyrus (BA10), left posterior cingulate cortex (BA20) and left parahippocampal gyrus (BA30) of the limbic system (see Figs 6 and 7 for a comparison between the three conditions). The right cingulate cortex (BA30/32) and right occipital areas, such as the inferior occipital gyrus (BA18/19) and the right precuneus, were commonly activated during the perception of old and new faces regardless of prejudice valence, therefore indicating an effect of face processing and not memory.
Magn. = Magnitude in nAm; H = hemisphere, BA = Brodmann areas.
The different colors represent differences in the magnitude of the electromagnetic signal (in nAm). The electromagnetic dipoles are shown as arrows and indicate the position, orientation and magnitude of dipole modeling solution applied to the ERP waveform in the specific time window. Numbers refer to the displayed brain slice in sagittal view: A = anterior, P = posterior. The images highlight the strong activation of the left medial temporal and parahippocampal gyri during memory recall of faces associated with a negative prejudice.
The images highlight the strong activation of the left medial frontal gyrus during memory recall of faces associated with a negative prejudice.
Discussion
The aim of the study was to investigate the effect of negative prejudicial information associated with a face on memory processes. Therefore, we presented fictional information relative to 200 male and female faces during an encoding task. After approximately 30 minutes, memory for faces was tested with an old/new paradigm involving 100 new faces. Correct response percentage was approximately 75%, and it was easier to correctly recognize a face as new (87%) than old (69%). However, RTs were much faster to old than new faces, regardless of prejudice valence. Overall, the prejudice did not improve recollection of negative vs. positive faces after 30 minutes in the encoding session. The lack of effect at the behavioral level is consistent with other studies that did not find an effect of prejudicial bias on old/new face recognition [34,35]. Therefore, the hypothesis can be extended; negative bias enhanced memory consolidation by involving emotional associative areas. In a very recent and interesting fMRI study [39] during encoding of emotionally arousing information, a robust increase in effective connectivity from the amygdala to the hippocampus was observed, regardless of stimulus valence (both positive and negative).
Indeed, stimuli of our study were created in a way that positive bias significantly modulated face trustworthiness, thereby improving the ability to recall faces and their learning context. However, ERP data provided evidence of a dissociation between familiarity and recollection, with an effect of valence on FN400 but not LPC amplitude, in ERPs recorded during the recognition test.
Because faces were carefully balanced for sensory and perceptual characteristics as well as for sex, age, and facial expression (see [40] for the effects of face age on N170 amplitude), prejudicial bias did not affect the amplitude of N170 response, which was thought to index the encoding of structural face properties [41,42]. The first effect of prejudice valence (during the encoding session) was found at the anterior frontal and prefrontal areas, between 600 and 800 ms after face presentation. It is interesting to note that ERPs were time locked to face onset because the verbal description (fictional story) was presented 3.5. sec in advance. Therefore, this effect (increase in anterior negativity) might index an additional processing of face-related information associated with a negative context (negative social traits). The role of the prefrontal cortex in episodic memory has been widely documented [43], whereas anterior negativity has been clearly related to stimulus complexity, load and working memory span [44,45]. Our data suggest that negative faces were encoded more deeply (as typical of aversive stimuli), which explains the difference in FN400 observed in the recognition test for NEG vs. POS faces. Indeed, anterior FN400 was both greater to new than old items, as expected based on the ERP literature on FN400 familiarity effects [31], and greater to POS than NEG stimuli as well. This finding might indicate that POS stimuli were less familiar than NEG stimuli because they were more superficially coded during the encoding session. Specifically, a dissociation was found between familiarity and recollection (indexed by LPC, i.e., the parietal/old new effect). Indeed, LPC was larger in response to old than new items, but it was not modulated by bias valence, which highly correlated with behavioral data. A similar correlation between behavioral performance and the parietal ERP effect was found in an ERP study involving memory for words and their source with an old/new paradigm [46]. Again, in strong agreement with our study comes a recent ERP investigation on memory [47] in which it was compared neural processing of faces associated with negative, positive or neutral contexts (cheating, cooperation, or neutral behavior in a social-dilemma game). Quite similarly to our study, it was found a frontal correlate of context valence and a parietal old-new response (which was not modulated by valence) in response to the faces presented in the memory test. Furthermore, there was no effect on early correlates of face processing such as the N170, which is consistent with the idea that the N170 is an index of perceptual face processing, not being much affected by memory factors. Overall, the present ERP pattern of results agrees with current knowledge about recollection/familiarity dissociation [48,27,28]and provides interesting information about the processing of aversive social information, which implicitly affected stimulus familiarity.
Our source reconstruction data obtained by applying swLORETA to scalp potentials recorded during the maximum amplitude of familiarity and recalling effects (450–550 ms) support our hypothesis that NEG faces were coded more robustly during the learning phase (as also evidenced by the enlarged prolonged anterior negativity elicited by NEG vs. POS faces during that experimental session).
Overall, remembering faces (old vs. new contrast) was associated with significant source activity in the right superior frontal cortex (BA10). Right prefrontal activity has been suggested to characterize episodic retrieval mode by many influential studies [49,50]. Another common activation by old but not new faces (regardless of valence) occurred in the left medial temporal gyrus (MTG, BA39). Neuroimaging studies also in addition to studies on amnesic patients have revealed that the regions of the medial temporal lobe have a crucial role in episodic memory and recognition memory [51], [52,53,54]. The left hemispheric asymmetry of MTG for episodic memory for people is consistent with previous investigations, suggesting a left hemisphere role in memory for people [55]. Old (vs. new) face processing was also associated with strong activity in the right fusiform area of the temporal gyrus, also named FFA. The role of this region in face configuration processing is well documented [56], in addition to the involvement of the right prefrontal cortex in face processing [57–60]. Because both areas were only activated during processing of old (and not new) faces, as the areas may be possible neural loci for storing face-related episodic memory information.
Remembered (OLD) and non-remembered (NEW) faces shared the activation of lateral occipital areas (BA18/19), possibly indicating the activity of the occipital face area (OFA) during face processing. Similar to FFA, OFA would also be involved in face configuration analysis [56] but would be more responsive to local details (e.g., the eyes, the mouth, the nose [61] than the holistic pattern). Remembered and non-remembered faces also shared the activation of the posterior cingulate cortex (involved in the emotional processing of visual images [62] and empathy [63] and the right precuneus. In regard to the effect of bias prejudice, processing of NEG vs. POS faces was associated with increased activity in the left superior frontal gyrus (BA10), left posterior cingulate cortex (BA20) and left parahippocampal gyrus (BA30) of the limbic system. Overall, brain activation was higher during processing of NEG vs. POS faces, therefore indicating a stronger recognition response to aversive (than positive) social information [10–13]. Indeed, the left lateral prefrontal cortex is involved in the processing of familiarity [64] and episodic recall [65]; whereas parahippocampal regions are thought to be involved in the recovery of memory contextual information [66–68]. Overall, a dissociation was found between face recollection and accuracy as evidenced by the parietal old/new effect, and face familiarity (indexed by FN400 component) showed a reduced response to NEG than POS or NEW faces. The enhanced anterior negativity during coding of NEG vs. POS faces in the learning session is consistent with FN400 behavior in the recognition session, as well as with the finding of enhanced brain activation in prefrontal, parahippocampal and limbic areas during recall of negative faces.
The results of our study, and particularly that negative social bias did not modulate perceptual N170 response to faces in the recognition task are in apparent contradiction with some ERP investigations showing how affective information and context can indeed modulate N170 component. However it should be considered that a crucial difference exists between our study and other experimental designs. In these studies, the context and prejudicial information was co-existent or pre-existent, or even rooted in subjects’ mind/brain while they perceived faces (thus affecting face processing), while in our study the random association between a face and a verbal description had to be specifically remembered, otherwise it was not available. For example, in a study by Hietanen and Astikainen (2013) [69], in which ERPs were recorded to happy and sad facial expressions preceded by positive and negative scenes (which acted as primes) faces were presented 300 ms after the prime, therefore the context was almost simultaneous to the faces. In another investigation by Ibanez et al.’s (2010) [70] N170 response was influenced by IAT in-group vs out-group effects (i.e., social prejudices possessed by subjects), therefore each face carried the prejudicial information (race), and nothing had to be remembered. Again, in another ERP study by Martínez-Galindo and Cansino (2015) [71], in which participants performed a betting-game task while the faces of their virtual opponents were presented in each trial, the N170 component was more negative for faces encoded in positive contexts (while winning a bet) than for those encoded in non-emotional contexts, but the general effect of context was not face-specific, and, according to the authors, it could not be excluded that the context might have modified the overall cerebral arousal. This effect is impossible in the present study, in that positive and negative prejudices were presented during the same experimental run, randomly intermixed. Last but not least, it would be mentioned that, in our knowledge, the memory literature about old/new effects has not shown a memory modulation for N170 response.
Our findings are strongly in agreement with previous fMRI data showing the role of the medial frontal cortex in representing social information referred to others, particularly outgroup stereotyping and prejudice [72]. They are also in absolute agreement with a quite recent ERP investigation testing memory for objects and showing a dissociation between FN400 and late old/new parietal effects, thought to index stimulus familiarity and memory recollection, respectively. In this interesting study a familiarity effect was found at mid-frontal level, which was dissociated by the (missing) parietal effect and failure in recollection [73]. This finding may further explain why, in our study, brain signatures of memory for negatively biased faces (indexed by FN400 in the recognition task and prefrontal activity in the learning phase) were dissociated by performance in recollection and LP amplitude, with specific regard to the lack of positive/negative gradient.
Study Limits
It would have been interesting to compare the coding of faces provided with valenced vs. neutral information, or valenced vs. without any additional information, and these comparisons should be investigated in future studies.
Currently, we cannot exclude an effect of prejudicial bias on memory recollection (in terms of accuracy or brain activation) in long-term memory (LTM). Indeed, in our study memory retrieval occurred 30 minutes after the learning session, whereas the long term effects of aversive episodic memory would have been observed, for example after a week [7] or 6 months after learning. This issue certainly deserves further investigation. Furthermore, the task was probably very difficult, because of the large number of faces to be remembered.
Because of the intrinsic nature of bioelectrical potentials, it is almost impossible to detect the activity of small subcortical nuclei, such as the amygdala, in source reconstruction analysis. This factor does not exclude its crucial role in aversive memory consolidation that has been widely shown in neurometabolic and cell recording neurophysiological studies.
Acknowledgments
Supported by 2012FAR grant from University of Milano-Bicocca to AMP. We wish to thank Federica Riva and Roberta Adorni for their help with the EEG recordings.
Author Contributions
- Conceptualization: AMP.
- Data curation: FL.
- Formal analysis: AMP, FL.
- Funding acquisition: AZ.
- Investigation: FL.
- Methodology: AMP, FL.
- Project administration: AMP.
- Resources: AZ.
- Supervision: AMP.
- Validation: AMP.
- Visualization: AMP, AZ, FL.
- Writing – original draft: AMP.
- Writing – review & editing: AMP, AZ.
References
- 1.
Allport G. The nature of prejudice. Perseus Books Publishing; 1979.
- 2. Ibáñez A, Manes F. Contextual Social Cognition and the Behavioral Variant of Frontotemporal Dementia. Neurology. 2012; 78(17):1354–62. pmid:22529204
- 3. Barrett LF, Bar M. See it with feeling: affective predictions during object perception. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 2009; 364: 1325–1334. pmid:19528014
- 4. Vokey JR, Read JD. Familiarity, memorability, and the effect of typicality on the recognition of faces. Mem Cognition. 1992; 20: 291–302.
- 5. Light LL, Kayra-Stuart F, Hollander S. Recognition memory for typical and unusual faces. J Exp Psycho- Hum L. 1979; 5: 212–228.
- 6. Rule NO, Slepian ML, Ambady N. A memory advantage for untrustworthy faces. Cognition. 2012; 125: 207–218. pmid:22874071
- 7. Mealey L, Daood C, Krage M. Enhanced memory for faces of cheaters. Ethol Sociobiol. 1996; 17: 119–128.
- 8. Ochsner KN. Are affective events richly recollected or simply familiar? The experience and process of recognizing feelings past. J Exp Psychol: General. 2000; 129: 242–261.
- 9. Keightley ML, Chiew KS, Anderson JAE, Grady CL. Neural correlates of recognition memory for emotional faces and scenes. Soc Cogn Affect Neur. 2011; 6(1): 24–37.
- 10. Bradley M M, Greenwald M K, Petry MC, Lang PJ. Remembering pictures: pleasure and arousal in memory. J Exp Psychol: Learn MemCogn. 1992; 18: 379–390.
- 11. Christianson SA. Emotional stress and eyewitness memory: a critical review. Psychol Bull. 1992; 112: 284–309. pmid:1454896
- 12. Chiu YC, Dolcos F, Gonsalves BD., Cohen NJ. On opposing effects of emotion on contextual or relational memory. Frontiers Psychol. 2013; 4: 103.
- 13. Dolcos F, Denkova E, Dolcos S. Neural correlates of emotional memories: a review of evidence from brain imaging studies. Psychologica. 2012; 55: 80–111.
- 14. Dolcos F, LaBar KS, Cabeza R. Interaction between the amygdala and the medial temporal lobe memory system predicts better memory for emotional events. Neuron. 2004; 42: 855–863. pmid:15182723
- 15. McGaugh JL. The amygdala modulates the consolidation of memories of emotionally arousing experiences. Annual Rev Neurosci. 2004; 27: 1–28.
- 16. Dolcos F, Denkova E. Neural Correlates of Encoding Emotional Memories: A Review of Functional Neuroimaging Evidence. Cell Science Reviews. 2008; 5(2): 78–122.
- 17. Murty VP, Ritchey M, Adcock RA, LaBar KS. Reprint of: fMRI studies of successful emotional memory encoding: a quantitative meta-analysis. Neuropsychologia. 2011; 49: 695–705. pmid:21414466
- 18. Sharot T, Yonelinas AP. Differential time-dependent effects of emotion on recollective experience and memory for contextual information. Cognition. 2008; 106: 538–547. pmid:17451666
- 19. Sergerie K, Lepage M, Armony JL A process-specific functional dissociation of the amygdala in emotional memory. J Cogn Neurosci. 2006; 18: 1359–1367. pmid:16859420
- 20. Smith AP, Stephan KE, Rugg MD, Dolan RJ. Task and content modulate amygdala-hippocampal connectivity in emotional retrieval. Neuron. 2006; 49: 631–638. pmid:16476670
- 21. Tulving E. Memory and consciousness. Canad J Psychol. 1985; 32: 130–147.
- 22. Skinner EI, Fernandez MA. Neural correlates of recollection and familiarity: a review of neuroimaging and patient data. Neuropsychologia. 2007; 45(10): 2163–2179. pmid:17445844
- 23. Addante RJ, Ranganath C, Yonelinas AP. Examining ERP correlates of recognition memory: evidence of accurate source recognition without recollection. Neuroimage. 2012; 62: 439–450. pmid:22548808
- 24. Evans LH, Wilding EL. Recollection and familiarity make independent contributions to recognition memory. J Neurosci. 2012; 32(21): 7253–7257. pmid:22623670
- 25. Henson RN, Rugg MD, Shallice T, Josephs O, Dolan RJ. Recollection and familiarity in recognition memory: an event-related functional magnetic resonance imaging study. J Neurosci. 1999; 19(10): 3962–3972. pmid:10234026
- 26. Rugg MG, Schloerscheidt AM, Doyle MC, Cox CJC, Patching GR. Event-related potentials and the recollection of associative information. Brain Res Cogn. 1996; 4: 297–304.
- 27. Wilding EL, Rugg MD. An event-related potential study of recognition memory with and without retrieval of source. Brain. 1996; 119: 889–905. pmid:8673500
- 28.
Wilding EL, Sharpe H. Episodic memory encoding and retrieval: Recent insights from event-related potentials. In The Cognitive Electrophysiology of Mind And Brain (Eds. Zani A. and Proverbio A.M.). San Diego, CA: Academic Press, 169–196; 2003.
- 29. Curran T. Brain potentials of recollection and familiarity. Mem Cognition. 2000; 28(6): 923–938.
- 30. Curran T, Cleary AM. Using ERPs to dissociate recollection from familiarity in picture recognition. Brain Research Cogn. 2003; 15: 191–205.
- 31. Curran T, Hancock J. The FN400 indexes familiarity-based recognition of face. Neuroimage. 2007; 36(2): 464–471. pmid:17258471
- 32. Duarte A, Ranganath C, Winward L, Hayward D, Knight RT. Dissociable neural correlates for familiarity and recollection during the encoding and retrieval of pictures. Brain Research Cogn. 2004; 18: 255–272.
- 33. Guillaume F, Tiberghien G. Impact of intention on the ERP correlates of face recognition. Brain Cogn. 2013; 81: 73–81. pmid:23174431
- 34. Bell R., Buchner A. Valence modulates source memory for faces. Mem Cognition. 2010; 38: 29–41.
- 35. Buchner A, Bell R, Mehl B, Musch J. No enhanced recognition memory but better source memory for faces of cheaters. Evol Hum Behav. 2009; 30: 212–224.
- 36. Oostenveld R, Praamstra P. The five percent electrode system for high-resolution EEG and ERP measurements. Clin Neurophysiol. 2001; 112: 713–719. pmid:11275545
- 37. Pasqual-Marqui RD, Michel CM, Lehmann D. Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. Int J Psychophysiol. 1994; 18: 49–65. pmid:7876038
- 38. Palmero-Soler E, Dolan K, Hadamschek V, Tass PA. SwLORETA. A novel approach to robust source localization and synchronization tomography. Phys Med Biol. 2007; 52: 1783–1800. pmid:17374911
- 39. Fastenrath M, Coynel D, Spalek K, Milnik A, Gschwind L, Roozendaal B, Papassotiropoulos A, de Quervain DJ. Dynamic modulation of amygdala-hippocampal connectivity by emotional arousal. J Neurosci. 2014; 34(42): 13935–47. pmid:25319690
- 40. Proverbio AM, Riva F, Zani A, Martin E. Is It a Baby? Perceived Age Affects Brain Processing of Faces Differently in Women and Men. J Cogn Neurosci. 2011, 23(11), 3197–3208. pmid:21557651
- 41. Bentin S, Allison T, Puce A, Perez E, McCarthy G. Electrophysiological studies of face perception in humans. J Cogn Neurosci. 1996; 8: 551–565. pmid:20740065
- 42. Bentin S, Deouell LY. Structural encoding and identification in face processing: ERP evidence for separate mechanism. Cogn Neuropsychol. 2000; 17 (1): 35–55. pmid:20945170
- 43. Bero AW, Meng J, Cho S, Shen AH, Canter RG, Ericsson M, Tsai L.H. Early remodeling of the neocortex upon episodic memory encoding. PNAS U S A. 2014; 111(32): 11852–7.
- 44. Ranganath C., Jhonson M.K., D’Esposito M. Left anterior prefrontal activation increases with demands to recall specific perceptual information. J Neurosci. 2000; 20(22): RC108. pmid:11069977
- 45. Vos SH, Gunter TC, Kolk HH, Mulder G. Working memory constraints on syntactic processing: an electrophysiological investigation. Psychophysiol. 2001; 38(1): 41–63.
- 46. Vallesi A, Shallice T. Prefrontal involvement in source memory: an electrophysiological investigation of accounts concerning confidence and accuracy. Brain Research. 2006; 1124(1): 111–25. pmid:17070783
- 47. Bell R., Sasse J., Möller M., Czernochowski D., Mayr S., Buchner A. Event-related potentials in response to cheating and cooperation in a social dilemma game. Psychophysiol. 2016; 53(2): 216–28.
- 48. Yonelinas AP. Receiver-operating characteristics in recognition memory: Evidence for a dual-process model. J Exp Psychol: Learning, Mem Cogn. 1994; 20: 1341–1351.
- 49. Lepage M, Ghaffar O, Nyberg L, Tulving E. Prefrontal cortex and episodic memory retrieval mode, PNAS USA. 2000; 97: 506–511. pmid:10618448
- 50. Nyberg L, Tulving E, Habib R, Nilsson LG, Kapur S, Houle S et al. Functional brain maps of retrieval mode and recovery of episodic information. Neuroreport. 1995; 7: 249–252. pmid:8742463
- 51. Davachi L, Mitchell JP, Wagner AD. Multiple routes to memory: Distinct medial temporal lobe processes build item and source memories. PNAS USA. 2003; 100(4): 2157–2162. pmid:12578977
- 52. Henson RN, Cansino S, Herron JE, Robb WG, Rugg MD. A familiarity signal in human anterior medial temporal cortex. Hippocampus. 2003; 13(2): 301–304. pmid:12699337
- 53. Squire LR, Wixted JT, Clark RE. Recognition memory and the Medial temporal lobe: a new perspective. Nature Rev Neurosci. 2007; 8(11): 872–883.
- 54. Shafer AT, Dolcos F. Dissociating retrieval success from incidental encoding activity during emotional memory retrieval, in the medial temporal lobe, Frontiers Behav Neurosci. 2014; 3;8: 177.
- 55. Proverbio AM, Lilli S, Semenza C, Zani A. ERP indexes of functional differences in brain activation during proper and common names retrieval. Neuropsychologia. 2001; 39(8): 815–27. pmid:11369405
- 56. Haxby JV, Hoffman EA, Gobbini MI. The distribuited human neural system for face perception. Trends Cogn Sci. 2000; 4(6): 223–233. pmid:10827445
- 57. Allison T, Ginter H, McCarthy G, Nobre AC, Puce A, Belger A. Face recognition in human extrastriate cortex. J Neurophysiol. 1994; 71: 821–825. pmid:8176446
- 58. Sergent J, Ohta S, MacDonald B. Functional neuroanatomy of face and object processing: a positron emission tomography study. Brain. 1992; 115: 15–36. pmid:1559150
- 59. Rossion B, Caldara R, Seghier M, Schuller AM, Lazeyras F, Mayer E. A network of occipital- temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing. Brain. 2003; 126(11): 2381–2395.
- 60. Rossion B, Hanseeuw B, Dricot L. Defining face perception areas in the human brain: a large scale factorial fMRI face localizer analysis. Brain Cognit. 2012; 79(2): 138–157.
- 61. Arcurio LR, Gold JM, James TW. The response of face-selective cortex with single face parts and part combinations. Neuropsychologia. 2012; 50(10): 2454–2459. pmid:22750118
- 62. Proverbio AM, Zani A, Adorni R. Neural markers of a greater female responsiveness to social stimuli. BMC Neurosci. 2008; 30(9): 56.
- 63. Schulte-Rüther M., Markowitsch HJ, Fink GR, Piefke M. Mirror neuron and theory of mind mechanisms involved in face-to-face interactions: a functional magnetic resonance imaging approach to empathy. J Cogn Neurosci. 2007; 19(8): 1354–1372. pmid:17651008
- 64. Yonelinas AP, Otten LJ, Shaw KN, Rugg MD. Separating the brain regions involved in recollection and familiarity in recognition memory. J Neurosci. 2005; 25: 3002–3008. pmid:15772360
- 65. Nolde SF, Jhonson MK, D’Esposito M. Left prefrontal activation during episodic remembering: An event-related fMRI study. Neuroreport. 9(15), 3509–3514 (1998). pmid:9855308
- 66. Eichembaum H, Yonelinas AR, Ranganath C. The medial temporal lobe and recognition memory (2007). Annual Rev. Neurosci. 2007; 30: 123–152.
- 67. Ranganath C, Yonelinas AP, Cohen MX, Dy CJ, Tom SM, D’Esposito M. Dissociable correlates of recollection and familiarity within the medial temporal lobes. Neuropsychologia. 2003; 42(1): 2–13.
- 68. Brown MW, Aggleton JP. Recognition memory: what are the roles of the Perirhinal cortex and Hippocampus? Nature Rev Neurosci. 2001; 2(1): 51–61.
- 69. Hietanen JK, Astikainen P. N170 response to facial expressions is modulated by the affective congruency between the emotional expression and preceding affective picture. Biol. Psychol. 2013; 92: 114–124. pmid:23131616
- 70. Ibáñez A, Gleichgerrcht E, Hurtado E, González R, Haye A, et al. Early neural markers of implicit attitudes: N170 modulated by intergroup and evaluative contexts in IAT. Front. Hum. Neurosci. 2010; 4, 188. pmid:21079750
- 71. Martinez-Galindo JG, Cansino S. Positive and negative emotional contexts unevenly predict episodic memory. Behav. Brain Res. 2015; 291:89–102. pmid:26003944
- 72. Mitchell J P, Macrae CN, Banaji MR. Dissociable medial prefrontal contributions to judgments of similar and dissimilar others. Neuron. 2006; 50: 655–663. pmid:16701214
- 73. Tsivilis D, Allan K, Roberts J, Williams N, Downes JJ, El-Deredy W. Old-new ERP effects and remote memories: the late parietal effect is absent as recollection fails whereas the early mid-frontal effect persists as familiarity is retained. Front. Hum. Neurosci. 2015; 9:532. pmid:26528163