Participants

The sample was composed of 47 university students, 24 women (M = 25.6 years; SD = 7.5) with normal or corrected to normal vision. The sample size was estimated by a power analysis performed in G*Power (^{Faul et al., 2007}) for a repeated-measures ANOVA, with a between-subjects factor for participants’ sex (women vs. men) with six measures in the within-subjects factors, emotional expression (happy, sad, neutral) and sex of faces (female vs. male faces). Considering a within-between interaction of effect size f(U) = .50 (η² = .20 as in SPSS), power (1 - β) = .95, and α = .05, a total sample size of 20 participants was estimated. For the correlation between accuracy and vividness judgment, the power analysis performed in GPower (Faul et al., 2007) indicated that a sample size of N = 30 yields a power of 1 -β = .80 (α = .05) to detect an effect of d _z = .50. This effect was based on Bona and Silvanto’s (²⁰¹⁴) study. The participants were recruited through local advertisements at the university. We ensured that participants met specific criteria to enter the study: they were not on medication or did not report any neurological or psychological problems.

Ethics

The present study was conducted in accordance with the national legislation regarding the assessment of human volunteers. The local University Research Ethics Committee approved the investigation of the present study (CAAE 68402117.5.0000.5407).

Material and Stimuli

The task was performed on a computer with a 21” monitor using the E-prime software (^{Schneider et al., 2002}). We used 21 pictures of the same female face and 21 pictures of the same male face from the Pictures of Facial Affect (POFA) (^{Ekman & Friesen, 1976}). There were 10 pictures of the same face with a different gradation of sad expression, 10 gradations with happy faces, and one neutral face. The VAS scale was a straight line of 8.30cm, with the extremes identified as “Less vivid” and “More vivid”.

Procedure

Participants performed a forced recognition task in which they were instructed to memorize a target face presented in the center of the screen for 200 milliseconds (ms). After a retention interval of 2000 ms, the participants judged the vividness of the initial face on a VAS scale. Following the vividness rating, two test faces were presented on the left and right sides of the screen. The participants had to decide which face was previously memorized between the two test faces. Feedback on the memory task was provided in each trial.

Participants judged the vividness of the target face by clicking on the continuous line scale with the mouse. The vividness judgment was registered in a 0-1 continuum. The participant gave the memory task response by clicking the left mouse button if the correct face was on the screen’s left side or by clicking the right mouse button if the correct face was presented on the screen’s right side. The target face was presented randomly between the faces-test on the left and right (Figure 1). The target face was chosen randomly.

The target face with an emotional expression of happiness or sadness always had an expression gradation between levels of three and seven. Among the two test faces presented, one was always the same as the memorized one, and the other was different by three gradations (up or down) from the emotion of the target face. For example, if the target face had a gradation of four, the other tested-face had a gradation of one or seven (Figure 1-A). In the case of the target face with level three, the other tested face was a neutral face. When the neutral emotion should be memorized, the tested face always had a gradation of four counterbalanced between happiness and sadness. The experiment consisted of 80 trials with the target face of emotional expression (happy and sad) and 40 trials with the target face of neutral expression, totaling 120 trials.

Figure 1: Example of Stimuli and Trial Procedure 

Note: A) Happy face gradation example. B) Sequence of a trial: Participants encoded a face and, after a retention interval, judged how vividly they remembered the memorized face (rating from "less vivid" to "more vivid") by clicking with the mouse on the continuous line scale. Then, two test faces appeared and the participant clicked on the memorized target face. Feedback on the memory task was presented after each trial.

Data Analysis

Accuracy, VAS, and the difference between accuracy and VAS, a confidence index (CI), were tested for conformity with normal distribution using the Shapiro-Wilks and were found to be normally distributed (all p > .173). The correct response rates and the vividness judgment were analyzed separately. The correct response rate and CI were submitted to separate ANOVAs considering the combination of the sex of the faces (female or male) and emotions (happy, neutral, and sad) as within-subjects factors and the participant’s sex as between-subjects factor. The vividness judgments were submitted to the same analysis, also considering the correct and incorrect response in the recognition task as a repeated measure factor. Due to a sphericity violation, the degrees of freedom were Greenhouse-Geisser-corrected.

The inferential parameter, the p-value, was fixed at .05. The partial eta squared (η² _p) was reported just as the proportion of variance associated with each main effect and interaction effect in the ANOVA. The Bonferroni post hoc test with a significance level of 5% was used when necessary.

Accuracy of the Memory Task

Performance was affected by the emotional target faces (F(1.87, 84.07) = 21.93, p < .001, η² _p = .33). The recognition of happy (M = .77, SE = .02) and neutral faces (M = .76, SE = .03) was better than sad faces (M = .67, SE = .02) (all p < .001). There was no difference in memory performance for female or male faces (F(1, 45) = 2.74, p = .10, η² _p = .06). However, there was an interaction between this factor and the emotional valence of faces (F(1.93, 86.72) = 14.18, p < .001, η² _p = . 0.24) (Figure 2-A). The performance with the sad female face (M = .65, SE = .02) was worse (p < .001) than with the happy (M = .83, SE = .01) and neutral face (M = .76, SE = .02), which differ from each other (p = .02). The performance with the male faces does not rely on the valence (p = 1). In addition, the accuracy of the memory task was affected by the interaction of the participants’ sex and face stimulus sex (F(1, 45) = 5.52, p = .02, η² _p = .11). This interaction (Figure 3-A) suggests that while female participants recognize both male (M = .74, SE = .03) and female faces (M = .75, SE = .03) with the same accuracy (p = 1), the male participants tend to be worse in recognizing female faces (M = .70, SE = .04) than male faces (M = .74, SE = .03) (p = .047). The accuracy was not affected by the participant’s sex (F(1,45) = 1.44, p = .24, η² _p = .03).

Vividness Judgment

The vividness judgments underwent the same analysis of variance as the accuracy, taking into account the correctness of responses in the recognition task. This analysis is particularly crucial for our study, as we anticipated a significant disparity in vividness judgments between correct and incorrect responses. Specifically, we hypothesized that correct responses would be associated with higher vividness ratings compared to incorrect responses. However, our findings revealed that this difference did not reach statistical significance (F(1, 45) = 2.33, p = .13, η² _p = .05).The perceived vividness was affected just by emotional valence (F(1.193, 53.69) = 6.32, p =.01, η² _p = .12). The happy face was perceived and judged as more vivid (M = .63, SE = .04, p < .01) than the neutral one (M = .59, SE = .04), but not than the sad face (M = .60, SE = .04, p = .08) and there was no significant difference between neutral and sad faces (p =.73). The vividness judgments made by male participants (M = .64, SE = .03) are higher than that of female participants (M = .57, SE = .03) (F(1, 45) = 4.25, p =.045, η² _p = .08). Although having not reached a significant effect, for the sake of comparability with the other dependent variables, the interactions between the stimulus face sex and stimulus valence (F(2, 90) = 2.47, p = .09, η² _p = .05) (Figure 2-B) and between sex of participants and the sex of the face stimulus (F(1, 45) = .65, p = .42, η² _p = .01) (Figure 3-B) were presented at Figure 2-B and Figure 3-B, respectively. No other factor had a significant effect on the perceived vividness.

Correlation between Accuracy and Vividness

There are several ways to estimate the correlation between accuracy and vivacity (^{Busey et al., 2000}). The within-participants correlation reflects the degree to which each participant’s average accuracy is associated with their average vividness judgment. We did a repeated measure correlation (^{Marusich & Bakdash, 2021}) considering the accuracy and vividness of each participant in the six treatments and found no significant correlation (all p > .13). This correlation analysis corroborates the non-significant variation of vividness judgment in correct and incorrect responses we found in the previous analysis of variance. The between-participants correlation considers that the participants with higher vivacity judgments also tend to be more accurate. To examine the correlation between accuracy and vivacity, we calculated Pearson’s correlation for each participant across the six treatments. These treatments were obtained by combining face sex (male or female) and valence (positive or negative). The correlation obtained with the happy male face was significant (rxy = -.31, p < .05).

Confidence Index

Finally, a weaker correlation can be estimated from the difference between the mean values of the vivacity judgments and accuracy in the recognition task. This measure is common in metacognitive studies and is considered an index of subjective confidence (CI) the participant has about their objective performance (see ^{Garcia-Cordero et al., 2021}; Pullford, 1996). According to Begué et al. (2019), positive confidence index values are found when participants are overconfident in their answers, while negative values are characteristic of underconfidence and tend to zero when the accuracy corresponds to the subjective estimate. In our case, we continue identifying this index as subjective confidence, although it was used to estimate the difference between the judged vividness of the memorized stimuli and the accuracy.

The confidence indices, estimated by the difference between the means of the estimated vivacity and accuracy, of each participant, in each treatment, were submitted to an ANOVA that considered the sex of the participants and repeated measures in the face sex (female vs. male) and in the emotional valence of the memorized face (happy, neutral and sad). A test t applied to the CIs in each treatment shows that they are all significantly different from zero (all p > .04). The analysis of variance shows that the female participants are more underconfident (M = -.18, SE = .03) than male participants [(M = -.08, SE = .03) (F(1, 45) = 5.52, p = .02, η² _p = .11)]. The CIs of female and male participants are dependent on the stimulus face sex (F(1, 45) = 4.51, p = .04, η² _p = .09) (Figure 3-C). Female participants judge female and male faces with the same confidence level (M = -.18, SE = .05 and SE = .06, respectively), but the male participants are less underconfident when judging male faces (M = -.05, SE = .06) than judging female faces (M = .10, SE = -.05) (p = .04). The CI changes with the emotional valence of the face stimuli (F(1.77, 79.58) = 12.29, p = .001, η² _p = .215). Participants are more underconfident in trials with neutral (M = -.16, SE = .04) and happy faces (M = -.14, SE = .04) than in trials with sad faces (M = -.07, SE = .03).

The effect of the emotional valence on CI also depends on the stimulus face's sex (F(1.91, 85.8) = 11.29, p < .001, η² _p = .20) (Figure 2-C). According to this interaction, participants are significantly more underconfident when judging happy female faces (M = -.20, SE = .02) than when judging happy male faces (M = -.09, SE = .03). The underconfidence does not change between faces sex for neutral (p = 1) or sad faces (p = 1).

Figure 2: WM Accuracy, Vividness Judgment and Confidence Index for Emotional Faces of Male and Female Stimuli 

Figure 3: WM Accuracy, Vividness Judgment and Confidence Index for Emotional Faces for Male and Female Participants 

Mental Vividness and Working Memory

Initially, in this experiment, we investigated whether the memory accuracy of the memorized emotional faces depends on their vividness judgment. Previous literature showed that the relationship between memory and mental vividness is controversial (^{Baddeley & Andrade, 2000}; ^{Bona et al., 2013}; ^{Bona & Silvanto, 2014}). For instance, Baddeley and Andrade (²⁰⁰⁰) investigated the relationship between WM and mental vividness rating. They found that unfamiliar stimuli are more dependent on WM resources, therefore, more susceptible to interference effects and, consequently, to vividness rating impairment. The interaction between the imagery modality (visual or auditory) and the type of interference may suggest that WM contributes to mental imagery. For example, presenting a visual stimulus and disrupting the visuospatial sketchpad through a concurrent task affect the vividness rating, and the same happens to auditory stimulus and the phonological loop. That is, the vividness rating was dependent upon the WM system. On the other hand, Bona et al., 2013 and Bona and Silvanto, 2014 found that vividness rating and memory accuracy positively correlated, but they were differentially susceptible to distracting visual inputs. This provides some evidence for different underlying mechanisms of these cognitive processes.

Our results showed no interdependence between memory accuracy and the vividness rating; however, we may consider some objections about interpreting our results. The stimuli of faces present a more complicated process than other stimuli, such as shapes or colors. It also may be involved with long-term memory and semantic processes that could influence the vividness judgment and the memory task (^{Wyer et al., 2015}). ^{Baddeley and Andrade (2000}) reported that long-term memory influences mental vividness through verbal-cued images. The vividness of mental images could incorporate sensory information with long-term memory, reducing mental vividness judgment. Our results may be congruent with Baddeley and Andrade (²⁰⁰⁰), who demonstrated the accessibility of long-term memory with mental vividness, which might imply the role of long-term memory in mental vividness and not only the WM.

Furthermore, our results for confidence rating support the view of no correlation between these processes. Our findings suggest an underestimation of performance in the WM task for happy and neutral faces, mainly on happy female faces, despite the accuracy analysis showing a WM benefit for happy female faces. Conversely, previous literature reports that people tend to inflate their estimation of their performance in the ability to detect emotion in pictures of faces and speech (^{Bègue et al., 2019}; see ^{Eflikides, 2016} for a review), overestimating the impact of emotional faces on memory (^{Nomi et al., 2013}). The confidence rates suggest that participants were not accurate in judging their emotional face vividness according to the actual WM performance.

Mental Vividness and Emotion

In this experiment, we also investigated whether the stimuli with an emotional load would be more vivid than neutral stimuli. Our results showed that the participants judged the happy faces as more vivid than neutral and sad ones. The better vividness of happy faces found in our study is supported by previous findings that reported an advantage for detecting happy expressions compared to other emotional expressions (^{Becker et al., 2012}; ^{Curby et al., 2019}). For instance, Becker et al. (²⁰¹²) investigated whether happy or angry faces had a better identification when participants viewed a sequence of morph-faces with a different gradation of emotion from a neutral face to an expressive extreme (i.e., Experiment 1). The authors found a strong effect of rapid identification of happy faces. According to Becker and Srinivasan (²⁰¹⁴), the advantage of detecting happy faces in visual search studies suggests that happy faces are particularly more vivid at the earliest stages of perception than other emotional faces. Our findings are consistent with previous evidence of a “happy superiority” effect in mental representations vividness.

Emotion and Memory

Concerning the recognition of emotional information in working memory, our study found that performance was worse on neutral and sad female expressions than on happy ones. This evidence suggests a better WM for happy faces, mainly female ones. The happy benefit of working memory has been reported in previous studies (^{Spotorno et al., 2018}). For instance, Spotorno et al. (²⁰¹⁸) found that happy faces were better remembered than angry faces when spatial information was required in a WM task. In the task, participants viewed happy and angry faces in different locations. After short or longer intervals (e.g., 1s - 6s), they had to relocate a neutral target face with the same identity as one of the previous faces into its original position. Results showed that the accuracy of the task face relocation was better when the target face had the same identity as the happy face on encoding. Strong evidence of the happy superiority effect also comes from characterization tasks where participants are asked to recognize the facial expression into a limited number of pre-existing categories (^{Calvo et al., 2016}). Although happy faces have a discriminate perceptual feature related to the “open-mouth smile” compared to neutral and negative faces, this visual trait cannot solely explain happy faces’ prioritization processing. In support of this view, ^{Becker et al. (2011}) showed benefits in recognizing a happy face, even without exposing the teeth. According to the authors, the discrimination of happy faces might have this evolutionary advantage because they are less ambiguous than the other expressions.

Moreover, studies have also reported a positive bias of emotional information in healthy individuals, in contrast to depressed patients (^{Lazarov et al., 2018}; ^{Levens & Gotlib, 2010}; ^{Zhou et al., 2020}). Support for this view has been shown in a WM task investigating WM’s emotional bias among healthy and depressed individuals (Levens & Gotlib, 2010). Participants performed an emotion n-back in which they had to indicate if each face of a sequence of faces has the same or a different emotional expression from one previous face. Results showed that non-depressed participants had a slower response time to disengage from happy faces than from neutral or sad faces; in contrast, depressed participants showed the opposite pattern, with a faster response time to disengage from happy than from the other emotional faces. This study suggests that healthy individuals exhibit a bias to process positive information (e.g., happy faces), and individuals in a depressed mood have a negative bias toward sad faces. Our results are consistent with a positive bias theory in WM, predicting a better accuracy for positive content, like happy faces, on memory.

However, our findings imply an interference of gender facial processing while participants had to identify the faces’ emotional expressions. Our results showed that the happy WM advantage was mainly found in female faces. These results are supported by ^{Becker’s (2017}) findings, which examined whether discrimination of negative (i.e., angry) and positive (i.e., happy) expressions could be separated from discrimination of gender (i.e., female vs. male). In the study, participants judged the sex or the expression of faces. Results showed better accuracy when negative male faces (e.g., angry) and positive female (e.g., happy) faces were presented, suggesting that the processing of facial gender interferes with the identification of emotional faces. This result also can be related to unconscious factors of men’s perception of women, based on evolutionary aspects (i.e., lower need for intraspecific competition) or even cultural factors. For instance, female and male mate choice play roles in the evolution of human facial (^{Puts et al., 2012}). In line with this perspective, the better identification of female faces expressing a positive emotion (e.g., happy) found in our study might have arisen from the influence of gender stereotypes, a topic to be explored in further studies.

Limitations

One possible criticism of this study is regarding the mood effects on performance. Although we guaranteed that our participants were not under medication or psychological treatments, there was no application of instruments to evaluate the participant’s psychological-emotional state (e.g., evaluation of depressed symptoms). This factor could have influenced the results. According to previous WM studies, mood disorders such as depression are associated with a negative bias toward sad faces (^{Linden et al., 2011}); that is, depressive patients tend to direct their attention to sad stimuli (^{Lazarov et al., 2018}), resulting in deeper processing of this emotional face. Besides, evidence has shown that the vividness of emotional faces is related to mood effects. For instance, ^{Bywaters et al. (2004}) reported that unpleasant mental images are experienced more often by depressed moods than by non-depressed moods. However, it is important to note that our results should have revealed a better accuracy and vividness of negative faces if this was the case.

In contrast, a happy WM benefit was found, in addition to a higher vividness judgment of happy faces compared to sad faces. Previous evidence of a positive bias found in healthy individuals (^{Lazarov et al., 2018}; ^{Levens & Gotlib, 2010}; ^{Zhou et al., 2020}) supports the claim that our participants were not influenced by a depressive mood related to a negative bias. Though we cannot ensure the absence of mood effects in our study, this evidence should be considered. Future studies should consider the participants’ emotional state as a control measure by evaluating the existence of depressive symptoms.