ARTICLE

 

LATEMO-E: a film database to elicit discrete emotions and evaluate emotional dimensions in Latin-Americans

 

LATEMO-E: banco de dados de filmes para suscitar emoções e avaliar dimensões emocionais para Latino-Americanos

 

LATEMO-E: set de fragmentos de películas para inducir emociones y evaluar dimensiones emocionales en Latinoamericanos

 

 

Yanina Michelini^I; Ignacio Acuña^II; Juan Ignacio Guzmán^III; Juan Carlos Godoy^IV

^IOrcid.org/0000-0003-0768-7645. Universidad Nacional de Córdoba, Córdoba, Argentina
^IIOrcid.org/0000-0001-6152-6530. Universidad Nacional de Córdoba, Córdoba, Argentina
^IIIOrcid.org/0000-0002-9996-6694. Universidad Nacional de Córdoba, Córdoba, Argentina
^IVOrcid.org/0000-0002-1622-1647. Universidad Nacional de Córdoba, Córdoba, Argentina. Instituto de Investigaciones Psicológicas, IIPsi, Unidad Ejecutora CONICET, Córdoba, Argentina

Mailing address

 

 

---

ABSTRACT

INTRODUCTION: Film clips are the most used stimuli to induce different emotional states.
OBJECTIVE: Identify, select, and evaluate a set of film clips according to dimensionality and discreteness of emotions. Film experts suggested 437 fragments, but only 70 met the criteria to be included in the LATEMO-E film-clip database. Study 1: The 70 film clips were tested for their ability to induce disgust, anger, fear, sadness, amusement, tenderness, and neutral (n = 147 participants; mean age = 20.64 years, SD ±2.21 years).
RESULTS: A total of 28 film clips presented adequate scores on intensity and discreteness. Study 2: A total of 28 selected film clips were assessed again in a new sample of 106 participants (mean age = 20.96 years, SD ±2.45 years).
RESULTS: Anger and amusement film clips were the most extreme in the valence and activation dimensions. Anger and fear film clips were the most extreme categories in the intensity, certainty, and control dimensions.
CONCLUSIONS: These stimuli were useful for inducing emotional states that can be judged from both the dimensional perspective of emotions and the basic emotions perspective. Evoked emotions could be differentiated according to cognitive patterns of evaluation that guide behavior in response to emotional stimulation.

Keywords: Film database, emotion elicitation, discrete emotions.

---

RESUMO

INTRODUÇÃO: os clipes são os estímulos mais utilizados para induzir diferentes estados emocionais.
OBJETIVO: identificar, selecionar e avaliar um conjunto de clipes de filme de acordo com a discrição e dimensionalidade da emoção. 437 fragmentos foram sugeridos por especialistas em filmes, mas 70 atenderam aos criterios de inclusão em um banco de dados de filmes. Estudo 1: os 70 clipes foram testados em sua capacidade de induzir: nojo, raiva, medo, tristeza, diversão, ternura e neutro (N = 147 participantes; M = 20,64; DP ±2,21). Resultados: 28 clipes apresentaram escores adequados de intensidade e discrição. Estudo 2: 28 clipes selecionados foram reavaliados em uma nova amostra de 106 participantes (M = 20,96; DP ±2,45).
RESULTADOS: os clipes de raiva e diversão foram os mais extremos nas dimensões de valência e ativação. Clips de raiva e medo foram as categorias mais extremas nos indicadores de intensidade, certeza e controle.
CONCLUSÕES: os estímulos foram úteis para induzir estados emocionais que podem ser julgados tanto pela perspectiva dimensional das emoções quanto pela perspectiva básica das emoções. Além disso, as emoções evocadas são diferenciadas de acordo com os padrões cognitivos de avaliação que guiam o comportamento em resposta à estimulação emocional.

Palavras-chave: Conjunto de fragmentos de filme, indução de emoções, emoções discretas.

---

RESUMEN

INTRODUCCIÓN: los fragmentos de películas están entre los estímulos más usados para inducir distintos estados emocionales.
OBJETIVO: identificar, seleccionar y evaluar un set de fragmentos de películas según la discreción y la dimensionalidad de la emoción. Expertos en cine sugirieron 437 fragmentos, pero solo 70 fueron identificados como aquellos que cumplieron con los criterios de inclusión en un set de fragmentos de películas. Estudio 1: los 70 fragmentos fueron evaluados en su capacidad para inducir: asco, ira, miedo, tristeza, diversión, ternura y neutro (N = 147 participantes; M = 20.64; DE ±2.21).
RESULTADOS: 28 fragmentos presentaron puntuaciones adecuadas en intensidad y discreción. Estudio 2: los 28 fragmentos fueron reevaluados en una nueva muestra de 106 participantes (M = 20.96, DE ±2.45). Resultados: los fragmentos de ira y diversión fueron los más extremos en las dimensiones valencia y activación. Ira y miedo fueron las categorías más extremas en los indicadores intensidad, certeza y control.
CONCLUSIONES: los estímulos resultaron útiles para inducir estados emocionales que pueden ser juzgados desde la perspectiva dimensional de las emociones, así como desde el enfoque de las emociones básicas. Además, las emociones provocadas se diferencian en función de patrones cognitivos de evaluación que guían la conducta en respuesta al estímulo emocional.

Palabras clave: Set de fragmentos de películas, inducción de emociones, emociones discretas.

---

 

 

Emotions are associated with brief mental states that may include multiple events: cognitive processes, physiological responses, motivational changes, motor activity, and subjective percep-tion of the emotional experience (Bianchin & Angrilli, 2012; LeDoux & Brown, 2017). The study of emotion can be divided into two major approaches. The dimensional approach (Bradley & Lang, 2007) argues that emotions are dispositions for action that are regulated by two basic motivational systems: appetitive and aversive. These systems give emotional valence to emotions, depending on which of the two systems is in primacy in a certain situation, and the arousal of emotions that depends on the intensity of these motivational systems.

The basic emotions approach considers emotions as a set of discrete affective states that are associated with specific facial expressions that are shared universally. For example, Ekman and Friesen (1971) suggested the existence of six basic emotions: happiness, surprise, fear, sadness, anger, and disgust. This approach, considered individually, has limitations in specifying different aspects of the emotional experience. For example, recent studies question the universality of emotions and provide evidence of some differences in the identification of emotional faces in communities that have limited contact with more globalized cultures (e.g., Crivelli, Jarillo, Rusell, & Fernández-Dols, 2016). Several authors have emphasized the importance of considering these two approaches in a complementary way (Gabert-Quillen, Bartolini, Abravanel, & Sanislow, 2015; Schaefer, Nils, Sanchez, & Philippot, 2010).

Many studies indicate that emotional experience affects perception, attention, memory and decision-making (Brosch, Scherer, Grandjean, & Sander, 2013; Michelini, Acuña, & Godoy, 2016). With regard to emotion-cognition interactions, the Appraisal Tendency Framework (ATF; Lerner & Keltner, 2000) is a theoretical approach that considers each discrete emotion that is experienced in a decision or judgment situation and activates a specific cognitive and motivational predisposition to evaluate future events. This predisposition implies an interplay between dimensions of cognitive evaluation: certainty, pleasure, attention activity, control, anticipatory effort, and responsibility (Ellsworth, 2013). These, in turn, imply individual evaluative judgments about whether an event is good or bad and whether the behavior and context that are associated with that event correspond to personal goals and expectations (Keltner & Lerner, 2010).

Among the several methods that are used to study emotions experimentally, film clips and images are the most frequent stimuli that are used to induce different emotional states (Schaefer et al., 2010). Film clips have some advantages over images. These kinds of stimuli can provoke intense and persistent emotional states over time (Michelini, Acuña, & Godoy, 2015). They also have greater ecological validity (Uhrig et al., 2016).

Over the past few years, some studies have developed and validated sets of film clips. Among the most recent studies, Carvalho, Leite, Galdo-Álvarez, and Gonçalves (2012) developed a 52 non-auditory film-clip battery using a dimensional approach that allows the possibility of introducing additional experimental manipulations of sound. The development of this battery also included physiological measures. In another study, the intensity, discreteness, valence, and arousal of 18 film clips were validated based on sex, race-ethnicity, and familiarity with the film clip (Gabert-Quillen et al., 2015). More recently, Samson, Kreibig, Soderstrom, Wade, and Gross (2015) developed a 199 film-clip battery to induce positive (amusement), negative (repulsion), mixed (amusement and repulsion), and neutral emotional states. Cabral, Souza Tavares, Weydmann, das Neves, and de Almeida (2017) tested discrete patterns of emotion elicitation using four films.

Although there are several ways to induce emotions, most of them are for English-speaking samples, thus making application in Latin America difficult. Although some advances have been made (e.g., Michelini et al., 2015), few tools are available that consider multiple perspectives of studying emotions in Spanish-speaking participants. Therefore, the main goals of the present study were to identify and select a set of film clips in the Latin American Spanish language that are able to induce seven discrete emotional states: disgust, amusement, anger, fear, neutral, tenderness, and sadness (Study 1). We then evaluated these film clips according to dimensional and discrete indices of emotion (Study 2). Overall, we expected to obtain stimuli that have the ability to induce and measure discrete and dimensional emotions in Latin-American college students.

 

Study 1: Identification and Selection of Film Clips

Preliminary Identification of Film Clips

The first step was to identify film clips that are able to induce seven emotional states: disgust, anger, fear, sadness, amusement, tenderness, and neutral. The emotional categories were based on previous research (Schaefer et al., 2010). To obtain the film clips, film experts (195 experienced film consumers, 65.1% women; mean age = 27.15 ± 10.16 years) were asked to identify, describe, and value at least one film clip for each of the seven emotional categories. A total of 17.4% of the sample had a daily frequency of film viewing, and 69.2% had a weekly frequency of film viewing. Moreover, 56% of the sample had a frequency of film viewing of 1-10 years, and 22.1% had a frequency of more than 10 years. All of the participants were recruited via social networks, and the task was performed voluntarily through an online survey.

All of the recommended film clips had to be: (1) a film production, (2) a film that was intended for a young-adult audience, (3) other than an Argentinian film (because of the lack of dubbing into Latin American Spanish), and (4) produced less than 10 years ago (this was later extended to 15 years to meet the rest of the criteria). As a result, the film experts made 437 suggestions, but only those that met the selection criteria were chosen. The selection criteria were the following: (1) the most frequent suggestions, (2) availability on YouTube or in video clubs in the Latin American Spanish language, (3) the coherence of scene argument (i.e., topic) with those considered in antecedent literature (Fernández Megías, Pascual Mateos, Soler Ribaudi, & Fernández-Abascal, 2011), and (4) the availability of scores on the valence and arousal subscales for the suggested clips, evaluated using the Self-Assessment Manikin (SAM; Bradley & Lang, 1994). Finally, 70 film clips (10 for each emotional category) that were dubbed in the Latin American Spanish language were selected. These 70 clips were extracted from films that were produced between 2000 and 2014, with an average duration of 152 seconds. Argentinian Law 11.723 covers the use of artistic pieces of work that are utilized for our research purposes (Legal Regime of Intellectual Property, Nº11.723, 1933).

 

Methods

Participants.

The final sample consisted of 147 college students (60.5% women; age range = 18-26 years, mean age = 20.64 years, SD ± 2.21 years) from Universidad Nacional de Córdoba (UNC) and Universidad Tecnológica Nacional (UTN), Argentina. The inclusion criteria were the following: university student, no neurological or mental disorder and receiving no treatment for such conditions, no difficulty seeing or hearing without corresponding visual or auditory correction, and watching movies at least six times per year. Additionally, for the inclusion of participants in the sample, scores on the State-Trait Anxiety Inventory (STAI) and Positive and Negative Affect Schedule (PANAS) were considered. The frequency of film viewing was 6.8% daily, 71.4% weekly, 20.4% monthly, and 1.4% annually. Ethical lineaments for investigations with humans by the American Psychological Association were used (2010).

Instruments

Sociodemographic Questionnaire. This questionnaire included questions about age, sex, neurological and mental disorders, use of psychoactive drugs, vision and hearing difficulties, and frequency of watching films. For women, a question was also included that asked about the current phase of their menstrual cycle, depending on the number of days that elapsed since their last menstruation.

The following measures were used to assess emotional experience:

Discrete Emotions Questionnaire (CED; Fernández Megías et al., 2011). Eighteen emotional labels were scored on a 1 to 7 scale (1 = nothing, no emotion to 7 = too much emotion, very intense). To test the pertinence of this instrument in the local environment, five judges assessed semantic quality and adaptation to the target population. Pilot tests of the modified instrument were then performed. A total of 20 emotional labels were obtained (six target labels [amusement, anger, disgust, fear, tenderness, and sadness] and 14 labels that described associated emotional states).

Self-Assessment Manikin (SAM). The SAM is a simple and brief non-verbal scale that mea-sures emotional reactions from a dimensional perspective. A version that was adapted to the local population was used. The Argentinian validation has adequate psychometric properties, with internal consistency scores that range from .69 to .96 (Irrazabal, Aranguren, Zaldúa, & Di Giuliano, 2015). It contains three pictographic subscales that correspond to the dimensions of emotional experience. For valence, the figures range from the saddest to the most cheerful. For arousal, the figures range from calm to intense arousal. For control, the figures range from the largest (a lot of control) to the smallest (little control of the emotion experienced). Nine pos-sible scores are presented for each subscale.

Visual Analogue Scale (VAS). The VAS is a 100-millimeter line that is widely used to measure emotional states in emotion induction studies (Aguiar-Bloemer & Diez-Garcia, 2018; van den Hout, Eidhof, Verboom, Littel, & Engelhard, 2014). In this study, the VAS was used to measure the degree of emotion that the participant felt while exposed to each film clip, indicating where he or she was relative to the extremes (0 = nothing moved to 100 = fully moved).

The following measures were used to control intervenient variables:

Positive and Negative Affect Schedule (PANAS). The PANAS consists of 20 items (10 for the positive affect subscale and 10 for negative affect subscale) that are used to describe different emotional states. Each item was scored on a 5-point scale (1 = verylittle or nothing to 5 = extremely). We used an Argentinian version with adequate psychometric properties. Cronbach's α coefficient was .73 for the positive affect subscale and .82 for the negative affect subscale (Moriondo, Palma, Medrano, & Murillo, 2012).

State-Trait Anxiety Inventory (STAI). The STAI consists of two separate 20-item self-assessment scales that measure two independent dimensions of anxiety: state and trait. The items on the state-anxiety scale were scored on a 4-point scale (0 = not at all to 3 = a lot). The items on the trait-anxiety scale were scored on a 4-point scale (0 = almost never to 3 = almost always). An adapted version was used. Cronbach's α coefficient of the total score for the Spanish version was .93 (Fonseca-Pedrero, Paino, Sierra-Baigrie, Lemos-Giráldez, & Muñiz, 2012).

Alcohol and other Substance Consumption Questionnaire (ASCQ; Pilatti, Montejano Rivarola, Lozano, & Pautassi, 2016). The ASCQ was used to determine the quantity (in grams of alcohol) and frequency of consumption of the participants' first and second most consumed beverages. Other substances (e.g., tobacco, marijuana, ecstasy, lysergic acid diethylamide) were also considered.

Procedure

Experimental sessions were conducted at the Instituto de Investigaciones Psicológicas (UNC) using previously described procedures (Michelini et al., 2015; Schaefer et al., 2010). We previously generated 31 sets of 10 film clips, each comprising seven film clips (one for each of the seven emotional categories that were proposed for this study). Three of these emotional categories were presented twice. The order of presentation of the film clips was established according to two criteria: two film clips that belonged to the same emotional category and two film clips of the same valence that were not consecutively presented. Each experimental session (~90 minutes) was conducted in groups of two to five participants. Sometimes, some of the participants did not attend, so individual sessions were held. The participants were first contacted by telephone to administer the sociodemographic questionnaire. Once in the experimental session and after signing an informed consent form, the participants completed the STAI, PANAS, and ASCQ. The participants were exposed to a series of 10 film clips and had to score them using the VAS, SAM, and CED. The film clips were presented individually on 19-inch color screens with headphones. Each film clip was preceded by a slide with the following instructions: Dear participant, now you will watch a film clip. At the end of each clip, another slide said: Now we ask you, please, to complete the three questionnaires. Once you complete them, we will ask you to please stay relaxed with your eyes closed until new instructions are announced. Finally, at the end of the audio-visual presentation, the third slide showed: The experimental session is over. Thank you!

Data Analysis

First, outliers and inconsistences in responses were identified. Second, the χ² test was performed, and Spearman's correlations and Pearson's correlations were calculated to determine whether the participants' performance was associated with individual/group experi-mental sessions, alcohol consumption (fre-quency and quantity of the first and second most consumed beverages), other substance consumption (yes or no), and whether they had seen the film clip previously. Third, to determine the ability of the film clips to elicit the target emotions, frequencies were calculated to obtain percentages of scores on the VAS, SAM, and CED for each film clip. For the VAS, scores < 50 were considered neutral, and scores > 50 were considered emotional. For the SAM-valence, scores between 1 and 3 were considered pleasant, scores between 4 and 5 were considered neutral, and scores between 7 and 9 were considered unpleasant. For the SAM-arousal, scores between 1 and 5 were considered low arousal, and scores between 6 and 9 were considered high arousal. For SAM-control, scores between 1 and 5 were considered a lot of control, and scores between 6 and 9 were considered little control. For the CED, scores between 5 and 7 were considered a lot of emotion. Kappa coefficients were also calculated to determine the concordance of responses for each target emotional label relative to the rest of the emotional CED tags for each film clip. Finally, because the CED does not have any label for the neutral category, the relative frequencies of scores between 1 and 3 (no emotion) for CED tags were considered.

 

Results

Preliminary Analysis

No multivariate atypical cases and three univariate atypical cases were identified (z > ± 3.29) but not eliminated. With regard to data distribution, most of the variables presented kurtosis and asymmetry values between -2 and +2. No inconsistencies were found in responses on the PANAS or STAI. Scores on the VAS, SAM, and CED for each film clip were not associated with substance use or with whether the participants had seen the film clips previously. Although the evaluations were conducted mainly in groups, in some cases only one participant attended the experimental session (38% of the participants were evaluated individually, but their performance was not affected by this).

Analysis of Emotional Experience

Table 1 shows the characteristics of the emotional experiences that were induced by each film clip: percentages of responses for each film clip on the VAS and SAM and target emotional label on the CED. Most of the indices were theoretically expected when considering scores that are presented in the data analysis section. Additionally, matching between the target emotional labels and the rest of labels from the CED (except for neutral) is also presented. Most of them were matching with no or low significance (Landis & Koch, 1977).

Table 2 shows the percentages of responses for each film clip from the neutral category for the VAS, SAM, and CED emotional tags. All of them were theoretically expected when considering the scores that are presented in the data analysis section. In this case, the percentages of expected scores (between 1 and 3) were also considered for the anxiety and bewilderment labels, of which percentages > 50% were obtained, except for the Amelie clip.

Some clips, such as those that corresponded to Dumplings (disgust), (500)Days of Summer (sadness), and Catch and Release (tenderness), did not present the expected scores in most of the indices that were considered. Additionally, other clips, such as Black Swan 1 (disgust), WhenAll Is Lost (anger), and Gladiator (anger), although they presented expected scores for the emotion and on the subscales of the SAM, the percentage of expected scores for the target emotion of the CED did not reach 20%. Therefore, of the 70 film clips, only those with optimal scores were selected. Considering the scores on the VAS, SAM, and CED, four film clips (underlined in Table 1 and Table 2) were selected for each emotional category with appropriate values according to the defined criteria (28 total).

 

Discussion

The aim of Study 1 was to identify and select a set of film clips in the Latin American Spanish language with the ability to induce seven emotional states: disgust, amusement, anger, fear, neutral, tenderness, and sadness. Consistent with previous studies (Fernández Megías et al., 2011; Schaefer et al., 2010), film clips that have been shown to work as inducers of emotional categories were obtained. In terms of the intensity of emotions, an average of 73% scored higher than 50% for the target emotion that was provoked by the film clip. This result shows that ~27% of the respondents did not experience the target emotion. However, according to the recommendations of Rottenberg, Kovacs, and Yaroslavsky (2018), reporting values of nonresponses to an emotion helps improve some methodological and theoretical aspects of the study of emotion. For example, nonrespondents could present difficulties with social cognition abilities or have an attitude of resistance to emotion induction. Scores for valence and arousal were similar, in which 66%, 73%, and 53% of the participants presented the expected scores on valence, arousal, and control, respectively. Moreover, the film clips were effective in terms of discreteness, in which the scores provided evidence of matching the target emotion and discrimination relative to the other emotions that were studied. Similar results were obtained in previous studies that used the same procedures (Gabert-Quillen et al., 2015; Gross & Levenson, 1995; Samson et al., 2015).

Neutral-category film clips, in most cases, did not induce anxiety or bewilderment, which could be associated with feelings of boredom. Importantly, boredom is an index of the malfunction of neutral stimuli because it might indicate a bias toward negative emotion induction (Bradley & Lang, 2007).

Among the 28 selected film clips, some presented significant concordance between scores that were assigned to the target emotion and scores that were assigned to other emotions, which could affect the discreteness of these clips. In fact, the film clips CON4 (fear) and NPR (sadness) presented significant concordance between the target emotion and tenderness. However, this concordance had nonsignificant power (Landis & Koch, 1977). Additionally, this concordance was expected because children act in both scenes. Several studies have shown that stimuli that include children provoke a bias toward pleasantness in the experimentation of emotion (e.g., Irrazabal et al., 2015). The film clip ECHII (disgust) presented significant concordance with anger and sadness. The film clip CEC (anger) presented significant concordance with disgust and fear. The film clip IRR (anger) presented significant concordance with sadness. However, the concordance of all of these had nonsignificant power. Finally, the film clip DJA (anger) presented significant concordance with sadness. Although this concordance was acceptable, this film clip was identified as optimal because of the scores that it received in the other criteria. Gross and Levenson (1995) stated that it is presumptuous to assume that a film clip provokes only a single emotion and not other emotions. Discreteness (i.e., a film clip's quality index) refers to the ability of the film clip to provoke an emotion intensely and other emotions to a lesser extent. Additionally, some studies have provided evidence that film clips can induce mixed emotions (e.g., Samson et al., 2015), and obtaining optimum scores for the discreteness of anger and fear is difficult (Gabert-Quillen et al., 2015).

 

Study 2: Evaluation of Selected Film Clips

Methods

Participants

A new sample consisted of 106 participants (62.3% women; age range = 17-29 years, mean age = 20.96 years, SD ± 2.45 years) from UNC and UTN. The inclusion criteria were the same as those in Study 1. The frequency of film viewing was 17.9% daily, 42.5% weekly, 33% monthly, and 6.6% annually.

Instruments

Film Database. The film clips that were selected in Study 1 are available at https://doi.org/10.6084/m9.figshare.5372782.v1

Post-Emotion Eliciting Questionnaire (PEEQ). Based on previous studies (Fernández Megías et al., 2011; Rottenberg, Ray, & Gross, 2007), we provided the following instructions: Please indicate with a cross what was the emotion you felt while watching the film clip (response options: disgust, amusement, anger, fear, neutral, tenderness, sadness, and other). The participants could choose just one of the given options. In Study 2, the other responses were recoded according to the conceptually most related emotional category. We also included the VAS and SAM valence and arousal subscales. The protocol also included six questions to measure certainty (e.g., Could you predict what will happen in that situation?) and control (e.g., What is happening in that situation would be controlled by circumstances beyond your control?; Ellsworth, 2013). Certainty and control are two dimensions of cognitive appraisal for discrete emotions. Finally, we included one question (Have you seen this film-clip before?) to control for prior exposure to the film clip that the participants had just watched.

Study 2 also included the sociodemographic questionnaire, PANAS, STAI, and ASCQ from Study 1.

Procedure

The study was conducted in a classroom that was equipped with computers in groups of five to 15 participants. All of the participants expressed interest in participating voluntarily and signed an informed consent form before starting the experiment. Each participant had a computer and headphones and was exposed to the 28 film clips that were selected in Study 1. The order of film clip presentation was based on the same criteria in Study 1. Three series of film clips were presented in random order to control for order effects. The procedure of Study 2 was similar to Study 1. Comprehension of the PEEQ was tested using the film clip Blue (Fernández Megías et al., 2011).

Data Analysis

First, we performed analyses to identify outliers and inconsistencies of responses on the PANAS and STAI and associations with substance use, whether the participants had seen the film clip previously, and the women's menstrual cycle (last week before menstruation, which is particularly sensitive to emotional processes; Sacher, Okon-Singer, & Villringer, 2013). Second, we placed the 28 stimuli in a two-dimensional emotional space, determined by mean scores for SAM-valence and SAM-arousal for the total sample and by sex, and calculated the quadratic correlation between both dimensions. We also performed repeated-measures analysis of variance (ANOVA) to analyze differences in SAM-valence and SAM-arousal scores based on the emotional categories and sex. Third, repeated-measures ANOVAs were performed to analyze differences in VAS, certainty, and control scores based on emotional categories and sex. Statistically significant interactions were further explored using planned comparisons.

 

Results

Preliminary Analysis

No univariate atypical cases were identified (z > ± 3.29), but two cases were multivariate atypical. Considering that the presence of atypical cases can distort results, the data analyses were performed both with and without atypical cases, and no differences were found between these statistical approaches. Concerning data distribution, most of the variables presented asymmetry and kurtosis values between -2 and +2. No inconsistencies were found in the responses on the PANAS and STAI. Scores on the VAS, SAM, and CED for each film clip were not associated with substance use, whether the participants had previously seen the film clips, or the women's menstrual cycle.

Dimensions of the Emotional Experience: Valence and Arousal

Two-Dimensional Emotional Space. Fi-gure 1 shows the 28 film clips (points) in the two-dimensional emotional space, determined by mean scores on SAM-valence and SAM-arousal. The quadratic correlation between the two dimensions was R² = .52. Figure 1 shows that the film clip distribution has a boomerang shape. From a central and neutral point, it extends two arms, one positive (upper) and one negative (lower). Film clips that belong to the amusement and tenderness categories are located toward the positive quadrant (upper left), and film clips that belong to the disgust, anger, fear, and sadness categories are in the negative quadrant (lower left). Film clips that belong to the neutral category are in a two-dimensional medial position. Although the positive-pole and negative-pole film clips are located to the left (indicating greater arousal), the negative pole has a greater leftward inclination than the positive pole.

Discreteness of the Emotional Experience: VAS, Certainty, and Control

VAS Score. The repeated-measures ANOVA showed a significant main effect of emotional category on VAS scores (F_6,522 = 51.11, p < .0001, ŋ2p = .37). Planned comparisons showed significant differences between all emotional categories, with the exception of differences between disgust and amusement, between disgust and tenderness, between amusement and tenderness, and between anger and sadness (Figure 2). Although no main effects of sex or interactions with sex were found, one-way ANOVAs showed that VAS scores were significantly higher in females than in males in sadness (F_1,103 = 4.70, p < .05, ŋ2p = .04) and tenderness (F_1,102 = 5.32, p < .05, ŋ2p = 0.05).

 

 

Certainty Score. The repeated-measures ANOVA showed a significant main effect of emotional category of certainty (F_6,516 = 75.05, p < .0001, ŋ2p = .47). Planned comparisons showed that certainty scores for anger were significantly higher than for fear; scores for amusement were significantly higher than for fear, and scores for anger were significantly higher than for amusement (Figure 3A).

 

 

Control Score. The repeated-measures ANOVA showed a significant main effect of emotional category on control (F_6,516 = 57.02, p < .0001, ŋ2p = .40). Planned comparisons showed that control scores for anger were significantly higher than for fear, scores for amusement were significantly higher than for fear, scores for anger were significantly higher than for sadness, scores for amusement were significantly higher than for sadness, and scores for anger were significantly higher than for amusement (Figure 3B).

 

Discussion

The aim of Study 2 was to evaluate the film clips that were selected in Study 1 according to dimensions of the emotion (valence and arousal), a subjective analogical report of the intensity of the emotional experience (VAS), and other dimensions of cognitive evaluation of the discrete emotions (certainty and control).

After locating the 28 film clips in the two-dimensional emotional space that was delimited by valence and arousal, the distribution of the stimuli adopted the typical boomerang shape that was previously reported in emotional image validation studies (Irrazabal et al., 2015; Moltó et al., 2013). This boomerang shape refers to a neutral center of low arousal from which two arms (one positive and one negative) split increases in arousal. Consistent with these previous studies (Aguado, Fernández-Cahill, Román, Blanco, & Echegaray, 2016; Irrazabal et al., 2015; Moltó et al., 2013; Zupan & Babbage, 2016), the boomerang placed neutral film clips in a neutral and calm center, and the rest of the clips were distributed at the poles. The negative pole (anger, disgust, fear, and sadness) has more arousal than the positive pole (amusement and tenderness). Some of the anger clips had the most valence and arousal in the negative pole, and some of the amusement clips had the highest valence and arousal in the positive pole. The set of film clips adopted the expected distribution in the two-dimensional emotional space, consistent with the dimensional properties of the emotional experience (Bradley & Lang, 2007). Additionally, the location of another kind of stimuli, in this case audio-visual stimuli, result in a novel contribution to research on emotions.

With regard to the results for the VAS for each emotional category, film clips that belonged to disgust, anger, fear, sadness, amusement, and tenderness provoked a significantly more intense emotional experience than neutral film clips, and anger and sadness were the most intense categories. This result may be considered an improvement over a previous study in which intense emotions were generated for sadness but not for anger (Zupan & Babbage, 2016).

Finally, the certainty and control indices were considered only for some of the emotional categories. Based on the ATF, specifically anger, fear, sadness, and amusement, these emotions are the most accurate with regard to the certainty and control indices (Ellsworth, 2013; Keltner & Lerner, 2010). Thus, anger and amusement were the emotional experiences that were most associated with certainty and control over fear. Additionally, for both certainty and control, anger scores were significantly higher than for amusement. Both anger and amusement, which are considered emotions that are associated with certain situations and for which it is possible to identify a human agent that controls the situation, are relevant when considering the impact on different cognitive processes. For example, evidence indicates that anger and amusement affect decision-making and increase optimism and tolerance to risky situations compared with fear and sadness (Angie, Connelly, Waples, & Kligyte, 2011). Furthermore, differences in certainty and control between anger and amusement may respond to differences in other properties. They are emotions with different valence and arousal levels. Furthermore, negative emotions are generally more precise and correspond to a response pattern, whereas positive emotions tend to be more diffuse (Ellsworth, 2013). Sadness is expected to show a similar pattern to fear, however, it presented greater certainty than fear. Ellsworth (2013) stated that certainty is not a simple evaluation pattern and is rather linked to numerous contextual factors. Therefore, some film clips likely provoked sadness that was associated with accurate evaluative judgments. For example, some studies reported that an anger stimulus also induced sadness (Cabral et al., 2017; Gross & Levenson, 1995). Sadness can also be distinguished from anger and amusement in more complex ways. Lench, Tibbett, and Bench (2016) explained that sadness, compared with anger, is associated with irreparable loss, lower emotional expression, and lower psychophysiological activity.

 

General Discussion

The aim of the present study was to construct a set of film clips in the Latin American Spanish language (LATEMO-E) that are able to induce discrete emotions and measure dimensional aspects of emotion. To achieve this, we performed two studies. In Study 1, film experts were asked to identify pertinent stimuli that induced seven emotions: disgust, anger, fear, sadness, amusement, tenderness, and neutral. Although the film clips were obtained from fictional and commercial stories, which may affect the power of these stimuli to induce emotions (Arminjon et al., 2015), it is important to highlight the advantages of such film clips. For example, all of the films pertained to different countries and cultures. Some were independent films, and all were produced in the last 15 years. Therefore, the film clips are stimuli that have moderate popularity and are applicable to young-adult audiences. LATEMO-E is an emotion-inducing tool with sound ecological validity and quality (Gabert-Quillen et al., 2015).

The LATEMO-E set of film clips includes both intense and discrete stimuli. Extensive evidence shows the existence of specific patterns of responses in subjective emotional reports and central and peripheral nervous system activity in response to specific emotions (Aguado et al., 2016). Several studies have shown that the impact of each discrete emotion on cognition, together with other factors (e.g., age and sex), is reflected by singular cognitive and behavioral patterns (Ferrer, Maclay, Litvak, & Lerner, 2016; Michelini et al., 2016). Nevertheless, in some cases, the discreteness of emotions is relative (Cabral et al., 2017; Gross & Levenson, 1995). Consequently, some emotions can overlap, which might hinder studies of the effects of different emotions.

Study 2 tested a database of 28 film clips with normative data on the dimensionality (valence and arousal), discreteness (intensity), and other dimensions of the cognitive evaluation of discrete emotions (certainty and control). We were able to assess both the dimensionality and discreteness of the same set of stimuli that induce emotions (e.g., Gabert-Quillen et al., 2015). For example, in the present study, anger had the most exciting and intense traits. However, in the anger category, it was easier to identify the source of control of the emotional situation. Fear was not associated with extreme values of valence, arousal, or intensity. In a fearful situation, an individual may be less able to have tools available to face the situation because of high uncertainty and control that is outside of unidentified circumstances (Ellsworth, 2013).

The integration of categorical and dimensional perspectives has been useful for explaining the mechanisms that underlie the perception (Fujimura & Umemura, 2018) and expression (Mehu & Scherer, 2015) of emotions. Importantly, emotional experiences should be understood as complex, multi-dimensional, and dynamic processes (Keltner & Lerner, 2010). Effects of specific aspects of emotions over cognitive and behavioral processes can thus be elucidated.

Among the limitations of this study, Study 2 should be replicated with a larger sample and with the inclusion of objective measures of emotion, such as heart rate, skin conductance, and electroencephalographic activity (Bianchin & Angrilli, 2012; Bos, Jentgens, Beckers, & Kindt, 2013). Likewise, it would be convenient to advance in replications that include data analysis strategies for each of the clips, which could contribute to report results more accurate for each one of the emotional categories (Gabert-Quillen et al., 2015). Based on the approach of Carvalho et al. (2012), it would also be interesting to test the ability of LATEMO-E to induce emotions without their linguistic features and validate LATEMO in other languages, such as Portuguese.

In summary, the present study advanced the study of emotions by assessing an audio-visual tool in the Latin American Spanish language. LATEMO-E was shown to be useful for inducing several categories of emotions. Normative data were obtained with regard to dimensionality, discreteness, and the cognitive evaluation of emotional experiences. We also evaluated sex differences. Finally, if considering recent criticisms to the theory of basic emotions proposed by Ekman (see, for instance, Crivelli et al., 2016) or, the critical views regarding the vast use of WEIRD people in psychological research (Henrich, Heine, & Norenzayan, 2010), such tools as LATEMO-E are becoming increasingly required for the experimental study of emotions, including studies of both development (Gambetti & Giusberti, 2016) and psychopathology (Seeley et al., 2016).

 

References

Aguado, L., Fernández-Cahill, M., Román, F. J., Blanco, I., & de Echegaray, J. (2016). Evaluative and psychophysiological responses to short film clips of different emotional content. Journal of Psychophysiology, 32(1),1-19. doi: 10.1027/0269-8803/a000180        [ Links ]

Aguiar-Bloemer, A. C., & Diez-Garcia, R. W. (2018). Influence of emotions evoked by life events on food choice. Eating and Weight Disorders, 23(1),45-53. doi: 10.1007/s40519-017-0468-8        [ Links ]

American Psychological Association. (2010). Ethical principles of psychologists and code of conduct: 2010 amendments. Retrieved from https://www.apa.org/ethics/code/index.aspx        [ Links ]

Angie, A. D., Connelly, S., Waples, E. P., & Kligyte, V. (2011). The influence of discrete emotions on judgement and decision-making: A meta-analytic review. Cognition and Emotion, 25(8),1393-1422. doi: 10.1080/02699931.2010.550751        [ Links ]

Arminjon, M., Preissmann, D., Chmetz, F., Duraku, A., Ansermet, F., & Magistretti, P. J. (2015). Embodied memory: Unconscious smiling modulates emotional evaluation of episodic memories. Frontiers in Psychology, 6,650. doi: 10.3389/fpsyg.2015.00650        [ Links ]

Bianchin, M., & Angrilli, A. (2012). Gender differences in emotional responses: A psychophysiological study. Physiology and Behavior, 105(4),925-932. doi: 10.1016/j.physbeh.2011.10.031        [ Links ]

Bos, M. G., Jentgens, P., Beckers, T., & Kindt, M. (2013). Psychophysiological response patterns to affective film stimuli. PLoS One, 8(4),e62661. doi: 10.1371/journal.pone.0062661        [ Links ]

Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment Manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1),49-59. doi: 10.1016/0005-7916(94)90063-9        [ Links ]

Bradley, M. M., & Lang, P. J. (2007). The International Affective Picture System (IAPS) in the study of emotion and attention. In J. A. Coan & J. J. B. Allen (Eds.), Handbook of emotion elicitation and assessment (pp. 29-46). New York: Cambridge University Press.         [ Links ]

Brosch, T., Scherer, K. R., Grandjean, D. M., & Sander, D. (2013). The impact of emotion on perception, attention, memory, and decision-making. Swiss Medical Weekly, 143,w13786. doi: 10.4414/smw.2013.13786        [ Links ]

Cabral, J. C. C., Tavares, P. S., Weydmann, G. J., das Neves, V. T., & de Almeida, R. M. M. (2017). Eliciting negative affects using film clips and real-life methods. Psychological Reports, 121(3),527-547. doi: 10.1177/0033294117730844        [ Links ]

Carvalho, S., Leite, J., Galdo-Álvarez, S., & Gonçalves, O. F. (2012). The emotional movie database (EMDB): A self-report and psychophysiological study. Applied Psychophysiology and Biofeedback, 37(4),279-294. doi: 10.1007/s10484-012-9201-6        [ Links ]

Crivelli, C., Jarillo, S., Russell, J. A., & Fernández-Dols, J. M. (2016). Reading emotions from faces in two indigenous societies. Journal of Experimental Psychology: General, 145(7),830-843. doi: 10.1037/xge0000172        [ Links ]

Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal ofPersonality and Social Psychology, 17,124-129. Retrieved from http://psycnet.apa.org/record/1971-07999-001        [ Links ]

Ellsworth, P. C. (2013). Appraisal theory: Old and new questions. Emotion Review, 5(2),125-131. doi: 10.1177/1754073912463617        [ Links ]

Fernández Megías, C., Pascual Mateos, J. C., Soler Ribaudi, J., & Fernández-Abascal, E. G. (2011). Validación española de una batería de películas para inducir emociones. Psicothema, 23(4),778-785. Retrieved from http://www.redalyc.org/html/727/72722232039/        [ Links ]

Ferrer, R. A., Maclay, A., Litvak, P. M., & Lerner, J. S. (2016). Revisiting the effects of anger on risk-taking: Empirical and meta-analytic evidence for differences between males and females. Journal of Behavioral Decision Making, 30(2),516-526. doi: 10.1002/bdm.1971        [ Links ]

Fonseca-Pedrero, E., Paino, M., Sierra-Baigrie, S., Lemos-Giráldez, S., & Muñiz, J. (2012). Propiedades psicométricas del "Cuestionario de Ansiedad Estado-Rasgo" (STAI) en universitarios. Behavioral Psychology, 20(3),547-561.         [ Links ]

Fujimura, T., & Umemura, H. (2018). Development and validation of a facial expression database based on the dimensional and categorical model of emotions. Cognition and Emotion. https://doi.org/10.1080/02699931.2017.1419936        [ Links ]

Gabert-Quillen, C. A., Bartolini, E. E., Abravanel, B. T., & Sanislow, C. A. (2015). Ratings for emotion film clips. Behavior Research Methods, 47(3),773-787. doi: 10.3758/s13428-014-0500-0        [ Links ]

Gambetti, E., & Giusberti, F. (2016). Anger and everyday risk-taking decisions in children and adolescents. Personality and Individual Differences, 90,342-346. doi: 10.1016/j.paid.2015.11.049        [ Links ]

Gross, J. J., & Levenson R. W. (1995). Emotion elicitation using films. Cognition and Emotion, 9(1),87-108. Retrieved from https://www.tandfonline.com/doi/abs/10.1080/02699939508408966        [ Links ]

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral andBrain Sciences, 33,61-83. doi: 10.1017/S0140525X0999152X        [ Links ]

Irrazabal, N., Aranguren, M., Zaldua, E., & Di Giuliano, N. (2015). Datos normativos del Sistema Internacional de Imágenes Afectivas (IAPS) en una muestra argentina. Revista Argentina de Ciencias del Comportamiento, 7(3),34-50. Retrieved from http://www.scielo.org.ar/scielo.php?script=sci_arttext&pid=S1852-42062015000300006        [ Links ]

Keltner, D., & Lerner, J. S. (2010). Emotion. In S. T. Fiske, D. T. Gilbert, & G. Lindzey (Eds.), Handbook of social psychology (5th ed., pp. 312-347). Hoboken, NJ: Wiley.         [ Links ]

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33,159-174.         [ Links ]

LeDoux, J. E., & Brown, R. (2017). A higher-order theory of emotional consciousness. Proceedings of the National Academy of Sciences of the United States of America, 114(10),e2016-e2025. doi: 10.1073/pnas.1619316114        [ Links ]

Legal Regime of Intellectual Property, Nº11.723. (1933). Honorable Congress of the Argentine Nation. Retrieved from http://servicios.infoleg.gob.ar/infolegInternet/verNorma.do?id=42755.         [ Links ]

Lench, H. C., Tibbett, T. P., & Bench, S. W. (2016). Exploring the toolkit of emotion: What do sadness and anger do for us? Social and Personality Psychology Compass, 10(1),11-25. doi: 10.1111/spc3.12229        [ Links ]

Lerner, J. S., & Keltner, D. (2000). Beyond valence: Toward a model of emotion-specific influences on judgement and choice. Cognition and Emotion, 14(4),473-493. Retrieved from https://www.tandfonline.com/doi/abs/10.1080/026999300402763        [ Links ]

Mehu, M., & Scherer, K. R. (2015). Emotion categories and dimensions in the facial communication of affect: An integrated approach. Emotion, 15(6),798-811. doi: 10.1037/a0039416        [ Links ]

Michelini, Y., Acuña, I., & Godoy, J. C. (2015). Características de la experiencia emocional inducida mediante fragmentos de películas en una muestra de jóvenes argentinos. Interdisciplinaria, 32(2),367-382. Retrieved from http://www.scielo.org.ar/scielo.php?script=sci_arttext&pid=S1668-70272015000200010        [ Links ]

Michelini, Y., Acuña, I., & Godoy, J. C. (2016). Emociones, toma de decisiones y consumo de alcohol en jóvenes universitarios. Suma Psicológica, 23(1),42-50. doi: 10.1016/j.sumpsi.2016.01.001        [ Links ]

Moltó, J., Segarra, P., López, R., Esteller, À., Fonfría, A., Pastor, M. C., & Poy, R. (2013). Adaptación española del "International Affective Picture System" (IAPS): tercera parte. Anales de Psicología, 29(3),965-984. doi: 10.6018/analesps.29.3.153591        [ Links ]

Moriondo, M., Palma, P., Medrano, L., & Murillo, P. (2012). Adaptación de la escala de afectividad positiva y negativa (PANAS) a la población de adultos de la ciudad de Córdoba: análisis psicométricos preliminares. Universitas Psychologica, 11(1),187-196. Retrieved from http://www.redalyc.org/html/647/64723234015/        [ Links ]

Pilatti, A., Montejano Rivarola, G., Lozano, O. M., & Pautassi, R. M. (2016). Relationship between impulsivity and alcohol consumption in Argentinean men and women. Quaderns de Psicologia, 18(1),75. doi:10.5565/rev/qpsicologia.1329        [ Links ]

Rottenberg, J., Kovacs, M., & Yaroslavsky, I. (2018). Non-response to sad mood induction: Implications for emotion research. Cognition and Emotion, 32(3),431-436. doi: 10.1080/02699931.2017.1321527        [ Links ]

Rottenberg, J., Ray, R. D., & Gross, J. J. (2007). Emotion elicitation using films. In J. A. Coan & J. J. B. Allen (Eds.), Handbook of emotion elicitation and assessment (pp. 9-28). New York: Oxford University Press.         [ Links ]

Sacher, J., Okon-Singer, H., & Villringer, A. (2013). Evidence from neuroimaging for the role of the menstrual cycle in the interplay of emotion and cognition. Frontiers in Human Neuroscience, 7,374. doi: 10.3389/fnhum.2013.00374        [ Links ]

Samson, A. C., Kreibig, S. D., Soderstrom, B., Wade, A. A., & Gross, J. J. (2015). Eliciting positive, negative and mixed emotional states: A film library for affective scientists. Cognition and Emotion, 30(5),827-856. doi: 10.1080/02699931.2015.1031089        [ Links ]

Schaefer, A., Nils, F., Sanchez, X., & Philippot, P. (2010). Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers. Cognition and Emotion, 24(7),1153-1172. doi: 10.1080/02699930903274322.         [ Links ]

Seeley, S. H., Mennin, D. S., Aldao, A., McLaughlin, K. A., Rottenberg, J., & Fresco, D. M. (2016). Impact of comorbid depressive disorders on subjective and physiological responses to emotion in generalized anxiety disorder. Cognitive Therapy and Research, 40(3),290-303. doi: 10.1007/s10608-015-9744-y        [ Links ]

Uhrig, M. K., Trautmann, N., Baumgärtner, U., Treede, R. D., Henrich, F., Hiller, W., & Marschall, S. (2016). Emotion elicitation: A comparison of pictures and films. Frontiers in Psychology, 7,180. doi: 10.3389/fpsyg.2016.00180        [ Links ]

van den Hout, M. A., Eidhof, M. B., Verboom, J., Littel, M., & Engelhard, I. M. (2014). Blurring of emotional and non-emotional memories by taxing working memory during recall. Cognition and Emotion, 28(4),717-727. doi: 10.1080/02699931.2013.848785        [ Links ]

Zupan, B., & Babbage, D. R. (2016). Film clips and narrative text as subjective emotion elicitation techniques. Journal of Social Psychology, 157(2),194-210. doi: 10.1080/00224545.2016.1208138        [ Links ]

 

 

Mailing address:
Yanina Michelini
Universidad Nacional de Córdoba, Facultad de Psicología, Boulevard de la Reforma y Enfermera Gordillo, Ciudad Universitaria
CP 5000, Córdoba, Argentina
Phone: +54 351 4334064x60201
E-mail: yaninamichelini@unc.edu.ar

Received: 25/03/2018
1^st revision: 13/06/2018
Accepted: 17/08/2018
This work was supported by the Science and Technology Office (SeCyT), National University of Córdoba, Argentina, under endorsement and subsidy no. 05/P179 and the National Council for Scientific and Technical Research (CONICET), Argentina, under doctoral grant no. 3424/14.
Chamada universal MCTI/CNPq nº 01/2016 Processo 408065/2016-8
No potential conflict of interest is reported by the authors. We thank Angelina Pilatti, Ph.D., Marcos Cupani, Ph.D., and Ricardo Pautassi, Ph.D., for their help with designing the experiments and data analysis.

 

Authors' Contributions below:
Substantial contribution to the concept and design of the study: Yanina Michelini and Juan Carlos Godoy
Contribution to data collection: Yanina Michelini, Ignacio Acuña and Juan Ignacio Guzman
Contribution to data analysis and interpretation: Yanina Michelini, Ignacio Acuña and Juan Ignacio Guzman
Contribution to manuscript preparation: Yanina Michelini and Juan Carlos Godoy
Contribution to critical revision, adding intellectual content: Yanina Michelini and Juan Carlos Godoy
Conflicts of interest
The authors declare that they have no conflict of interest related to the publication of this manuscript.