ARTICLES
PSYCHOLOGICAL ASSESSMENT

 

Investigation of psychometric properties of an instrument for assessing child resilience indicators

 

Investigación de las propiedades psicométricas de un instrumento de evaluación de indicadores de resiliencia infantil

 

 

Karina da S. Oliveira^I; Tatiana de Cássia Nakano^II

^IFederal University of Minas Gerais (UFMG), Belo Horizonte, MG, Brazil
^IIPontifical Catholic University of Campinas (PUC-Campinas), Campinas, SP, Brazil

Correspondence

 

 

---

ABSTRACT

Although resilience is considered an important characteristic of coping with stressful situations, there are no Brazilian instruments available for its childhood assessment. Therefore, this paper aimed to investigate validity evidence and reliability for the instrument named "Children's Resilience Markers." The researchers investigated discriminant validity by comparing the instrument's scores under evaluation and the Child Stress Scale. A total of 136 children (67 girls) aged from 8 to 12 participated (M = 9.66; SD = 1.27). Low and negative correlations were identified, suggesting differences between the constructs. The second study aimed to understand the measure's reliability by test and retest. The participants were 155 children (71 girls), at the ages mentioned (M = 10.10; SD = 0.41). Correlation coefficients were moderate and strong, pointing to good temporal stability. We suggest that future studies should assess other psychometric qualities.

Keywords: resilience; childhood; psychological assessment; validity; reliability.

---

RESUMEN

Aunque la resiliencia es considerada un fenómeno relevante para hacer frente a situaciones estresantes, no hay instrumentos brasileños disponibles para su evaluación en la infancia. Así, este estudio buscó investigar evidencia de validez y precisión para el instrumento "Marcadores de Resiliencia Infantil". El primero estudio investigó la validez discriminante comparando los puntos del instrumento y la Escala de Estrés Infantil. Participaron 136 niños (67 niñas) de ocho a 12 años (M=9.66; SD=1.27). Se identificaron correlaciones bajas y negativas entre los instrumentos, lo que sugiere la diferenciación entre los constructos. El segundo estudio investigó la precisión del instrumento por test y el retest. Los participantes fueron 155 niños (71 niñas), con las edades mencionadas (M=10.10; SD=1.41). Los coeficientes de correlación se consideraron moderados y fuertes, mostrando una buena estabilidad temporal. Se sugiere que se realicen futuros estudios para lograr una mayor comprensión de sus cualidades psicométricas.

Palabras clave: resiliencia; infancia; evaluación psicológica; validez; precisión.

---

 

 

1. Introduction

At the end of the 1970s, motivated to understand the development of psychopathological conditions, researchers faced an unexpected phenomenon in their investigations. This phenomenon referred to some individuals' ability to endure long periods of adversity without developing psychological and psychiatric illnesses (Masten, 2018; Oliveira & Nakano, 2018). Over the years, given the range of variables involved and the complexity of positive adaptation's procedural aspects, the selected term representing this phenomenon was "resilience" (Garmezy, 1974; Rutter, 2012).

Although there is no consensus on the best definition, there is a tendency to affirm that resilience involves the idea of a positive characteristic that promotes individual adaptation and moderates the adverse effects of stress, allowing individuals to develop themselves positively when exposed to adversity situations (Satapathy, Dang, Sagar, & Dwivedi, 2020). Resilience can manifest itself whenever an individual is subjected to an adverse, stressful, real, or perceived risk condition (Castillo, Castillo-López, López-Sánchez, & Dias 2016; Masten, 2018; Oliveira & Nakano, 2018). To be understood as resilient, a coping process requires that the individual uses personal and social resources, so it would be possible to present a positive adaptation or even to overcome the risk or stressful condition (Masten, 2018; Reppold, Mayer, Almeida, & Hutz, 2012; Rutter, 2012).

For its relevance, resilience has been studied throughout the development, especially in childhood (Rutter, 2012). However, there are some specificities in children's resilience compared to adults. One example is that, due to age differences, protective factors such as self-esteem, which is important for adults, may not be relevant for children. For instance, the literature has shown that children's and adolescents' ability to prosper, despite exposure to adversity, will depend on the quality of the social interactions and the support received from the environment. In this matter, the environment can provide resources for developing or maintaining their psychological, social, and physical well-being (Jefferies, Ungar, Aubertin, & Kriellaars, 2019). Therefore, attention to adverse life situations or psychologically traumatic events experienced in childhood is justified because overall child development and mental health can be negatively affected, and its effects endure until adulthood (Satapathy et al., 2020).

Considering the impact of adverse life situations throughout development, it is important to have objective assessment strategies to evaluate children's resilience. This would allow identifying internal and external resources that could facilitate psychological intervention (Satapathy et al., 2020) and the acquisition and training of coping skills (Masten & Barnes, 2018). However, objective resilience assessment processes are possibly one of the most challenging topics in this field.

This difficulty may be related to theoretical and methodological refinements carried out over the decades, which ended up causing essential changes in understanding resilience (Harihana & Rana, 2017). Probably, for this reason, many professionals tend to evaluate aspects related to resilience through qualitative strategies, such as interviews, behavioral observations, documentary data collections, among other possibilities (Oliveira & Nakano, 2018).

Given this situation, the development and continuous application of evaluations should be encouraged to improve knowledge on resilience, and understand how it is possible to measure this characteristic and its processes (Vannest, Ura, Lavadia, & Zolkoski, 2019). Especially in childhood, an aspect that must be considered refers to the researchers' concern about the consequence of assessing resilience in childhood, that is, the later use of information resulting from the assessment process. In this case, when identifying high levels, or even high scores of resilience measures, such a result does not affirm that the individual is invulnerable (Masten, 2018).

To avoid this kind of misjudgment, resilience assessment must be understood as representing a particular moment in a person's life. In this matter, the assessment process should consider that conditions and situations can change contextual and individual characteristics, such as the presence and absence of risk and protective factors. Therefore, the evaluation of resilient traits should not be understood as definitive and immutable, especially in childhood. At this point in the life cycle, assessment processes, marked by intense changes and demands for specific care (Borges & Baptista, 2018), should identify both deficits and potentials to offer the necessary support to each individual (Masten, 2018).

Given the relevance of assessing resilience, several instrument proposals can be found in the international scientific literature. The literature review points to the existence of different tests aimed at adults (Oliveira & Nakano, 2018; Reppold et al., 2012): Resilience Scale, Connor-Davidson Resilience Scale, Suicide Resilience Inventory, and Deployment Risk and Resilience Inventory.

The authors also found various instruments for children and adolescents (Satapathy et al., 2020; Vannest et al., 2019): Child Psychosocial Distress Screener, Child Health and Illness Profile-Child Edition, Adolescent Resilience Scale, Devereux Student Strength Assessment, Resilience Scale, Health Kids Resilience Questionnaire, The Resilience Attitude and Skills Profile, Resilience Scale for Children & Adolescents, Resilience Scale for Adolescents, Adolescent Resilience Questionnaire-Revised, Assessing Developmental Strengths Questionnaire, Child and Resilience, The Bharathiar University Resilience Scale, Social and Emotional Assets and Resources Scale, Resilience Youth Development Module, Resilience Skills and Abilities Scales, Child and Youth Resilience Measure.

Despite a large number of instruments, Reppold et al. (2012) point out that most of them have items in a self-report format whose contents focus on social adjustment data. As for the instruments aimed at childhood, the authors highlighted that they are instruments that aim to assess emotional damage conditions, the presence of traumatic experiences, and adjustment. In other words, although resilience is understood as a health phenomenon, the evaluation strategy takes place through aspects more associated with psychopathological issues.

In Brazil, the objective measurement of resilience through instruments that show evidence of their psychometric qualities is not yet a reality. Considering the lack of approved instruments for assessing resilience in any age group by the Psychological Testing Assessment System (Oliveira & Nakano, 2018; Reppold et al., 2012), the authors started developing an instrument named Child Resilience Markers (CRM, Oliveira, 2019).

The instrument has, as a theoretical basis, the work of Castillo et al. (2016), which understands resilience as a complex process, with six fundamental elements: vulnerability (the individual's ability to identify the presence of a risk situation), coping (problem management strategies in order to favor positive outcome), emotional intelligence (a person's ability to observe, understand and regulate his/hers emotions), subjective well-being (related to the individual's assessment of his life history concluding that he/she is satisfied with his/her experiences), locus of control (ability to assess the contingencies arising from his/her behaviors, especially those related to academic development) and ability (capacity to use cognitive resources, seeking excellence, and positive results).

It is important to highlight that Castillo et al. (2016) proposed that such elements point to aspects associated with healthy development and identified as essential components to the positive adaptation processes (Harihana & Rana, 2017; Masten, 2018; Masten & Barnes, 2018).

Bearing in mind that assessment processes with children must consider developmental issues, using preferably playful strategies (Borges & Baptista, 2018), the instrument's items were developed in a short stories format, which is interrupted when the main character must decide on how to act. At this point, the child is asked to answer what he/she would do if he/she were the main character, and he/she must choose one of three options presented. It is important to inform you that all the items are illustrated to facilitate understanding of the history and engagement with the task.

The instrument has already been subjected to different studies aimed at investigating its psychometric properties. Among those studies, it is possible to cite the ones related to validity evidence based on the response process, validity evidence based on content, validity evidence based on internal structure, and precision and validity evidence based on the relationship with external criteria-type variables (Oliveira, 2019; Oliveira & Nakano, 2020; Oliveira, Nakano, & Silva, 2019), which pointed to positive evidence of validity and precision.

However, considering that the development of measures must involve, procedurally, different sources of validity evidence and precision (American Educational Research Association, American Psychological Association, & National Council of Measurement in Education, 2014), the present study was conducted, aiming to search for validity evidence based on the relationship with external variables the discriminant type. The investigation of the relationship between the scores of an instrument with another that evaluates something that, theoretically, is not related to the construct is an important part of the process of developing psychological measures (Ambiel & Carvalho, 2017; Freitas & Damásio, 2017; Reppold, Gurgel, & Hutz, 2014). Thus, if we consider that the literature states that resilience is associated with coping with stressful situations, acting to help the process of achieving a good outcome in the face of adverse conditions (Garmezy, 1974; Masten, 2018; Rutter, 2012), the hypothesis thought for the study presented here was based on the idea that these two phenomena should be negatively related.

A second study focused on the instrument's accuracy was also conducted to add evidence to the previous ones. Among the different possibilities of precision studies, we can mention the investigations with test and retest design selected for the second study. It is worth clarifying that this type of study consists of calculating the correlations obtained from the same instrument at two different times (Pasquali, 2011), with high magnitude correlations expected between the participants' results at these two moments.

Given the above, this research study had the following objectives: to verify the validity evidence based on the relationship with external variables and precision using temporal stability for an instrument which aims to assess resilient characteristics in Brazilian children aged between 8 and 12 years old to be possible to deepen the understanding of the instrument's psychometric properties.

 

2. Study 1 - Search for validity evidence based on the relationship with external variables of the discriminant type

2.1 Participants

Participants were selected by convenience, in a total of 136 children, 67 female (49.26%), aged between 8 and 12 years old (M= 9.66; SD= 1.27), 3rd-year students (n = 35), 4th year (n = 37), 5th year (n = 27) and 6th year (n = 37) of an elementary school in a public school located in the interior of São Paulo State. Figure 2.1.1 presents more details about the participant's characteristics.

2. 1 Instruments

2.1.1 Child Resilience Markers - CRM (Oliveira, 2019)

The authors developed the instrument to assess resilient characteristics in Brazilian children aged between 8 and 12 years old, using 22 illustrated items in short stories. The main characters are named Nina and Nino, and they have the same age as the respondent. In each item, the characters are in a challenging situation. It is up to the child to decide what to do if he/she were in the characters' place by selecting one of the three outcome alternatives, one non-resilient, one adequate, and the other resilient. It is important to highlight that the presentation of these options is randomized, so a pattern is not established during the task.

It is possible to apply the instrument individually or collectively. Also, if the child shows any difficulty with reading skills during the application, the items can be read aloud. On the other hand, if the child already mastered this skill, he/she can proceed autonomously. The estimated response time is approximately 30 minutes.

The items are divided into six subscales: 1) Vulnerability (four items, maximum 8 points); 2) Coping (three items, maximum 6 points); 3) Emotional intelligence (four items, maximum 8 points); 4) Subjective well-being (three items, maximum 6 points); 5) Locus of control (five items, maximum 10 points); and 6) Ability (three items, maximum 6 points). The total of possible points on the instrument is 44.

2. 1.2 Child Stress Scale - CSS (Lipp & Lucarelli, 2005)

This instrument aims to verify the presence of stress symptoms in children aged 6 to 14. The scale assesses these symptoms through four factors: physical reactions (CSS 1), psychological reactions (CSS 2), psychological reactions with depressive components (CSS 3), and psychophysiological reactions (CSS 4), as well as a total score. Altogether, 35 items are presented on a five-point Likert scale, ranging from 0 to 4.

The process of applying the instrument involves asking the child to read a statement and, then, signal how often he/she feels the symptom described, using a circle divided into four equal parts, which is presented at the end of each item. To do so, the child should paint the number of parts of the circle that he/she thinks corresponds to the frequency with which symptoms he/she feels (1 part = a little; 2 parts = sometimes; 3 parts = almost always; 4 parts = always).

A series of studies on its psychometric qualities were developed and pointed to positive evidence of validity and precision (Lipp & Lucarelli, 2005; Lucarelli & Lipp, 1999).

2. 2 Procedures

The authors submitted the present study to the evaluation of the Ethics Committee in Research with Human Beings of the Pontifical Catholic University of Campinas - SP. After approval (CAAE 66606517.5.0000.5481), data collection was carried out collectively in the classroom, with an average duration of 80 minutes. It is essential to highlight that the two instruments selected for this study were applied in a single session, initiated by CSS and followed by CRM. Four visits to the school were necessary, with each application taking place in one school year. It is also important to inform you that there were no refusals from participants during the applications. After data collection, the researchers offered lectures on the development of resilience in children and its relationship with child stress, addressed to teachers and other school employees.

2. 3 Results

After the applications, a database was built containing participants' sociodemographic information (sex, age, and education level) and their responses to each of the instruments' items. With assistance of the Statistical Package for the Social Sciences (SPSS) v. 21, the authors performed the sample normality test to verify which type of analysis would be more appropriate. The results obtained through the Kolmogorov-Smirnov test varied between 0.156 and 0.251 with p values less than 0.001, being interpreted as highly significant for the CRM. In turn, values between 0.060 and 0.159 were observed with p values less than 0.05, understood as significant for CSS measurements. Such information indicated the absence of normality for all factors and the total score, both for the CRM and the CSS. Hence the authors used the analysis of Spearman's correlation coefficients.

The correlation between the instruments is shown in Figure 2.3.1 and indicated positive correlations of low magnitudes and negative correlations between the variables analyzed in the instruments, which varied between rs = -0.219 and rs = 0.052. Among the dimensions evaluated by the CRM, it can be seen that only three of them showed significant correlations with any measure of the CSS: Emotional intelligence (with psychological reactions with depressive components, psychophysiological responses, and total score), locus of control (with psychological reactions with depression components), ability (with psychological reactions) and the total of the instrument with psychological reactions with depressive components. It should be noted that the results point to a small magnitude relationship between the instruments, as expected from theoretical assumptions and for studies of a divergent nature.

It is worth highlighting that no significant relationship was found between the CSS's physical reactions and CRM measures. The factors physiological reactions, psychophysiological, and the CSS total score showed a significant and negative correlation with only one CRM measure. On the other hand, the measure of psychological reactions with depressive components (CSS) was the one that was most related to the total CRM.

 

3. Study 2 - Instrument's precision through temporal stability

3.1 Participants

Participants were selected by convenience, and this second study had the collaboration of 155 children, students from another public school, also located in the interior of São Paulo State. From these 155 participants, 71 (45.80%) were girls, with ages varying between 8 and 12 years old (M = 10.10; SD = 1.41) and were in the 3rd year (n = 39), 4th year (n = 39), 5th year (n = 43) and 6th year (n = 34).

It is important to highlight that this is the number of participants who responded to the instrument at both times (test and retest). The sample loss resulting from this criterion was 26 children who were not present at school at the time of data collection, and no information on the reasons for the absence was provided. Further details on the characteristics of the participants of Study 2 are presented in Figure 3.1.1.

3. 2 Instrument

For this study, the only instrument used was the Child Resilience Markers, previously described in Study 1.

3. 3 Procedures

It is important to consider that there is no suggestion about the adequate period to establish the interval between the first and the second application (Bonanno, Romero, & Klein, 2015). This lack of guidance is based on the fact that the resilient process can vary for each individual due to aspects related to life history, the intensity of the stressful event, chronicity of risk exposure, among other variables (Masten, 2018; Reppold et al., 2012). Because of that, the study followed the general recommendations of the psychometric guidelines for this investigation, which suggest an interval between two and four weeks from the first to the second test application (Pasquali, 2011), having decided on the time of two weeks between the tests to avoid that any other external event could interfere with the investigation of the instrument's temporal stability.

As reported in Study 1, it is important to inform the reader that the study was approved by the Human Research Ethics Committee of the Pontifical Catholic University of Campinas - SP (CAAE 66606517.5.0000.5481). Initially, the researchers gave a lecture on the development of resilience and socio-emotional competencies throughout childhood for teachers from different educational levels. In the same period, the Terms of Free and Informed Consent and Terms of Assent were sent to the school's students. After these actions, the researchers scheduled the days for the first and the second test application, carried out collectively, with an average duration of 30 minutes each, observing 15 days between them.

 

4. Results

After each application moment, a database was built containing sociodemographic information of the participants and their answers, indicating to which moment (test or retest) the answers belonged. With the assistance of the SPSS v.21 statistical package and the Jamovi software (The Jamovi Project, 2019), the authors performed the normality test to verify the most appropriate analysis type. The Kolmogorov-Smirnov test results were significant for this sample, confirming the need to use non-parametric statistics. The values observed varied between 0.111 and 0.194, with p values below 0.05 for all CRM measurements. In its turn, there was a variation in values between 0.095 and 0.236 with p less than 0.05 for all CRM measurements in the second moment. Consequently, the authors employed Spearman's correlation coefficient analysis.

According to the guidelines for this type of study, high magnitude correlations are expected (Pasquali, 2011). To support the interpretation of the data, Miot's (2018) guidelines were used, according to which, values between 0.31 and 0.50 are understood as weak correlations, between 0.51 and 0.70 are understood as moderate, between 0.71 and 0.90 are strong correlations, and above 0.90 are understood as robust correlations.

From the results (see Figure 4.1), it is possible to observe that the correlations were analyzed considering the interaction between the first testing moment (identified as test application 1) and the second testing moment (identified as test application 2). The correlation coefficients between each dimension of the MRI varied between rs = 0.313 and rs = 0.777. All coefficients were positive and significant, as expected for this type of study.

The highest values were found for the total CRM (rs = 0.777, p<0.001), followed by the Coping factor (rs = 0.700; p<0.001). In contrast, the lowest value identified was for the Subjective Well-being factor (rs = 0.313; p<0.001). Therefore, as it is possible to observe, the correlations obtained can be interpreted as strong for the instrument's total, moderate for Vulnerability, Coping, Emotional Intelligence, Locus of Control and Ability, and weak for the Subjective Well-Being factor. In general, these results positively pointed to the instrument's temporal stability.

 

5. Discussion

Based on the recommendation of the scientific literature about the importance of the cumulative and continuous character of studies of evidence of validity (Ambiel & Carvalho, 2017; Primi, 2012), in this study, we sought to deepen the understanding of the psychometric properties of an instrument for assessing children's resilience to add evidence to those investigated in previous studies (Oliveira, 2019; Oliveira & Nakano, 2020; Oliveira et al., 2019).

Although this recommendation is present in the literature, what is seen in practice is that most of the international measures for the assessment of resilience in children have focused their efforts on the investigation of a limited amount of validity evidence, especially those based on the content and relationship with external criteria-type variables (Jordans et al., 2009; Riley et al., 2013). Few instruments had investigated their validity evidence based on external variables of the consequential, criterion, or construct types (Vannest et al., 2019). Because of that, some measures have insufficient studies on their psychometric properties, for example, not identifying cutoff points or groups for comparison or being predominantly tested in school-age children who did not experience traumatic events (Satapathy et al., 2020).

The proposition of new studies to investigate the psychometric qualities of CRM is based on these gaps and in the perception that resilience differs among locations, context, and nature of adverse circumstances. These characteristics make it difficult to establish a generic scale that adapts to all places, contexts, and events. In this sense, the instrument reported here was designed to form a culturally contextualized and operationalized scale for use in the Brazilian children population (Oliveira, 2019; Oliveira & Nakano, 2020; Oliveira et al., 2019).

Considering the result reported by Satapahty et al. (2020), after reviewing the instruments for assessing child resilience, no scale tested the divergent validity, considering psychopathologies or impairments in global functioning, which was the study's purpose 1, presented here. The literature review shows the importance of resilience understood, not only as an action to cope with adverse situations, but also that it is involved in resistance processes and combating stress (Garmezy, 1974; Masten, 2018; Rutter, 2012). There are also theoretical indications that high perceptions of stress have been associated with lower resilience (Castillo et al., 2016). Based on these assumptions, the authors chose to conduct this study, promoting a comparison between the CRM and CSS scales scores. It is important to remember that CSS has the main goal of identifying the presence of stress and its phase.

Therefore, from these theoretical guidelines, it was possible to develop the hypothesis that the comparison between these two instruments would result in correlations of low magnitudes with negative directions, since resilience has been pointed out as a construct that helps coping with stress. The results confirmed this hypothesis, since six significant interactions were found, and all of these presented low and negative magnitudes. Such results corroborate the hypothesis that resilience is involved in processes to combat stress (Garmezy, 1974; Masten, 2018; Rutter, 2012), possibly associated with reducing psychological symptoms, psychophysiological symptoms and psychophysiological symptoms with depressive components.

More specifically, seeking an objective understanding between CRM and CSS factors, what could be observed is that the Emotional intelligence factor of the CRM instrument was related to the factor Psychological reactions with depressive components, Psychophysiological responses, and the Total of the CSS instrument. According to Castillo et al. (2016), the concept of Emotional Intelligence, as a dimension involved in the resilient process, refers to an individual's ability to identify, understand, and regulate her/his emotions. Psychological reactions with depressive components are represented by symptoms such as desires or aggressive behaviors, not wanting, or even unwilling to perform an activity that previously interested the child (Lipp & Lucarelli, 2005). In its turn, Psychophysiological reactions are represented by symptoms such as feeling shy, ashamed, and nervous (Lipp & Lucarelli, 2005). It is possible to infer that emotional management capacity is associated with coping with these stress symptoms. It is also worth pointing out that Emotional intelligence was significant for the total of CSS's items, pointing out that emotional intelligence may help dealing with broad stress symptoms.

The authors also identified a low and negative correlation between the Locus of control factor (CRM) and the Psychological reactions with the depressive components (CSS) factor. For Castillo et al. (2016), this dimension of resilience (Locus of control) refers to an individual's ability to control her/his behavior, especially regarding academic performance, in order to achieve adequate results. For Lipp and Lucarelli (2005), psychological reactions with depressive components also include symptoms of stress related to academic issues, such as being able (or not) to learn things, good memory capacity ("I have been feeling very forgetful lately"), and interest for studies. Therefore, the relationship presented by these factors is theoretically coherent. It is possible to affirm that controlling academic behavior is involved in processes of coping with psychological symptoms with depressive components.

Concerning the Ability factor (CRM), results showed a low and negative correlation along with the Psychological Reactions (CSS) factor. According to Castillo et al.'s (2016) model, this dimension refers to an individual's ability to seek excellence in their activities, behaving objectively for his/her goals. In turn, Lipp and Lucarelli (2005) state that stress psychological reactions involve concerns about future adverse events, sleeping difficulty, feeling scared, nervous, and distressed. Therefore, from the results, it is possible to infer that individuals who exhibit behaviors seeking excellence in their activities have a lower incidence of psychological symptoms of stress.

Finally, it was observed that the general factor, CRM's total, presented a low, significant, and negative correlation with the factor of Psychological reactions with depressive components (CSS). Thus, based on the assumptions of Castillo et al. (2016), it is possible to affirm that resilience is characterized by the dynamic process between the different fundamental elements (factors), which would be personal resources that would be involved in a positive adaptation response/behavior in the face of a real or perceived adversity. As previously presented, Lipp and Lucarelli (2005) argue that psychological reactions with associated depressive components involve aspects related to aggressive behavior, lack of desire, or willingness to perform activities that previously had one's interest, and also related to academic difficulties. From this, it is possible to state that resilient skills, as assessed in CRM, are associated with coping with such stress symptoms.

Therefore, after conducting this study, it is possible to state that validity evidence was found based on the relationship with discriminant external variables. As expected, low-magnitude and negative correlations were found between the instrument's measurements to confirm that they are different constructs and that they act in reverse; that is, the higher the level of resilience, the lower the possibility of presenting symptoms of stress. Thus, resilient characteristics could act as a protective factor, helping the individual to face, in a more positive way, potentially stressful events. Such findings are in line with theoretical postulates developed by researchers such as Bonanno et al. (2015), Garmezy (1974), Masten (2018), Masten and Barnes (2018), and Rutter (2012) regarding the negative relationship between resilience and stress.

About study 2, its importance is based on the fact that precision is not a quality of the instrument, but it concerns, above all, the resulting scores. When evaluating both the subscales and the total score of the instrument, the results obtained could demonstrate that CRM can be used safely, since the positive results complemented previous results obtained by Oliveira (2019) instrument's accuracy through the internal consistency.

Considering that studies of this nature have not been conducted for other measures developed to assess childhood resilience (Jordans et al., 2009; Riley et al., 2013), Study 2 was chosen to investigate its accuracy through temporal stability. From the results, it is possible to affirm that good evidence for the instrument's accuracy was identified, since the coefficients obtained, for the most part, can be classified as of strong and moderate magnitude. Only in the Subjective Well-Being factor can the observed coefficient be classified as weak (Miot, 2018). It is possible that, as pointed out by Giacomoni (2002), variables such as the age of the participants, economic conditions, and perception of the quality of social interactions may have influenced these results. According to the author, such variables can directly affect children's perception of this phenomenon, that is, Subjective well-being.

Still, about this dimension, it is worth considering that items contents dealing with this aspect present themes in which the child is encouraged to consider his/her choices (example: in a story, the main character must respond to a provocation of his/her sister who says her toy is more fun compared to his/her), lived experiences (example: in a story, the main character listens to a friend report on the activities carried out during holidays and must inform how he/she feels when comparing the friend's activities with his/hers) and evaluation of their own story (example: in a story, the main character is invited to write about his/her life, and the child is asked to reflect on the content of this writing). Given these issues, it is noted that the influences suggested by Giacomoni (2002) are adequate hypotheses to justify the low level of temporal stability in this factor. Therefore, in future studies, investigating the influence of variables such as sex, age, and type of school (whether private or public) could help to better understand the variables that have action on this factor.

To conclude, the relevance of developing instruments aimed at assessing children's resilience is justified because several situations that may require this skill are increasingly present nowadays (traumas, disasters, domestic violence, mistreatment, neglect, sexual abuse, medical conditions, bullying, drug use, among others). In this sense, the child's specific assessment on resilience factors is essential for identifying vulnerabilities and protection factors and, subsequently, their integration with psychological intervention.

In this sense, the continuous search for validity evidence during the development of measures can improve our understanding of this evolving construct and provide more sensitive ways to monitor children's and adolescents' growth and progress. Such relevance is reinforced by Vannest et al.'s (2019) perception that the need for valid high-quality resilience measures remains.

It is important to note that the limitations were present throughout this investigation. We can mention the differences between the instruments, as CRM is characterized by items in the format of short stories, in which competencies related to the resilience domains are evaluated. In contrast, CSS is characterized by being a self-report instrument, in which it assesses the presence of stress symptoms. Given these differences in the response process, different cognitive resources are required from the subject. While CRM is based on answers that inform what the child would do in the problem situation, the CSS items involve other processes related to understanding the item, identifying (or not) with the content, and transforming the occurrence frequency among the different possible levels. The differences between processes, including the question regarding self-knowledge, may have influenced the results.

Another issue that must be considered is that the participant's sample in each study came from only one school, so that sampling variability was not achieved. In future studies, the expansion of the participant's sample is recommended in order to understand, more deeply and based on a more varied and representative sample, the results presented here.

Likewise, it is necessary to highlight that future studies must be conducted to achieve greater depth about CRM's psychometric qualities, such as standardization involving participants from different regions of Brazil, analysis of independent variables influence such as sex, age, and type of school, as well as item analysis, so that psychologists can interpret the results obtained through the instrument safely. Consequently, it could be made available for professional use.

 

References

Ambiel, R. A. M., & Carvalho, L. F. (2017). Validade e precisão de instrumentos de avaliação psicológica. In M. R. C. Lins, & J. C. Borsa (Orgs.), Avaliação psicológica: Aspectos teóricos e práticos (pp. 115-125). Petrópolis, RJ: Vozes.         [ Links ]

American Educational Research Association, American Psychological Association, & National Council of Measurement in Education (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.         [ Links ]

Bonanno, G. A., Romero, S. A., & Klein, S. I. (2015). The temporal elements of psychological resilience: An integrative framework for the study of individuals, families, and communities. Psychological Inquiry, 26(2),139-169. doi:10.1080/1047840X.2015.992677        [ Links ]

Borges, L., & Baptista, M. N. (2018). Avaliação psicológica e psicoterapia na infância. In M. R. C. Lins, M. Muniz, & L. M. Cardoso (Orgs.), Avaliação psicológica infantil (pp. 71-90). São Paulo, SP: Hogrefe.         [ Links ]

Castillo, J. A. G., Castillo-López, A. G., López-Sánchez, C., & Dias, P. C. (2016). Conceptualización teórica de la resiliencia psicosocial y su relación con la salud. Health and Addictions, 16(1),59-68. doi:10.21134/haaj.v16i1.263        [ Links ]

Freitas, C. P. P., & Damásio, B. F. (2017). Evidências de validade com base nas relações com medidas externas: Conceituação e problematização. In B. D. Damásio, & J. C. Borsa (Orgs.), Manual de desenvolvimento de instrumentos psicológicos (pp. 101-117). São Paulo, SP: Vetor.         [ Links ]

Garmezy, N. (1974). The study of competence in children at risk for severe psychopathology. In E. J. Anthony, & C. Koupernik (Eds.), The child in his family: Children at psychiatric risk (pp. 77-97). New York, NY: Wiley.         [ Links ]

Giacomoni, C. H. (2002). Bem-estar subjetivo infantil: Conceito de felicidade e construção de instrumentos para avaliação (Doctoral thesis). Universidade Federal do Rio Grande Sul, Porto Alegre, RS, Brazil.         [ Links ]

Harihana, M., & Rana, S. (2017). Conceptual complexity of resilience: Synergy approach to measurement. In U. Kumar (Ed.), The routledge international handbook of psychosocial resilience (pp. 101-118). New York, NY: Routledge.         [ Links ]

Jefferies, P., Ungar, M., Aubertin, P., & Kriellaars, D. (2019). Physical literacy and resilience in children and youth. Frontiers in Public Health, 7, 346. doi:10.3389/fpubh.2019.00346        [ Links ]

Jordans, M. J. D., Komproe, I. H., Tol, W. A., & De Jong, J. T. V. M. (2009). Screening for psychosocial distress amongst war-affected children: Cross-cultural construct validity of the CPDS. Journal of Child Psychology and Psychiatry, 50(4), 514-523. doi:10.1111/j.1469-7610.2008.02028.x        [ Links ]

Lipp, M. E. N., & Lucarelli, M. D. M. (2005). Escala de stress infantil: ESI: manual. São Paulo, SP: Casa do Psicólogo.         [ Links ]

Lucarelli, M. D. M., & Lipp, M. E. N. (1999). Validação do inventário de sintomas de stress infantil - ISS - I. Psicologia: Reflexão e Crítica, 12(1),71-88. doi:10.1590/S0102-79721999000100005        [ Links ]

Masten, A. S. (2018). Resilience theory and research on children and families: Past, present, and promise. Journal of Family: Theory & Review, 10(1),12-31. doi:10.1111/jftr.12255        [ Links ]

Masten, A. S., & Barnes, A. J. (2018). resilience in children: Developmental perspectives. Children, 5(98), 1-16. doi:10.3390/children5070098        [ Links ]

Miot, H. A. (2018). Análise de correlação em estudos clínicos e experimentais. Jornal Vascular Brasileiro, 17(4),275-279. doi:10.1590/1677-5449.174118        [ Links ]

Oliveira, K. S. (2019). Marcadores de Resiliência Infantil: Construção de Instrumento e investigação de suas qualidades psicométricas (Psychology Doctoral Thesis). Pontifícia Universidade Católica de Campinas, Campinas, SP, Brazil.         [ Links ]

Oliveira, K. S., & Nakano, T. C. (2020). Desenvolvimento e investigação de evidências de validade para o instrumento Marcadores de Resiliência Infantil. Psico-USF, 25(4),737-749. doi:10.1590/1413/82712020250412        [ Links ]

Oliveira, K. S., & Nakano, T. C. (2018). Avaliação da resiliência em Psicologia: Revisão do cenário científico brasileiro. Psicologia em Pesquisa, 12(1),73-83. doi:10.24879/2018001200100283        [ Links ]

Oliveira, K. S., Nakano, T. C., & Silva, G. M. (2019). Children's Resilience Markers: Criterion-based validity evidences. Estudos de Psicologia (Campinas), 36, e180142. doi:10.1590/1982-0275201936e180142        [ Links ]

Pasquali, L. (2011). Técnicas do exame psicológico - TEP: Manual. São Paulo, SP: Casa do Psicólogo/Conselho Federal de Psicologia.         [ Links ]

Primi, R. (2012). Psicometria: Fundamentos matemáticos da teoria clássica dos testes. Avaliação Psicológica, 11(2),297-307. Retrieved from http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712012000200015        [ Links ]

Reppold, C. T., Gurgel, L. G., & Hutz, C. S. (2014). O processo de construção de escalas psicométricas. Avaliação Psicológica, 13(2),307-310. Retrieved from http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712014000200018        [ Links ]

Reppold, C. T., Mayer, J. C., Almeida, L. S., & Hutz, C. S. (2012). Avaliação da resiliência: Controvérsia em torno do uso de escalas. Psicologia: Reflexão e Crítica, 25(2),248-255. doi:10.1590/S0102-79722012000200006        [ Links ]

Riley, A. W., Coghill, D., Forrest, C. B., Lorenzo, M. J., Ralston, S. J., & Spiel, G. (2006). Validity of the health-related quality of life assessment in the ADORE study: Parent Report Form of the CHIP - Child Edition. European Child & Adolescent Psychiatry, 15(1),63-71. doi:10.1007/s00787-006-1009-6        [ Links ]

Rutter, M. (2012). Resilience as a dynamic concept. Development and Psychopathology, 24(1),335-344. doi:10.1017/S0954579412000028        [ Links ]

Satapathy, S., Dang, S., Sagar, R., & Dwivedi, S. N. (2020). Resilience in children and adolescents survived psychologically traumatic life events: A critical review of application of resilience assessment tools for clinical referral and intervention. Trauma, Violence, & Abuse, 1-13. doi:10.1177/1524838020939126        [ Links ]

The Jamovi Project. (2019). Jamovi (Version 0.9) [Computer Software]. Retrieved from https://www.jamovi.org        [ Links ]

Vannest, K. J., Ura, S. K., Lavadia, C., & Zolkoski, S. (2019). Self-report measures of resilience in children and youth. Contemporary School Psychology. doi:10.1007/s40688-019-00252-1        [ Links ]

 

 

Correspondence:
Karina da Silva Oliveira
Universidade Federal de Minas Gerais, Secretaria Da Pós-Graduação Em Psicologia: Cognição e Comportamento
Av. Antônio Carlos, 6.627
Belo Horizonte, MG, Brazil. CEP 31270-901
E-mail: karina_oliv@yahoo.com.br

Submission: 05/02/2020
Acceptance: 09/10/2020
Funding: The authors would like to thank the Coordination for the Improvement of Higher Education Personnel (CAPES) and the Instituto Ayrton Senna for their financial support.

 

 

Authors' notes:
Karina da S. Oliveira, Postgraduate Program in Psychology: Cognition and Behavior (PPGCOGCOM/UFMG), Federal University of Minas Gerais (UFMG); Tatiana de Cássia Nakano, Postgraduate Program in Psychology (PPGPsicologia), Pontifical Catholic University of Campinas (PUC-Campinas).