ARTICLES
PSYCHOLOGICAL EVALUATION

 

Assessment of metatextual awareness and its prediction of reading comprehension

 

Evaluación de la conciencia metatextual y su predicción de comprensión de lectura

 

 

Neide de B. Cunha^I; Acácia Aparecida A. dos Santos^II

^IEducation Master's Degree, Postgraduate Course Stricto Sensu, University of do Vale do Sapucaí (Univás), Minas Gerais, MG, Brazil
^IIGraduate Program in Psychology, University of São Francisco (USF), São Paulo, SP, Brazil

Mailling address

 

 

---

ABSTRACT

This study aimed to reveal evidence of the validity of the Assessment Questionnaire of Metatextual Awareness (QACM). The main objective was to verify whether metatextual awareness could predict reading comprehension, and the secondary objectives were: to verify metatextual awareness and reading comprehension through two Cloze tests and the QACM; to search criterion validity evidence based on the variable "school year"; and to identify the internal consistency index of the items. The participants were 277 elementary school students from a public school in the state of São Paulo. The results showed that the students' average scores increased from the 3^rd to the 5^th years of school, demonstrating a criterion validity of the school year. The QACM got a good internal consistency index of 0.80. The performance on the QACM explains 46% of the performance variance on the Cloze, showing that they are shared skills.

Keywords: metatextual awareness; reading comprehension; elementary school; Cloze; validity.

---

RESUMEN

Este estudio buscó evidencias de validez para el Cuestionario de Evaluación de la Conciencia Meta textual (QACM). El objetivo principal fue verificar si la consciencia meta textual podría ser predictiva de la comprensión de lectura, y los secundarios fueron: averiguar la consciencia meta textual y la comprensión de lectura con dos testes de Cloze y el QACM; buscar evidencias de validez de criterio con base en el año escolar; e identificar el índice de consistencia interna de los ítems. Participaron 277 estudiantes de educación primaria de una escuela pública de São Paulo-Brasil. Los resultados mostraron que los puntajes medios de los estudiantes aumentaron del 3º al 5º año, lo que demuestra la validez de criterio para el año escolar. El QACM tuvo un buen índice de consistencia interna. Se encontró que el 46% de la varianza en el rendimiento del Cloze se explica por el desempeño en el QACM, mostrando que son habilidades compartidas.

Palabras clave: consciencia meta textual; comprensión de lectura; educación primaria; Cloze; validez.

---

 

 

1. Introduction

Cognitive Psychology conceptualizes reading as a skill that involves several cognitive components. Sternberg (2010) appointed that reading comprehension comprises basic procedures, including attention, operating memory, and lexical processes (phonological, morphological, and syntactic awareness). In addition, other more complex processes, called high-level, are also involved, including inference and reader monitoring.

For the 1^st to the 4^th grade, the National Curriculum Parameters for Portuguese Language (Brasil, 1997) considers that, in the linguistic analysis, some activities can be classified as epilinguistic and metalinguistic, which are actions of reflection on the language, but which differ in their purposes. In the epilinguistic activities, the reflection is focused on the use, within the linguistic activity in which it is realized, while the metalinguistic activities are related to a type of analysis directed to the description, by means of the categorization and systematization of the linguistic elements, that is to say, it is the use of a metalanguage that makes it possible to speak about the language. As an integral part of a didactic situation, metalinguistic activity develops in the sense of allowing the student to survey regularities of aspects of the language, to systematize and classify their specific characteristics.

Among the metalinguistic abilities, Gombert (1992) conceptualizes the metatextual knowledge skills, which is the knowledge about the properties of the text, analyzed based on intentional monitoring, in which the subject focuses on the elements that compose it. Gombert (2013) explains that, in metalinguistic ability, the reflection and analysis of the text need to occur in an intentional way, which does not prevent the consideration of the implicit learning issues present throughout the development and learning process of the subject.

These considerations imply the need for students' systematic exposure to different types of textual genres at school. Zaboroski and Oliveira (2014) observed, however, that teachers did not have the necessary knowledge about them and the discourse typology and genres in the pedagogical practices that involved the textual genres. It should be noted that many textual genres exist, such as letters, recipes, chronicles, packaging insert leaflets, articles, reviews, among others. Each text genre serves a kind of need, providing a form of social interaction. In this sense, the genre is structured in a specific way, using the so-called textual types, which are ways of organizing texts. Most authors identify three textual types: narration, description, and argumentation. The more textual knowledge the readers have and the greater their exposure to all kinds of text, the easier the understanding of the text becomes, as the knowledge of textual structures and types of discourse considerably determines their expectations regarding the texts and these expectations play an important role in the comprehension (Kleiman, 2004). We can affirm that the genres guide the understanding, as they are forms of social action, and not only mere textual forms. Spinillo and Simões (2003) indicate that the structure may be more prominent in certain textual genres than in others, favoring their identification by children. Thus, different textual genres require different forms in the comprehension process, considering that the textual genre is an important indicator, as the production and the treatment of a scientific article are distinct from those of a narrative story, a newspaper comic, or a horoscope (Marcuschi, 2010). In a later work, Spinillo and Almeida (2015) argue that the properties of texts associated with the nature of inferences play an important role in the comprehension, both in relation to performance and to how the reader deals with different classes of inference.

Admitting that metatextual knowledge favors reading comprehension, Cunha and Santos (2014) sought to develop an instrument for the evaluation of metatextual awareness, which the authors have called the Metatextual Awareness Assessment Questionnaire (QACM). Two studies have already been developed to derive validity evidence for the QACM. In the first one, containing 11 open questions, the content validity was evidenced through an analysis by judges; considering the school years; the validity of related constructs was evidenced with regard to reading comprehension. The internal consistency of the items was also verified (Santos & Cunha, 2012).

In the second study, containing 11 closed questions and four response options, the following were evidenced: validity based on the relationship with other variables by the school years, and related constructs, with reading comprehension, however, there was a ceiling effect for the children in the 5^th grade of elementary education (Cunha & Santos, 2014). Thus, the authors considered that the instrument could contain a lag regarding the number of questions, with the need to include more textual genres, and also the number of alternatives to choose the correct answer.

The importance of reading comprehension has favored many diagnostic studies, involving elementary school students, as problems in this skill can compromise the students' academic performance, as appointed in the study by Oliveira, Boruchovitch, & Santos (2008). The most used tool in Brazil has been the Cloze Test, as reported in the study by Cunha, Suehiro, Oliveira, Pacanaro, & Santos (2009). Next, studies that survey metatextual skills will be presented.

As for research focusing on metatextual awareness, we found studies that explored texts such as the letter and the newspaper article to analyze the criteria children adopted to identify them. They also analyzed the possible progression in relation to these criteria as age, schooling, and social class advance; and the role of specific interventions on metatextual awareness and textual production (Albuquerque & Spinillo, 1998; Ferreira & Spinillo, 2003; Spinillo & Melo, 2008).

More recently, the studies by Oliveira and Braga (2012) were found, which problematized the conception of text genres present in the National Curricular Parameters, in relation to a Brazilian institutional school model, suggesting a program to support the pedagogical practice, using the narrative text genre, based on its macrolinguistic aspects. The research results showed that the written productions of the students participating in the study evidenced important indicators of growth and appropriation of the text genre discussed. Rodrigues and Vilela (2012) also worked with narrative texts and analyzed the influence of the production situation of this type of text. The authors concluded that metatextual awareness sessions increased the story writing skills, especially in the 3^rd grade, being more noticeable in the characteristics than in the levels of the narratives.

On the other hand, Pinheiro and Guimarães (2013) focused on the production of opinion texts as an activity facilitated by the metacontextual skills related to coherence and cohesion. The authors considered that the intervention proved to be effective both as a research tool and as a pedagogical instruction instrument. Spinillo and Silva (2014), then, investigated the metatextual awareness in children, based on the analysis of the criteria they adopt when defining history, letter, and newspaper article. The students' definitions were classified according to the adopted criteria (linguistic conventions, structure, content, functions, material characteristics of the bearer) and how they were considered (isolated or combined). According to the authors, the data contribute to the understanding of the development of metatextual knowledge.

To proceed with the scarce research on metatextual awareness, this paper aims to find evidence of validity for a construct measurement tool, the QACM. Therefore, the main objective was to verify if the metatextual awareness could predict the reading comprehension. The secondary objectives were: to ascertain the metatextual awareness and the reading comprehension through two Cloze tests and the QACM; to seek evidence of criterion validity based on the variable school year; and to identify the internal consistency index of the items.

 

2. Method

2.1 Participants

The study participants were 277 students from a public school in the countryside of Sao Paulo state. Of the total, 148 were boys and 129 were girls, whose ages ranged from 7 to 11 years, with an average age of 9.22 and a standard deviation of 0.85. The students were taking the 3^rd (n = 70, 25.3%), 4^th (n = 104, 37.5%), and 5^th (n = 103, 37.2%) years of Elementary Education I.

2.2 Instruments

2.2.1. Cloze Tests

These are texts for the evaluation of reading comprehension. The stories used were elaborated by Santos (2005): "The Princess and the Ghost" and "An Unfortunate Revenge," specially created to verify the reading comprehension of elementary-school students. The stories were prepared according to the Cloze technique, in its traditional version, omitting the fifth words. At the place of the omission, a line was placed that was proportional to the excluded word.

The texts contain approximately 100 words each and 15 blanks, which the participants should complete with words they consider appropriate to provide meaning to the text. Here is an excerpt from the text "The Princess and the Ghost": "Once upon a time, there was a princess who lived very unhappy in her palace. She was in love with ___ ghost who lived in hiding ___".

2.2.2. Metatextual Awareness Assessment Questionnaire

Constituted by 20 texts, created by Santos and Cunha (2011), presented without configuring the respective supports, such as comic, newspaper, computer screen, among others. Each item was elaborated with five alternative answers options, all corresponding to text genres. Some texts for the preparation of the questionnaire were obtained from the Read and Write program of the São Paulo State Government Secretary of Education, and from the Literacy Teacher Training Program of the Ministry of Education Secretary of Elementary Education, Module 1, January 2001. Others were invented, such as lists and emails, or taken from the internet, and several of them underwent adaptations. The types of texts suggested in the National Curricular Parameters (Brazil, 1997) for the first cycle of Elementary School, Volume 2 - Portuguese Language were considered. There were four argumentative, six narrative, four descriptive, three injunctive or instructive, and three dialogical texts. Some linguists considered the latter as narrative. Here is an example question:

 

 

2.3 Correction criteria

For the Cloze tests, each correct answer received one point. Words were considered correct if identical to the words the author of the text had used. This form of correction is called literal or verbatim and is recommended because it avoids subjectivism in the evaluation. Hence, the maximum score for the two tests was 30 points, corresponding to the number of omissions.

For the QACM, each correct answer received one point. The maximum score was 20 points, corresponding to the number of questions in the instrument.

 

3. Procedures

3.1 Data collection

The project was submitted to the Ethics Committee, under CAAE Protocol number 0277.0.142.000-10 and approved. Then, authorization was obtained from the Secretary of Education and the heads of the schools were contacted to agree on the details of the data collection. The participants were assured that the research would not influence their grade, would not affect their academic performance in school and would be strictly confidential. The students' parents were free to let their children participate in the research or not. Those who agreed to participate received the Free and Informed Consent Term for signing.

The application of the instruments was done collectively, in a single session, with the children in their classrooms, at times granted by the teachers. First, the students answered the Cloze tests, who were asked to read the full text and then write the words that best completed the gaps in order to give meaning. The students were alerted that the size of the line corresponded to the size of the word that fits into each gap. The application time did not exceed 20 minutes for each text. Then, the students answered the QACM. The explanation was provided that there were 20 texts and that they should read each one and mark a single correct answer, choosing the name of the text type among the five options. The application time did not exceed 20 min. for this instrument.

3.2 Data analysis

Data were analyzed according to nonparametric descriptive and inferential statistical procedures, as the Shapiro-Wilk test (p < 0.001) revealed the absence of normal data distribution. Therefore, the Kruskal Wallis test was used to compare the performance of students from different school years. The internal consistency analysis was also performed, using the alpha coefficient estimation. Finally, linear regression was used to analyze whether the QACM would predict the Cloze scores.

 

4. Results

To achieve the first objective, the mean scores for metatextual awareness (QACM) and the reading comprehension evaluation tests (Total Cloze) were verified. The results of the descriptive analyses are shown in Table 4.1:

 

 

The students' metatextual awareness was evaluated based on the results of the QACM, whose score could range from 0 to 20 points. The averages increased progressively from the 3^rd to the 5^th year and the total number of students (N = 277) was 11.15, with a standard deviation of 4.07. The minimum score was 1 and the maximum 18. The reading comprehension ability was evaluated based on the results of two texts in Cloze. The score of each can vary from zero to 15, with a total of 30 points. The students' average score was 13.46, with a standard deviation of 6.07. The minimum number of correct answers was zero and the maximum 27.

To achieve the purpose of searching for evidence of criterion validity based on the variable school year, the students' classifications in each instrument were compared using the Kruskal-Wallis test. The values obtained are shown in Table 4.2.

 

 

There was a significant difference between the mean ranks of the instruments as seen in Table 4.2. It could be observed that the classifications went in descending order from the most (5^th) to the least advanced year (3^rd). As it was important for the instrument to be reliable and present evidence of validity, the internal consistency of the questionnaire was verified through Cronbach's alpha coefficient. The value obtained was 0.80, superior to the adopted cutoff value of 0.70 (Hair, Black, Babin, Anderson, & Tatham, 2009).

Finally, it was verified if the obtained QACM score could predict the Cloze score, for which the Simple Linear Regression Analysis was performed. This relationship is displayed in Figure 4.1, showing the Scatter Plot.

 

 

The analysis revealed that the performance on the QACM explains 46% of the variance in the performance on the Cloze. This result shows that, for the studied sample, the QACM score was able to predict the Cloze score, that is, the metatextual awareness predicts the performance in reading comprehension, demonstrating that they are shared skills.

 

5. Discussion and final considerations

Student averages increased from the 3^rd to 5^th year, showing that both QACM and Cloze were sensitive measures to capture the development of metatextual awareness and reading comprehension of children with evolution in the school years in the sample studied. This result is in line with Kleiman's (2004) considerations that the understanding of the text becomes easier the more textual knowledge the reader has. As students did not have access to the text support, they were supposed to deliberately reflect on gender, while also taking into account the implicit learning issues present in their development and learning process (Gombert, 2013).

The explanation for these results lies in the fact that the students are increasingly aware of textual structures and types of discourse, determining their expectations of texts, and these expectations play a considerable role in the comprehension (Kleiman, 2004). In this sense, the exposure to several text genres requires different forms in the comprehension process, as the reader deals with different classes of inference (Marcuschi, 2010; Spinillo & Almeida, 2015). In addition, this exposure provides increasingly complex processes such as reader monitoring (Sternberg, 2010) and textual production (Spinillo & Melo, 2018).

The QACM obtained a good internal consistency index (Streiner, 2003) and this result, together with the evidence of criterion validity, obtained based on the school year variable, indicates that it is an instrument that can be used to diagnose the students' metatextual awareness (Muñiz, 2004; Pasquali, 1998; Streiner, 2003). After all, the metalinguistic activities cited in the PCNLP (1998) are basic for knowledge acquisition in other disciplines. Thus, its diagnosis and possible intervention are very important for academic success. Oliveira and Braga (2012) and Zaboroski and Oliveira (2014) found in their research that teachers did not have the necessary knowledge about typology and textual genres though, indicating that there is a need to invest in pedagogical activities that contemplate the subject matter in teacher training as well. Another result that confirms the importance of developing this knowledge in both students and teachers was the fact that the QACM performance explained the performance on the Cloze.

Reading is a highly discussed subject in Brazil, especially due to Brazilian students' last positions in the International Student Assessment Program (Pisa) and in the external examinations promoted by the Brazilian government. In order to prepare readers, favorable conditions are necessary in relation to the material resources available and their use in reading practices. It is desirable that schools have several reading materials, that the teacher plans regular reading activities, exposing students to the widest possible range of texts that circulate socially. As a consequence, according to the National Curriculum Parameters for Portuguese Language (Brasil, 1998), from the 5^th to the 8^th grades, the competent reader should be able to read between the lines, identify, from what is written, implicit elements, establish relations between the text and his background knowledge or between the text and other texts already read. Furthermore, according to the PCNLP (Brasil, 1998), it is during this period that students either give up reading because they are unable to the school's reading demands, or they start using the procedures built in previous cycles to deal with the challenges reading poses, with increasing autonomy.

The study is not free from limitations, one of them referring to the questionnaire itself. Students complained that it was too long and this may be a variable that affects the children's performance. For example, future studies, using the Item Response Theory (TRI), for example, can be performed to assess if there are items that can be eliminated without the instrument suffering losses in psychometric aspects. In addition, a qualitative analysis of these items can be performed to identify if some textual genres are very salient (as explained by Spinillo & Simões, 2003). In such cases, they may be potentially cancellable and the analysis may indicate the possibility of elimination.

These assertions reveal the importance of developing valid instruments that permit diagnosing basic skills for the preparation of the reader so that, if necessary, the interventions are applied as early as possible so as not to jeopardize the academic performance of the student, as disclosed in research based on Cognitive Psychology. In this sense, the importance of this study is highlighted in view of the lack of research that has been found on metalinguistic activities in Brazil. Further research is needed, applying the QACM at other types of institutions and in other States to solve the limitations of this research.

It is important to underline the questionnaire's potential application in the final years of Elementary Education I, as it can appoint elements to the teachers which can be included in daily pedagogical practices, especially in the school years focused on here. In addition, intervention programs for reading comprehension can be elaborated, using different text genres, in Cloze, as the findings of this study indicate its sharing with metatextual awareness.

 

References

Albuquerque, E. B. C., & Spinillo, A. G. (1998). Consciência textual em crianças: Critérios adotados na identificação de partes de textos [Textual awareness in children: Criteria adopted in the identification of text parts]. Revista de Estudios e Investigación en Psicología y Educación, 3,145-158.         [ Links ]

Brasil. (1997). Secretaria de Educação Fundamental. Parâmetros Curriculares Nacionais: Ensino de primeira a quarta séries: Língua portuguesa [National Curriculum Parameters: First to fourth-grade teaching: Portuguese language]. Brasília, DF: MEC/SEF. Retrieved from http://portal.mec.gov.br/seb/arquivos/pdf/portugues.pdf        [ Links ]

Brasil. (1998). Secretaria de Educação Fundamental. Parâmetros Curriculares Nacionais: Terceiro e quarto ciclos do ensino fundamental: Língua portuguesa [National Curriculum Parameters: Third and fourth cycles of elementar education: Portuguese language]. Brasília, DF: MEC/SEF. Retrieved from http://portal.mec.gov.br/seb/arquivos/pdf/portugues.pdf        [ Links ]

Cunha, N. B., & Santos, A. A. A. (2014). Estudo de validade do questionário de avaliação de consciência metatextual. [Validity study of the metatextual awareness assessment questionnaire]. Psicologia: Teoria e Prática, 16(1),141-154. doi: 10.15348/1980-6906/psicologia.v16n1p141-154        [ Links ]

Cunha, N. B., Suehiro, A. C. B., Oliveira, E. Z., Pacanaro, S. V., & Santos, A. A. A. (2009). Produção científica da avaliação da leitura no contexto escolar [Scientific production on reading assessment in the school contexto]. Psico, 40(1),17-23.         [ Links ]

Ferreira, A. L., & Spinillo, A. G. (2003). Desenvolvendo a habilidade de produção de textos em crianças a partir da consciência metatextual. [Developing the text production skill in children based on metatextual awareness]. In M. R. Maluf (Org.), Metalinguagem e aquisição da escrita: Contribuições da pesquisa para a prática da alfabetização (pp. 119-148). São Paulo: Casa do Psicólogo.         [ Links ]

Gombert, J. E. (1992). Metalinguistic development. Hertfordshire: Harverster Wheatsheaf.         [ Links ]

Gombert, J. E. (2013). Epi/meta versus implícito/explícito: Nível de controle cognitivo sobre a leitura e sua aprendizagem [Epi/meta versus implicit/explicit: Level of cognitive control over reading and its learning]. In M. R. Maluf & C. Cardoso Martins (Org.), Alfabetização no século XXI: Como se aprende a ler e escrever (pp. 109-123). Porto Alegre: Editora Penso.         [ Links ]

Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2009). Análise multivariada dos dados. (6 ed). Porto Alegre: Bookman.         [ Links ]

Kleiman, A. (2004). Texto e leitura: Aspectos cognitivos da leitura [Text and reading: Cognitive aspects of reading]. (9 ed). Campinas, SP: Pontes.         [ Links ]

Marcuschi, L. A. (2010). Gêneros textuais: definição e funcionalidade [Text genres: Definition and functionality]. In A. P. Dionísio, A. R. Machado & M. A. Bezerra (Orgs.), Gêneros textuais e ensino. (2nd ed, pp. 19-36). São Paulo: Parábola Editorial.         [ Links ]

Muñiz, J. (2004). La validación de los testes. Metodología de las Ciencias del Comportamiento, 5(2),121-141.         [ Links ]

Oliveira, J. P., & Braga, T. M. S. (2012). PRONARRAR: Programa de intervenção metatextual - Apoio para escolares com atraso no processo de alfabetização [PRONARRAR: Metatextual intervention program - Support for students with a delay in the literacy process]. Curitiba: CRV.         [ Links ]

Oliveira, K. L., Boruchovitch, E., & Santos, A. A. A. (2008). Leitura e desempenho escolar em português e matemática no ensino fundamental [Reading and school performance in Portuguese and mathematics in elementary education]. Paidéia, 18(41),531-540.         [ Links ]

Pasquali, L. (1998). Psicometria: Teoria e aplicação [Psychometrics: Theory and application]. Brasília, DF: Ed. UnB.         [ Links ]

Pinheiro, L. R., & Guimarães, S. R. K. (2013). A coerência e a coesão nos textos de opinião de alunos do 5º ano de ensino fundamental. [Coherence and cohesion in opinion texts of students in the 5^th year of elementary education]. Estudos e Pesquisas em Psicologia, 13(2),500-523.         [ Links ]

Rodrigues, M. R. F., & Vilela, F. C. (2012). Resolução da situação-problema e desfecho em histórias de crianças de 7 e 9 anos [Solution of the problem situation and outcome in the stories of seven and nine-year-old children]. Psicologia: Ciência e Profissão, 32(2),422-437. doi: 10.1590/S1414-98932012000200011        [ Links ]

Santos, A. A. A. (2005). Evidências de validade de critério para o Teste de Cloze. (Unpublished manuscript), Universidade São Francisco, São Paulo.         [ Links ]

Santos, A. A. A., & Cunha, N. B. (2011). Questionário de avaliação da consciência metatextual: Relatório técnico [Metatextual awareness assessment questionnaire: Technical report]. Itatiba: Universidade São Francisco.         [ Links ]

Santos, A. A. A., & Cunha, N. B. (2012). Consciência metatextual: Evidências de validade para instrumento de medida [Metatextual awareness: Validity evidence for a measuring tool]. Psico-USF, 17(2),233-241. doi: 10.1590/S1413-82712012000200007        [ Links ]

Spinillo A. G., & Almeida D. D. (2015). Compreendendo textos narrativo e argumentativo: Há diferenças? [Understanding narrative and argumentative texts: Are there differences?]. Arquivos Brasileiros de Psicologia, 66(3),115-132.         [ Links ]

Spinillo, A. G., & Melo, K.L. R. (2008, June). The production of written stories and metatextual awareness: An intervention study with elementary school children. 11th International Conference of the EARLI Special Interest Group on Writing (p. 59). Sweden: Lund University.         [ Links ]

Spinillo, A. G., & Melo, K. L. R. (2018). O papel do conhecimento acerca da estrutura do texto na escrita de histórias por crianças [The role of the knowledge on the text structure in the writing of stories by children]. Educar em Revista, 34(69),277-292. doi :10.1590/0104-4060.54654        [ Links ]

Spinillo, A. G., & Silva, A. P. (2014). "O que é história, carta e notícia de jornal?" A definição de textos por crianças ["What is a story, letter and newspaper article?" The definition of texts by children]. Psicologia USP, 25(2),176-184. doi: 10.1590/0103-6564A20133613        [ Links ]

Spinillo, A. G., & Simões, P. U. (2003). O desenvolvimento da consciência metatextual em crianças: Questões conceituais, metodológicas e resultados de pesquisas [The development of metatextual awareness in children: Conceptual and methodological aspects and research results]. Psicologia: Reflexão e Crítica, 16(3),537-546.         [ Links ]

Sternberg, R. J. (2010). Psicologia cognitiva [Cognitive psychology]. Porto Alegre: Artmed.         [ Links ]

Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1),99-103.         [ Links ]

Zaboroski, A. P., & Oliveira, J. P. (2014). Proposta de avaliação da linguagem escrita com foco para a habilidade metatextual [Assessment proposal of written language focused on metatextual skill]. In J. P. Oliveira, T. M. S. Braga, F. L. P. Viana & A. S. Santos (Org.), Alfabetização em países de língua portuguesa: pesquisa e intervenção (pp. 217-236). Curitiba: CRV.         [ Links ]

 

 

Mailling address:
Neide de Brito Cunha
Av. Pref. Tuany Toledo, 470
Pouso Alegre, MG, Brazil. CEP 37550-000
E-mail: neidedebritocunha@gmail.com

Submission: 18/06/2017
Acceptance: 13/11/2018