ORIGINAL ARTICLE

 

Performance profile of reading and metalinguistic skills in students with dyslexia, learning difficulties and learning disorders

 

Perfil do desempenho em habilidades metalinguísticas e leitura de escolares com dislexia, dificuldades e transtornos de aprendizagem

 

 

Bianca dos Santos; Simone Aparecida Capellini

Universidade Estadual Paulista "Júlio de Mesquita Filho" Faculdade de Filosofia e Ciências (UNESP) - Marília (SP), Brasil

Correspondence

 

 

---

ABSTRACT

INTRODUCTION: The reading process involves several cognitive skills, such as metalinguistic skills and memory. These processes lead the student to understand the ideas of a text for the creation of mental models, based on the context and your point of view
OBJECTIVE: To analyze the performance of metalinguistic and reading skills in students with dyslexia, learning difficulties, and learning disorders
METHODS: This is an observational and cross-sectional study. The group comprised 80 students from the 3rd to the 5th year of elementary, both gender, aged between eight and ten years and 11 months. The sample, distributed in four groups as follows: Group I composed of 20 students with an interdisciplinary diagnosis of dyslexia, Group II make up of 20 students with a multidisciplinary diagnosis of learning disorders, Group III by 20 students with learning difficulties and Group IV comprised 20 students with excellent academic performance. Every student, individually, underwent the application of the Metalinguistic and Reading Skills Tests Protocol
RESULTS: The Groups I and II had more errors than Group III and IV. Group III showed a higher number of mistakes than Group IV. Scores were obtained from the metalinguistic tests, word reading and pseudowords, and repetition of no -monosyllable and polysyllable words
CONCLUSION: Students with dyslexia and learning disorders had a higher number of mistakes in syllabic and phonemic tests, reading words, and pseudowords than students with learning difficulties and excellent academic performance

Keywords: learning, reading, education, dyslexia, learning disorder.

---

RESUMO

INTRODUÇÃO: O processo de leitura envolve diversas habilidades cognitivas como a decodificação das palavras, a aquisição de vocabulário, a percepção dos sons, as habilidades metalinguísticas e a memória, de forma que esses processos levam o escolar à compreensão das ideias de um texto para a criação de modelos mentais, com base no contexto e no seu ponto de vista
OBJETIVO: Caracterizar e comparar o desempenho em habilidades metalinguísticas e leitura de escolares com dislexia, dificuldades e transtornos de aprendizagem
MÉTODO: Este é um estudo observacional e descritivo de corte transversal. Participaram 80 escolares do 3º ao 5º ano do ensino fundamental, ambos os sexos, na faixa etária de oito a dez anos e 11 meses, distribuídos em quatro grupos, sendo Grupo I composto por 20 escolares com diagnóstico interdisciplinar de dislexia, Grupo II composto por 20 escolares com diagnóstico interdisciplinar de transtornos de aprendizagem, Grupo III composto por 20 escolares com dificuldades de aprendizagem e Grupo IV composto por 20 escolares com bom desempenho acadêmico. Todos os escolares foram submetidos à aplicação do Protocolo de provas de habilidades metalinguísticas e de leitura, individualmente
RESULTADOS: Os resultados foram analisados estatisticamente e revelaram maior número de erros de Grupo I e Grupo II em relação ao Grupo III, e Grupo IV e Grupo III em relação ao Grupo IV nas provas metalinguísticas, de leitura de palavras e de pseudopalavras e de repetição de não-palavras monossílabas e polissílabas
CONCLUSÃO: A partir deste estudo foi possível concluir que escolares com dislexia e transtorno de aprendizagem apresentaram um maior número de erros em provas silábicas e fonêmicas e leitura de palavras e pseudopalavras quando comparados aos escolares com dificuldades de aprendizagem e bom desempenho acadêmico

Palavras-chave: aprendizagem, leitura, educação, dislexia, transtornos de aprendizagem.

---

 

 

Authors summary

Why was this study done?

It is widely known that the diagnostic criteria and the manifestations of students with learning disorders, dyslexia, and learning difficulties; however, few studies compare the performance of these students, mainly concerning metalinguistic skills.

What did the researchers do and find?

The researchers applied a protocol that evaluates metaphonoligical skills at the phonemic and syllabic levels, reading of words and pseudowords, and phonological working memory. From the application of this protocol and the analysis performed, statistically, significant differences were found between students' performances. Students with learning disorders had more difficulties, and therefore, a higher number of errors in the skills were assessed compared with students with dyslexia and those with learning difficulties. The study also allowed to know which tests/skills the students found more comfortable or more challenging to perform.

What do these findings mean?

These findings bring knowledge about the level of difficulty faced by the students to perform some tests. The evidence collected is also a help for both health and education professionals to cooperate in the identification and diagnosis of these clinical conditions.

 

INTRODUCTION

The reading process involves several cognitive skills such as the decoding of words, the acquisition of vocabulary, the perception of sounds, metalinguistic skills, and memory so that these processes lead the student to understand the ideas of a text for the creation of mental models, based on the context and his point of view^1-4.

Among these skills, decoding stands out. This process involves the skills of recognizing the written word, which refers to the ability to transform the orthographic signs of words written into language, that is, to divide the word into its constituent sounds, converting the letter into speech sounds to form a word^5-7.

Phonological working memory is a limited capacity system that allows the storage and temporary manipulation of verbal or visual information needed for complex tasks, such as comprehension, learning, reading, reasoning, and planning⁸.

Metaphonoligical skills, phonological working memory, and so-called lexical processes such as knowledge of orthographic structure are considered basic operations. They are distinguished from high-level processes, as these require a higher capacity for abstraction or mental elaboration^9,10.

Scientific literature^11-13 describes that students with learning disorders and dyslexia have changes in reading fluency, such as changes that can happen due to difficulties in the reading decoding mechanism and by problems in metalinguistic skills.

The so-called learning disorders are due to dysfunctions of the central nervous system and related to a "failure" in the process of acquisition and processing of information. Therefore, they cannot be confused with learning difficulties, since this, unlike learning disorders, are due to issues related to pedagogical, emotional, or sociocultural problems¹⁴.

Dyslexia is a specific reading disorder; its manifestation can reveal changes in different types of processing, often characterized by difficulties at the beginning of literacy, affecting the acquisition of reading, writing, and spelling. Changes in cognitive, phonological, and visual processes can manifest in different ways characterizing the different auditory, visual or mixed subtypes^15-17.

By definition, learning difficulties are obstacles or barriers that hinder the student's learning process^18-20; they can be long-lasting or transient, which may or may not lead the student to drop out, fail, reduced learning time, or need specialized help²¹.

Thus, it is necessary to develop more studies to establish the performance profile between metalinguistic and reading skills in different populations of students aiming to verify the impact of these skills on reading isolated words. The literature has described that students with specific learning disorders such as dyslexia and global disorder have difficulties in accessing and retrieving phonological information necessary for a performance considered adequate in reading tasks. Still, according to the authors' knowledge, so far, there are no studies that compare those parameters.

Therefore, this study aims to analyze the performance profile of metalinguistic and reading skills in students with dyslexia, learning difficulties, and learning disorders.

 

METHODS

An observational, cross-sectional study carried out with a sample comprised 80 students from elementary school, of both genders, in the age range of eight to ten years and 11 months.

Each group contained students of different age range: five were eight years old (2 female and three male), six aged nine years (5 male and one female), and nine aged ten years old (4 male and five female).

The four groups divided as follows:

Group I (G-I): 20 students with an interdisciplinary diagnosis of dyslexia of the phonological subtype;

Group II (G-II): 20 students with an interdisciplinary diagnosis of learning disorders;

Group III (G-III): 20 students with learning difficulties;

Group IV (G-IV): 20 students with excellent academic performance.

Participants in all groups were students enrolled in municipal public schools designated by the Psychopedagogical Support Center of the Municipal Education Secretariat of the city of Marília, São Paulo, Brazil. All these educational centers use the same academic material and apply the same teaching method, minimizing or avoiding bias in the results of this current study.

Eligibility: Inclusion and exclusion criteria

Participants of G-I and G-II were diagnosed with dyslexia and learning disorders by an interdisciplinary team at the Learning Deviations Research Laboratory, including speech, neurological, pedagogical, and neuropsychological assessment. These students were on the waiting list for speech therapy at the Center for Education and Health Studies. They had not undergone any type of speech therapy, psycho-pedagogical, or pedagogical intervention.

The students in G-III was indicated by their teachers for having learning difficulties, and those with excellent academic performance were placed in G-IV. Good academic performance was defined as achieving a score equal or higher 5.0 in the assessments of Mathematics and Portuguese Language during four consecutive months (evaluated every two months). It was considered learning difficulties when the students had unsatisfactory performance in two successive bi months in assessments of Portuguese Language and Mathematics, with a score below 5.0.

From this indication, the students underwent the School Performance Test²². Students who obtained high or medium/high performance in the sections reading, writing, and arithmetic were placed in G-IV, and those underperformed were placed in G-III.

All groups underwent the evaluation of guideline PROHMELE (Metalinguistic and Reading Skills Evidence protocol)²³, which consists of the following tests:

- Syllable and phonemic identification tests, initial phoneme identification, final syllable identification, final phoneme identification, medial syllable identification, phoneme identification medial;

- Syllabic and phonemic manipulation tests: syllabic segmentation, phonemic segmentation, syllabic addition, phonemic addition, syllabic substitution, phonemic substitution, subtraction syllabic, phonemic subtraction, a combination of syllables and combination of phonemes;

- Reading Tests: Reading of real words, composed of 133 actual words. Pseudoword reading: consisting of 27 pseudowords.

- Non-word repetition test: Monosyllabic non-word repetition; Repetition of non-syllable words; Repetition of trisyllabic non-words; Repetition of polysyllabic non-words with four syllables; Repetition of polysyllabic non-words with five syllables; Repetition of polysyllabic non-words with six syllables.

In the current study, pseudowords are understood as a logo; that is, a syllable or a sequence of syllables that belong to the language, but that does not form a meaningful word. The pseudoword is derived from a real word, such as "bafata," derived from "cockroach" ("barata," in Portuguese), changing only one element and maintaining the syllabic pattern.

The non-word repetition test assesses phonological working memory because it requires the student to evoke various phonological processes, such as perception, coding, storage, retrieval, and production, regardless of their lexical knowledge⁵.

The application of metalinguistic skills tests was performed in such a way that the students did not obtain a visual clue to the articulation of the sounds produced by the examiner. Their responses were copied on the PROHMELE answer sheet. The students were previously instructed and trained with examples similar to the test, so they knew what they should do.

The reading tests were performed out loud and filmed for later analysis. Each student received instructions on how to read the word lists, presented in Arial font, 14 point size, double space, divided into columns according to word length (monosyllabic, disyllabic, trisyllabic and polysyllabic - 4 to 7 syllables) and pseudowords (monosyllabic, disyllabic, trisyllabic). In the pseudoword reading test, the students were told that they would read words that did not exist and that therefore were not part of their vocabulary. The tests were applied individually in a session of about 50 minutes.

All collected evidence was analyzed according to the error criterion. The characterization of types of errors in the reading of real words and pseudowords was carried out using standards established for Brazilian Portuguese, described as follows:

D1 - Graphophonemic correspondence rule independent of context, related to regular words with univocal relation. In words like a duck, cake, knife, candle, armadillo, die, dough, girl, I go down, rain, window, nail, car, glasses, put water, lamp, frog, etc.

D2 - Context-dependent graphophonemic rule, applied to irregular words. For example house, zebra, coast, pool, tree, mouse, face, goose, bag, man, branch, people, cinema, guard, rattle, fan, picture, exception, cup, exam, chest, box, swarm, etc.

D4 - Values of the letter "X" depending exclusively on the mental and orthographic lexicon, which appear in words such as pineapple (abacaxi), taxi, oxygen (oxigênio), help (auxílio), etc...

In the pseudoword reading test, only rule D1 was considered, as its objective was to verify the univocal correspondence between letter and sound.

After approval by the Research Ethics Committee of the home institution, this study was carried out with the Protocol number 836/2013. The parents or guardians of the participants signed the Free and Informed Consent Form.

Statistical analysis

The SPSS (Statistical Package for Social Sciences) program, version 22.0, was used for statistical analysis of the data by the next tools: Wilcoxon Signed Posts Test, Kruskal-Wallis Test and Mann-Whitney Test adjusted by Bonferroni Correction.

 

RESULTS

It verified a higher number of errors in students of G-II students than G-IV in the initial phoneme identification test, and a higher number of mistakes in G-II students to those from G-III and G-IV in the final syllable identification test.

The results from G-I and G-II showed a higher number of errors compared to those from G-III and G-IV in the tests of final phoneme identification, syllable identification, and medial phoneme, subtraction, addition and syllable combination, the addition of phoneme and substitution of a phoneme. The test results of subtraction and phonemic segmentation had a higher number of failures in G-I and G-II than G-III and G-IV (tables 1, 2 and 3).

Regarding the test of reading real words and pseudowords, table 4 describes a statistically significant difference related to the performance of students from G-I, G-II, G-III, and G-IV.

Table 5 indicates a statistically significant difference when comparing students from GI, GII, GIII, and GIV in the repetition of monosyllabic and polysyllabic non-words with four and six syllables.

 

DISCUSSION

According to our findings, the students diagnosed with Learning Disorder had a higher number of errors in all tests applied. The students in the control group (G-IV), who have an excellent academic performance, presented difficulties in only some tests, such as phonemic tests, repetition of non-words and tests of identification, subtraction, and segmentation of syllables. However, those difficulties were not statistically significant, while students diagnosed with dyslexia showed more errors in phonemic tests and reading words and pseudowords.

It was verified that in the majority of phonemic tests of metalinguistic skills, students from all groups had a higher number of mistakes than syllabic tests.

These results show that the most considerable difficulty for students with dyslexia was to identify, combine, add, subtract, segment, and replace phonemes. This characteristic is due to, students first acquire syllabic perception, and only after doing a reading training, they receive phonemic perception^23,24.

A study²⁵ revealed that, in general, students have difficulties in understanding that the elements of speech correspond to unique units of sounds, but at an abstract level, that is, that they exist in mind. This finding aligns with the results of this study, since students with learning difficulties, learning disabilities, and dyslexia had more mistakes in the tests that contained phonemic manipulation.

The highest level of errors was found in tests performed by students diagnosed with dyslexia and learning disorders, respectively, when we compared the syllabic and phonemic tests. This situation may have occurred, as these students have difficulty in accessing and recovering phonological information²⁶.

Even if tasks of metaphonological skills, such as the identification of syllables and phonemes are considered pure, they require only one operation followed by an answer. Other tasks are more complex, such as syllabic and phonemic manipulation that requires performing two processes: saving a unit in memory. In contrast, a new operation is performed; some of our participants had difficulty in some of these tests, especially students with learning disorders and learning difficulties.

Therefore, the answers may vary by type of operation asked to the students and according to the kind of linguistic-cognitive impairment that they have¹⁰. Thus, students with learning disorders showed a higher number of mistakes in several tests involving the identification of syllables and phonemes than students with dyslexia, indicating that children with learning disorders have more difficulties in tests of metalinguistic skills in activities that require only one operational process.

Findings make evident the importance need to develop metalinguistic knowledge in these students, based on the ability to reflect on phonemes and syllables during reading, as well as in the form of analysis of the constituent parts of the word. In this manner, the student can identify them not only orally, but also in written content for reading²⁷.

In the word reading test, both real words and pseudowords, it was noticed a higher number of errors in students with learning disorders followed by students with dyslexia. According to the literature¹¹, students with some learning difficulties present changes in reading, due to the complexity of decoding that may present.

A reading considered fluent will depend on the individual's ability to decode words quickly and recognize isolated words automatically and efficiently, this comprehension ability being impaired when the student has some difficulty to identify the words²⁸. As the students in this study with learning disorders and dyslexia have a deficit in phonological identification and decoding skills, according to the manifestations that these students have, this nature's errors will be present not only in reading real words but also in reading actual terms and in the reading of pseudowords.

Our study also revealed that students with learning difficulties showed changes in metalinguistic and reading skills. This fact may be explained because when the alphabetic principle of the Portuguese language is not taught systematically in the classroom, students in the development phase of reading and writing may have flaws in their abilities to detect, discriminate, compare and memorize sounds and syllables. This condition can cause difficulty in activating the generating mechanism for the formation of words for reading and writing²⁸.

Regarding the repetition test of words and non-words, responsible for assessing the phonological working memory of these students, an index of a high number of failures was found in all groups, with a higher rate of errors in the group of students with learning disorders. This situation was predominant mainly in the repetition test of polysyllabic non-words. The execution of this test requires the use of operational memory, with information management capacity. Students with some type of learning alteration present a decrease in the phonological information storage capacity for quick recovery. It is possible that the higher the extension of the stimulus for retention, the less storage memory¹².

As previously highlighted in the literature²⁹, the results of this study also reveal that when there is the phoneme perception, it exists to store the phonological information and consequently perform the grapheme-phoneme association. Then the child realizes that the phonemes can be combined to form another word, causing the generative memory mechanism to be activated and the conversion of phonology to spelling. This mechanism will allow the student to read any new word, despite making mistakes in irregular words.

Thus, students with learning disorders or even learning difficulties may present a lack of phoneme perception, resulting in a decrease in the capacity of phonological information, in the establishment of a grapheme-phoneme relationship for reading and the generative capacity of the phonological working memory for the storage of information read or spoken.

 

CONCLUSION

Our findings reflect the need to evaluate metalinguistic skills in the academic performing test of school children. There are significant differences in the performance of metalinguistic skills in students with dyslexia, learning disorders, and learning difficulties. Further studies are needed to analyze the difference in performance among these students based on scientific evidence.

 

REFERENCES

1.Kintsch W. Comprehension. Cambridge: Cambridge University Press, 1998; p. 241.         [ Links ]

2.Kintsch W, Van Dijk T. Toward a model of text comprehension and production. Psychol Rev. 1978;85(5):363-94. DOI: http://dx.doi.org/10.1037/0033-295X.85.5.363        [ Links ]

3.Johnson T, Archibald T, Tenenbaum G. Individual and team annotation effects on students' reading comprehension, critical thinking, and meta-cognitive skills. Comp Hum Behavior. 2010;26(6):1496-1507. DOI: https://doi.org/10.1016/j.chb.2010.05.014        [ Links ]

4.Cunha VLO, Silva C, Capellini SA. Correlation between basic reading skills and reading comprehension. Estud Psicol. 2012;29(Suppl 1):799-807. http://dx.doi.org/10.1590/S0103-166X2012000500016        [ Links ]

5.Cunha VLO, Oliveira AM, Capellini SA. Compreensão de leitura: princípios avaliativos e interventivos no contexto educacional. Rev Teias. 2010;11(23):221-40.         [ Links ]

6.Saine NL, Lerkkanen MK, Ahonen T, Tolvanen A, Lyytinen H. Predicting word-level reading fluency outcomes in three contrastive groups: Remedial and computer-assisted remedial reading intervention, and mainstream instruction. Learn Individ Differ. 2010;20(5):402-14. DOI: https://doi.org/10.1016/j.lindif.2010.06.004        [ Links ]

7.Leybaert J, Alégria J, Deltour JJ, Skinkel R. Aprender a ler: o papel da linguagem, da consciência fonológica e da escola. In: Grégoire J, Piérart B (Orgs.). Avaliação dos problemas de leitura: os novos modelos teóricos e suas implicações diagnósticas. Porto Alegre: Artes Médicas, 1997; p.143-66.         [ Links ]

8.Zanella LW, Valentini NC. Como funciona a Memória de Trabalho? Influências na aprendizagem de crianças com dificuldades de aprendizagem e crianças com desordem coordenativa desenvolvimental. Medicina. 2016;49(2):160-74. DOI: https://doi.org/10.11606/issn.2176-7262.v49i2p160-174        [ Links ]

9.Sánchez EM. La comprensión lectora. In: Millán JA. La lectura en España. Leer para aprender. España: Fundación Germán Sánchez Ruipérez y Federación de Gremios de Editores de España, 2008; p.191-205.         [ Links ]

10.Cunha VLO, Capellini SA. Habilidades metalinguísticas no processo de alfabetização de escolares com transtornos de aprendizagem. Rev Psicopedag. 2011;28(85):85-96.         [ Links ]

11.Lyon R, Shaywitz S, Shaywitz, B. Part 1Defining dyslexia, comorbidity, teachers' knowledge of language and reading. Ann Dyslexia. 2003;53(1):1-14.         [ Links ]

12.Germano GD, Capellini SA. Performance of students with dyslexia, learning disabilities and learning difficulties in metaphonological abilities tests (PROHFON). Rev Soc Bras Fonoaudiol. 2011;23;(2):135-41. DOI: http://dx.doi.org/10.1590/S2179-64912011000200010        [ Links ]

13.Oliveira DG, Silva PB, Dias NM, Seabra AG, Macedo EC. Reading component skills in dyslexia: word recognition, comprehension and processing speed. Front Psychol. 2014;5:1339. DOI: https://doi.org/10.3389/fpsyg.2014.01339        [ Links ]

14.Ukrainetz T, Nuspl J, Wilkerson K, Beddes S. The effects of syllable instruction on phonemic awareness in preschoolers. Early Child Res Q. 2011;26(1):50-60. DOI: https://doi.org/10.1016/j.ecresq.2010.04.006        [ Links ]

15.Bosse ML, Tainturier MJ, Valdois S. Developmental dyslexia: the visual attention span deficit hypothesis. Cognition. 2007;104(2):198-230. https://doi.org/10.1016/j.cognition.2006.05.009        [ Links ]

16.Pereira DM, Araújo RCT, Braccialli LMP. Análise da relação entre a habilidade de integração visuo-motora e o desempenho escolar. Rev Bras Crescimento Desenvolv Hum. 2011;21(3):808-17. DOI: https://doi.org/10.7322/jhgd.20033        [ Links ]

17.Germano GD, Reilhac C, Capellini SA, Valdois S. The phonological and visual basis of developmental dyslexia in Brazilian Portuguese reading children. Front Psychol. 2014;5:1169. DOI: https://doi.org/10.3389/fpsyg.2014.01169        [ Links ]

18.Scarborough H. Very Early Language Deficits in Dyslexic Children. Child Dev. 1990;61(6):1728-43. DOI: https://doi.org/10.2307/1130834        [ Links ]

19.Rebelo JAS. Dificuldades da leitura e da escrita em alunos do ensino básico. Portugal: Edições Asa, 1993.         [ Links ]

20.Martin E, Marchesi A. Desenvolvimento metacognitivo e problemas de aprendizagem In: Coll C, Palácios J, Marchesi A. Desenvolvimento psicoló-gico e educação: necessidades educativas especiais e aprendizagem escolar. Porto Alegre: Artes Médicas, 1996; p.4-35.         [ Links ]

21.Suehiro ACB, Magalhães MMS. Relação entre medidas de Avaliação da Linguagem Escrita em estudantes do Ensino Fundamental. Psico USF. 2014;19(3):489-98. DOI: https://doi.org/10.1590/1413-82712014019003011        [ Links ]

22.Stein LM. TDE - Teste de Desempenho Escolar: Manual para aplicação e interpretação. São Paulo: Casa do Psicólogo, 1994.         [ Links ]

23.Cunha VLO, Capellini SA. PROHMELE - Provas de habilidades metalingüísticas e de leitura. Rio de Janeiro: Revinter, 2009.         [ Links ]

24.Moura O, Moreno J, Pereira M, Simões M. Developmental dyslexia and phonological processing in european portuguese orthography. Dyslexia. 2014;21(1):60-79. DOI: https://doi.org/10.1002/dys.1489        [ Links ]

25.Cárnio MS, Stivanin L, Vieira MP, Amaro L, Martins VO, Carvalho E, et al. Phonological awareness abilities and literacy en elementary school children. Rev Soc Bras Fonoaudiol. 2006;11(4):231-42.         [ Links ]

26.Rotta NT, Ohlweiler L, Riesgo RS. Transtornos da aprendizagem: abordagem neurobiológica e multidisciplinar. Porto Alegre: Artmed, 2015; p. 137.         [ Links ]

27.Fletcher JM, Lyons GR, Fuchs LS, Barnes MA. Transtornos de aprendizagem: da identificação à prevenção. Porto Alegre: Artmed, 2009.         [ Links ]

28.Oliveira AM, Cardoso MH, Pinheiro FH, Germano GD, Capellini SA. Desempenho de escolares com dislexia e transtorno do déficit de atenção e hiperatividade nos processos de leitura. J Hum Growth Dev. 2011;21(2):344-55. DOI: https://doi.org/10.7322/jhgd.20022        [ Links ]

29.Chappell JC, Stephens TL, Kinnison L, Pettigrew J. Educational Diagnosticians' Understanding of Phonological Awareness, Phonemic Awareness, and Reading Fluency. Assessment Effective Interv. 2009;35(1):24-33. DOI: https://doi.org/10.1177/1534508409331653        [ Links ]

 

 

Correspondence:
Bianca dos Santos
biasantosfono@hotmail.com

Received: September 2019
Revised: May 2020
Accepted: September 2020