SciELO - Scientific Electronic Library Online

 
vol.26 número2Maternidade na adolescência: ressignificando a vida?Relação entre o Índice de Desenvolvimento Humano Municipal e taxas de mortes violentas nas Unidades Federativas Brasileiras índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

artigo

Indicadores

Compartilhar


Journal of Human Growth and Development

versão impressa ISSN 0104-1282versão On-line ISSN 2175-3598

J. Hum. Growth Dev. vol.26 no.2 São Paulo  2016

https://doi.org/10.7322/jhgd.119273 

ORIGINAL RESEARCH

 

Maze computer performance in Down syndrome

 

 

Weliton Folli PossebomI; Thais MassettiI, II; Talita Dias da SilvaIII; Silvia Regina Pinheiro MalheirosI; Lilian Del Ciello de MenezesII; Fatima Aparecida CaromanoII; Alessandro Hervaldo Nicolai RéIV; Italla Maria Pinheiro BezerraIV; Carlos Bandeira de Mello MonteiroI, II

IDepartment of Scientific Writing, Faculty of Medicine ABC, Santo André, Brazil
IIPost-graduate Program in Rehabilitation Sciences - Faculty of Medicine - University of São Paulo, São Paulo, SP, Brazil
IIIPost-graduate Program in Cardiology - Federal University of São Paulo - Paulista School of Medicine - São Paulo, SP, Brazil
IVEACH- School of Arts, Sciences and Humanities - University of São Paulo, São Paulo, SP, Brazil

 

 


ABSTRACT

INTRODUCTION: These changes are the main causes of deficits in perceptual-motor skills responsible for motor skill acquisition and performance of functional activities. : The current study aimed at verifying the quantitative performance of people with DS in undertaking a computer task to compare the performances of typical development (TD).
METHODS: 60 subjects participated in the study, 30 with Down's syndrome and 30 with typical development, matched by sex. Individuals were aged from 10-36. The groups were divided into three subgroups by age: Group 1 (G1) 10-18; Group 2 (G2) 18-25; Group 3 (G3) 25-36. They performed a computer maze task. During the acquisition phase all groups attempted the maze 30 times, and then after 5 minutes they performed 5 repetitions of Maze 1 for the retention phase. Finally, for the transfer phase, they performed five repetitions in Maze 2. The dependent variables were submitted to a group, age group, gender and block ANOVA with repeated measures on the last factor.
RESULTS: In acquisition phase there was a significant decrease in movement time (MT) between the first and last acquisition block, but only for the DS-group. In retention, there was a significant effect of Group, and an interaction between Block and Group, indicating that MTs increased from retention to transfer, but only for the DS-group. : It was found that participants with DS improved performance during acquisition and retention, but they had difficulty in performing the transfer of the computational task for a similar situation. The age and gender were not significant in any of the stages of the study.

Key words: computer task, motor learning, Down's syndrome.


 

 

INTRODUCTION

Down's syndrome (DS) is caused by trisomy of human chromosome 21 (HSA21) and results in a large number of phenotypes, including learning difficulties, cardiac defects and distinguishing facial features1,2. It is associated with approximately 1/800 live births and is one of the leading causes of intellectual disabilities3. Down's syndrome is accompanied by multisystem pathologies, which involve delays in basic motor skills, motor impairments, and abnormalities in gait and posture4.

As well as several major motor disorders when compared to healthy persons5, previous studies suggest that children with DS, demonstrate motor problems and cognitive deficits6. The delay in the neuropsychomotor development is an indication that there are alterations in the acquisition of skills and other development areas7.

Children with DS are especially vulnerable to attention impairments because of a genetic neurodevelopmental disorder; attentional constraints will likely exacerbate an already compromised social function. These in turn may reduce learning capacity and increase risk of academic failure8, poor social relationships and long-term behavioral and emotional problems9.

According to Torriani-Pasin et al.10, one of the main problems for DS individuals is the deficit of the perceptual-motor abilities, which are responsible for the abilities that support the acquisition of several motor skills. As well as short-term memory performances, long-term memory and visuoperceptual abilities are impaired because of a dysfunction of different parts of the brain11.

One possible task that may require perceptual motor abilities and is important for people with DS to adapt to the modern environment is to use the computer as assistive technology. Computers and assistive technology devices can maximize independence, productivity, and participation of people with disabilities in academic programs, employment, recreation, and other activities. Meyer e Kieras12 cited that on the basis of executive-process interactive control (EPIC), computational models may be formulated to simulate multiple-task performance under a variety of circumstances.

Therefore, the computer can be a very useful tool in supporting the global development of children with motor disorders when used as an instrument for stimulation and communication. In addition, it can potentially serve as a therapeutic tool because they can combine functionality, pleasure, discovery, and autonomy13.

Despite recognition of the importance of computer use for the functioning of people with disabilities, few studies have examined the computer task performance of subjects with DS. Heath et al.14 cited that the perceptual motor abilities of persons with DS are not optimized to respond to externally paced stimuli, this because the performance of the DS group was slower and more variable than the control. However, while all individuals with DS exhibit some form of learning and memory impairment, this varies in severity2.

Hedges et al.15 cited that insights into the behavioral and neurobiological connections relate to the formation of specific abilities, such as spatial ability, and that learning in formal environments would be beneficial not only for understanding play, attention, and learning individually, but also for the development of more efficacious systems for learning and for the treatment of neurodevelopmental disorders.

In order to analyze the performance of people with DS on a computer, this study used a maze task that used the keyboard arrows (up, down, right, and left) to complete a maze in the shortest time (see pilot study done by Possebom et al.16). According to Silva et al. and Courbois et al.17,18, tasks using technological devices have shown positive results in the participation of the individual, encourage activities in daily life, and can be achieved with the higher performance of sensory, motor and cognitive functions in a way that stimulates and motivates individuals in their own recovery.

The current study aimed at verifying the quantitative performance of people with DS in undertaking a computer task. Specifically, the goal of the research is to compare the performances of typical development (control group) matched for gender and age with subjects with DS on a simple computational task.

 

METHODS

This study was approved by the Ethics Committee for review of research projects of the Faculdade de Medicina do ABC - FMABC, CAEE: 39122214.6.0000.0082.

Participants

A total of 60 subjects participated, 30 with Down's syndrome and 30 typical developing (TD), sex- and age-matched. The minimum age of the individuals was 10, maximum 36. The groups were divided into three subgroups by age: Group 1(G1)10-18 years; Group 2 (G2) 18-25 years; and Group 3 (G3) 25-36 years old.

In order to characterize and homogenize the sample for this study the International Classification of Functioning, Disability and Health (ICF) was used. According to the World Health Organization (WHO), this classification system is useful for organizing homogenous groups in scientific research.

All participants in the study had the following characteristics analyzed independently by three therapists:

(a) Intellectual functions: General mental functions, required to understand and constructively integrate the various mental functions, including all cognitive functions and their development throughout life, with moderate difficulty (b117.2)

(b) Maintenance of attention: Mental functions that allow concentration for the period of time required with moderate difficulty (b1400.2).

(c) Focus attention: Focus intentionally attention on specific stimuli, turning off noises that distract with moderate difficulty (d160.2).

(d) Performing a complex task: Preparing, initiating and arranging the time and space required for a complex task with two or more components, which can be performed sequentially or simultaneously, with slight difficulty (d2101.1).

(e) Communicate and receive spoken messages: Understand the literal and implied meanings of messages in spoken language with moderate difficulty (d310.2). (f) Communicate and receive messages using body language with slight difficulty (d3150.1).

(g) Acquire complex skills: Learning integrated sets of actions, according to rules, and carry out and coordinate the movements themselves so sequenced with moderate difficulty (d1551.2).

Exclusion criteria included other diseases associated with changes in cognitive functions that prevent collaboration and understanding simple commands in the proposed activities.

Instruments

Maze task

To verify computer task performance, we used a maze task proposed by Souza, França and Campos 19 and used by Malheiros et al.20 with disabled people. It consisted of traversing a path on the computer screen in the shortest possible time. The maze had one entrance, one exit, and only one way to achieve the goal. Souza, França and Campos19 stated that the maze task could be used in the diagnostic evaluation of individuals with changes in control and motor learning to identify aspects that are compromised during motor task execution.

Procedure

Participants or their legal guardians signed a consent form. Participants completed the maze task individually in a room equipped with a computer, table, chair, and with an evaluator responsible for providing instruction.

The chair and footrest were adjusted according to the height and needs of the individual so that they would be properly positioned to view the beginning and end of the task. Before starting the task, the researcher demonstrated and described how to perform the task once; then all participants completed a single trial test to verify that they understood the instructions. They were asked to perform the task in the shortest time possible, taking the pawn to the "X" using the keyboard arrows (up, down, right, and left).

The protocol was similar to that used by Malheiros et al.20. During the acquisition phase, all groups attempted the maze 30 times, and then after 5 minutes they performed 5 more repetitions of the same maze for the retention phase. Finally, for the transfer phase, they performed five repetitions in a maze with different way. The study design task is shown in Figure 2.

 

 

 

 

The variables were categorized by: Group (DS, TD), Age (G1: 10-18 years old, G2: 18-25 years old, G3: 25-36 years old) and Gender (male, female).

Data analysis

The dependent variables were submitted to a 2 (group: DS, TD) by 3 (age group: G1, G2, G3) by 2 (gender: male, female) by 2 (block) ANOVA with repeated measures on the last factor. For the factor block, separate comparisons were made for acquisition (first acquisition block A1 versus final acquisition block A6), retention (A6 versus retention block R) and transfer (R versus transfer block T).

In order to examine how many attempts were necessary for stabilization of the movement time, further analyses were made between the blocks A1-A2, A2-A3, A3-A4, A4-A5 and A5-A6 (results in Figure 3). Post hoc comparisons were carried out using Tukey-HSD test (p < .05).

 

 

RESULTS

Acquisition

Significant effects were found for Group F(1, 48) = 24.3, p < .001, ŋ2 = .34, Block F(1, 48) = 30.9, p < .001, ŋ2 = .39, and Block by Group, F(1, 48) = 19.6, p < .001, ŋ2 = .29. Post hoc comparisons indicated a significant decrease in movement time (MT) between the first and last acquisition block, but only for the DS-group (M= 33.1s and 16.1s, respectively) and not for the TD-group (M= 7.7s and 5.7s, respectively) (Fig. 3). In addition, the TD-group showed shorter MTs (M= 6.7s) than the DS-group (M= 24.6s) across practice.

Retention

The comparison of the final practice and retention blocks did not reveal any significant main or interaction effects for Block. In other words, for both groups the pattern of MT in the final practice block and retention were similar, indicating that the learning of the DS-group had consolidated. The main effect for Group, however, was remained present F(1, 48)= 30.0, p <0.001, ŋ2=.39, with a much larger MT in the DS-group (M= 16.1) than in the TD-group (M= 5.8s).

Transfer

There was a significant effect of Group, F(1, 48) = 38.9, p < .001, ŋ2 = .45, and an interaction between Block and Group F(1, 48) = 6.3, p = .015, ŋ2 = .12, indicating that MTs increased from retention to transfer, but only for the DS-group (M = 16.1s and 17.4s, respectively) and not for the TD-group (M = 5.8s and 5.4s, respectively). In addition, the MT in the DS-group was still much larger (M = 16.7s) than in the TD-group (M = 5.6s).

For all phases of the study (acquisition, retention and transfer), there was no difference between groups of age or gender.

 

DISCUSSION

People with Down's Syndrome can benefit from the use of various supporting technologies that facilitate communication, interaction and even mobility. Understanding performance-computing tasks with individuals with disabilities can help develop important strategies to maintain or improve their functionality. According to Almeida et al.21, activities using the computer have been used to improve social communications in DS In the case of individuals with Down's syndrome it is reasonable to expect that the performance may result in a reduced ability to engage in same skills 22.

However, in the study by Wuang et al.23 after a therapy with games the authors realized that individuals had a greater pre-post change on motor proficiency, visual-integrative abilities, and sensory integrative functioning. Considering the maze task, the results show that only participants in the DS group had a significant performance verified by differences between the beginning and end of the acquisition phase, with reduced time to perform the task. Subjects with DS were able to maintain their performance, and although there was a significant difference between retention and transfer, the value was very distant from the first acquisition phase.

Menezes et al.24 conducted a systematic review about motor learning and concluded that motor learning in individuals with Down's syndrome can occur following different tasks, allowing better motor control and specific skills often independently of the knowledge of results (KR), and is more effective in the presence of visual feedback and self-control.

Nevertheless, the developmental progress in children with Down's syndrome is undermined by their failure to practice and maintain new skills and by their counter-productive approach to learning new skills and these factors must be taken into account when assessing cognitive functioning in individuals with Down's syndrome 25. In the same way, learning and memory problems that begin to emerge in late infancy become considerably more noticeable in adolescence and seem to be related to an inability to consolidate the information that is acquired26,27.

Opposite results of the Downs syndrome group were found in the TD group: participants did not improve their performance during the training period, and it can be seen that the first block of acquisition was already almost this group's fastest performance, suggesting that the task was easier for the TD group; the performance of the DS group was always worse than that of the control group since the runtime always remained higher. Thus, we can say that people with DS have slower movements in achieving the same task compared to the TD group.

According to the study by Menezes et al.28, the comparison between the groups (DS and TD) performing a maze task on a cellphone, showed that the Down's syndrome individuals had greater difficulty in carrying out the task, as evidenced by the longer time required for execution. Menezes et al.28 in turn assert that, regardless of the difficulty encountered in the task, participants were able to maintain performance, allowing an adaptation from a stabilization phase, featuring the motor learning.

A study by Davis et al.22, conducted an experiment in which they evaluated route-learning of youth with DS in computational tasks. Two experiments were performed that evaluated the route-learning of youth with DS, youth with intellectual disability (ID) and not DS, and typically development (TD) children, matched on mental age (MA). The participants learned routes with eight choice points, presented via computer. Several objects were placed along the route that could be used as landmarks. Participants navigated the route once with turn indicators pointing the way, and then retraced the route without them. The results pointed to a problem in navigation for persons with DS that exceeds expectations based solely on intellectual level22.

Elliot et al.29 realized that the movement performed by the person with Down's syndrome took twice as long to complete, was typically performed with slower movements in order to achieve the same degree of accuracy, compared with young adults from the general population. Considering the difference between the DS and Control Group Elliot et al.29 speculate that the slower movement performed by the person with Down's syndrome is partly because of a lower peak in the magnitude of acceleration, but the primary reason for the extra time is associated with the multiple discontinuities in acceleration after peak velocity was achieved. Because these corrections are necessary, the assumption is that the performers with Down's syndrome have problems with movement planning and feed-forward control.

Corroborating the study, Palisano et al.30, indicated that individuals with DS need more time to learn certain movements as they increase in complexity when compared to individuals with normal development. This is because individuals with DS exhibit some form of learning and memory impairment, which varies in severity27. However, despite similarities in overall speed and accuracy outcomes, kinematic data indicate that reaching and aiming are compromised in these groups for some very specific reasons29.

The main problem for DS individuals can be in the deficit of perceptual-motor abilities, which are responsible for the abilities that support the acquisition of several motor skills10. This is because, in this population manual tasks feature characteristics such as slowness, selection of unusual strategies, and delay in the acquisition of key patterns of movement31.

For this measurement our study characterized the performance of people with DS in undertaking a basic task maze on the computer, based on a motor learning protocol. The results show that persons with DS were able to learn a new task using a computer, but the performances were worse in all phases of the experiment when compared with the control group.

 

CONCLUSION

It was found that participants with DS improved performance during acquisition and retention, but they had difficulty in performing the transfer of the computational task to a similar situation. However, a difference in MT was observed in all attempts among individuals from both groups. The age and gender were not significant in any of the stages of the study.

 

AUTHOR' CONTRIBUTIONS

All authors participated in the acquisition of data and revision of the manuscript. All authors determined the design, interpreted the data and drafted the manuscript. All authors read and gave final approval for the version submitted for publication.

 

DECLARATION OF INTEREST

The authors report no conict of interest. All authors were responsible for the content and writing of this paper.

 

REFERENCES

1. Bartesaghi R, Haydar TF, Delabar JM, Dierssen M, Martinez-Cue C, Bianchi DW. New Perspectives for the rescue of cognitive disability in Down Syndrome. J Neurosci. 2015;35(41):13843-52. DOI: http://dx.doi.org/10.1523/JNEUROSCI.2775-15.2015        [ Links ]

2. Lana-Elola E, Watson-Scales SD, Fisher EM, Tybulewicz VL. Down syndrome: searching for the genetic culprits. Dis Model Mech. 2011;4(5):586-95. DOI: http://dx.doi.org/10.1242/dmm.008078        [ Links ]

3. Roizen NJ. Medical care and monitoring for the adolescent with Down syndrome. Adolesc Med. 200213(2): 345-58.         [ Links ]

4. Horvat M, Croce R, Tomporowski P, Barna MC. The infl uence of dual-task conditions on movement in young adults with and without Down syndrome. Res Dev Disabil. 2013;34(10):3517-25. DOI: http://dx.doi.org/10.1016/j.ridd.2013.06.038        [ Links ]

5. Biec E, Zima J, Wojtowicz D, Wojciechowska-Maszkowska B, Krecisz K, Kuczynski M. Postural stability in young adults with Down syndrome in challenging conditions. PLoS One. 2014;9(4):e94247. DOI: http://dx.doi.org/10.1371/journal.pone.0094247        [ Links ]

6. Schott N, Holfelder B. Relationship between motor skill competency and executive function in children with Down's syndrome. J Intellect Disabil Res. 2015;59(9):860-72. DOI: http://dx.doi.org/10.1111/jir.12189        [ Links ]

7. Malak R, Kotwicka M, Krawczyk-Wasielewska A, Mojs E, Samborski W. Motor skills, cognitive development and balance functions of children with Down syndrome. Ann Agric Environ Med. 2013;20(4):803-6.         [ Links ]

8. Fernani DCGL, Prado MTA, Fell RF, Reis NL, Bofi TC, Ribeiro EB, et al. Motor intervention on children with school learning dificulties. Rev Bras Crescimento Desenvolvimento Hum. 2013;23(2):209-14. DOI: http://dx.doi.org/10.7322/jhgd.61301        [ Links ]

9. Cornish K, Steele A, Monteiro CR, Karmiloff-Smith A, Scerif G. Attention deficits predict phenotypic outcomes in syndrome-specific and domain-specifi c ways. Front Psychol. 2012;3:227. DOI: http://dx.doi.org/10.3389/fpsyg.2012.00227        [ Links ]

10. Torriani-Pasin C, Bonuzzi GMG, Soares MAA, Antunes GL, Palma GCS, Monteiro CBM, et al. Performance of Down syndrome subjects during a coincident timing task. Int Arch Med. 2013;6(1):15. DOI: http://dx.doi.org/10.1186/1755-7682-6-15        [ Links ]

11. Delavarian M, Afrooz GA, Gharibzadeh S. Virtual reality and down syndrome rehabilitation. J Neuropsychiatry Clin Neurosci. 2012; 24(2):E7.         [ Links ]

12. Meyer DE, Kieras DE. A computational theory of executive cognitive processes and multiple-task performance: Part 1. Basic mechanisms. Psychol Rev. 1997;104(1):3-65.         [ Links ]

13. Scardovelli TA, Frere AF. The design and evaluation of a peripheral device for use with a computer game intended for children with motor disabilities. Comput Methods Programs Biomed. 2015;118(1):44-58. DOI: http://dx.doi.org/10.1016/j.cmpb.2014.10.002        [ Links ]

14. Heath M, Grierson L, Binsted G, Elliott D. Interhemispheric transmission time in persons with Down syndrome. J Intellect Disabil Res. 2007;51(Pt 12):972-81. DOI: http://dx.doi.org/10.1111/j.1365-2788.2007.01009.x        [ Links ]

15. Hedges JH, Adolph KE, Amso D, Bavelier D, Fiez JA, Krubitzer L, et al. Play, attention, and learning: how do play and timing shape the development of attention and infl uence classroom learning? Ann N Y Acad Sci. 2013;1292(1):1-20. DOI: http://dx.doi.org/10.1111/nyas.12154        [ Links ]

16. Possebom WF, Silva TD, Ré AHN, Massetti T, Belisário LZ, Ulian E, et al. Aprendizagem motora em pessoas com síndrome de Down: tarefa de labirinto no computador. Temas Desenvolv. 2013;19(104):54-60.         [ Links ]

17. Silva LM, Schalock M, Garberg J, Smith CL. Qigong massage for motor skills in young children with cerebral palsy and Down syndrome. Am J Occup Ther. 2012;66(3):348-55. DOI: http://dx.doi.org/10.5014/ajot.2012.003541        [ Links ]

18. Courbois Y, Farran EK, Lemahieu A, Blades M, Mengue-Topio H, Sockeel P. Wayfi nding behaviour in Down syndrome: a study with virtual environments. Res Dev Disabil. 2013;34(5):1825-31. DOI: http://dx.doi.org/10.1016/j.ridd.2013.02.023        [ Links ]

19. Souza D, França F, Campos T. Teste de labirinto: instrumento de análise na aquisição de uma habilidade motora. Rev Bras Fisioter. 2006;10(3):355-60. DOI: http://dx.doi.org/10.1590/S1413-35552006000300016        [ Links ]

20. Malheiros SR, da Silva TD, Favero FM, de Abreu LC, Fregni F, Ribeiro DC, et al. Computer task performance by subjects with Duchenne muscular dystrophy. Neuropsychiatr Dis Treat. 2015;12:41-8. DOI: http://dx.doi.org/10.2147/NDT.S87735        [ Links ]

21. Almeida AMP, Ramos FMS. Collaborative networked framework for the rehabilitation of children with Down's Syndrome. Italy: University of Reading; 2000.         [ Links ]

22. Davis M, Merrill EC, Conners FA, Roskos B. Patterns of differences in wayfi nding performance and correlations among abilities between persons with and without Down syndrome and typically developing children. Front Psychol. 2014;5:1446. DOI: http://dx.doi.org/10.3389/fpsyg.2014.01446        [ Links ]

23. Wuang YP, Chiang CS, Su CY, Wang CC. Effectiveness of virtual reality using Wii gaming technology in children with Down syndrome. Res Dev Disabil. 2011;32(1):312-21. http://dx.doi.org/10.1016/j.ridd.2010.10.002        [ Links ]

24. Menezes LDC, Massetti T, Oliveira FR, Abreu LC, Malheiros SRP, Trevizan IL,et al. Motor Learning and Virtual Reality in Down Syndrome; a literature review. Int Arc Med. 2015;8(119):1-11. DOI: http://dx.doi.org/10.3823/1718        [ Links ]

25. Gilmore L, Cuskelly M. A longitudinal study of motivation and competence in children with Down syndrome: early childhood to early adolescence. J Intellect Disabil Res. May 2009;53(5):484-92. DOI: http://dx.doi.org/10.1111/j.1365-2788.2009.01166.x        [ Links ]

26. Lanfranchi S, Jerman O, Dal Pont E, Alberti A, Vianello R. Executive function in adolescents with Down Syndrome. J Intellect Disabil Res. 2010;54(4):308-19. DOI: http://dx.doi.org/10.1111/j.1365-2788.2010.01262.x        [ Links ]

27. Lott IT, Dierssen M. Cognitive defi cits and associated neurological complications in individuals with Down's syndrome. Lancet Neurol. 2010;9(6):623-33. DOI: http://dx.doi.org/10.1016/S1474-4422(10)70112-5        [ Links ]

28. Menezes LDC, Gomes KSC, Massetti T, Silva TD, Possebom WF, Capelini CM, et al. Motor learning in mobile (cell phone) device in Down syndrome patients-pilot project. Medical Express. 2015;2(4):1-5. DOI: http://dx.doi.org/10.5935/MedicalExpress.2015.04.05        [ Links ]

29. Elliott D, Hansen S, Grierson LE, Lyons J, Bennett SJ, Hayes SJ. Goal-directed aiming: two components but multiple processes. Psychol Bull. 2010;136(6):1023-44. DOI: http://dx.doi.org/10.1037/a0020958        [ Links ]

30. Palisano RJ, Walter SD, Russell DJ, Rosenbaum PL, Gémus M, Galuppi BE, et al. Gross motor function of children with down syndrome: creation of motor growth curves. Arch Phys Med Rehabil. 2001;82(4):494-500. DOI: http://dx.doi.org/10.1053/apmr.2001.21956        [ Links ]

31. Block ME. Motor development in children with Down syndrome: A review of the literature. Adap Phys Activ Quart. 1991;8(3):179-209. DOI: http://dx.doi.org/10.1123/apaq.8.3.179        [ Links ]

 

 

Manuscript submitted: Jun 18 2016
Accepted for publication Jun 27 2016

 

 

Corresponding author: Thais Massetti. E-mail: thaismassetti@gmail.com

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons