ARTICLES

 

The effects of two computer-supported collaborative learning (CSCL) scripts on university students' critical thinking

 

Os efeitos de dois roteiros de aprendizagem colaborativa baseada em computador (ACBC) sobre o pensamento crítico de estudantes universitários

 

Los efectos de dos guías de aprendizaje ayudada basada en computadora (ACBC) sobre el pensamiento crítico de estudiantes universitarios

 

 

Tammy Schellens; Hilde Van Keer; Bram De Wever; Martin Valcke

Department of Education - Belgium

Correspondence to

 

 

---

ABSTRACT

The present study focuses on the use of two different types of scripts as possible ways to structure university students' discourse in asynchronous discussion groups and consequently promote their learning. More specifically, the aim of the study is to determine how requiring students to label their contributions by means of De Bono's Thinking Hats (script 1) and Weinberger's script for the construction of argumentation sequences (script 2) affects the ongoing critical thinking processes reflected in the discussion. The results suggest that both scripts successfully facilitated critical thinking. The results showed that the labeling condition (script 1) surpasses the argumentation script (script 2) with regard to the overall depth of critical thinking in the discussion, and the critical thinking processes during the stages of problem identification and problem integration in particular. Further, it can be argued that students in the labeling condition are engaged in more focused, more critical, and more practically-oriented discussions.

Keywords: Critical thinking, Problem solving, Learning.

---

RESUMO

O objetivo desse estudo é enfatizar o uso de dois diferentes roteiros como possibilidades para estruturar o discurso de estudantes universitários em grupos de discussão assíncrona e, conseqüentemente, promover o aprendizados dos mesmos. Mais especificadamente, esse estudo têm o objetivo de ensinar alunos a dar suas contribuições por meio do roteiro de De Bono's Thinking Hats (roteiro 1) e pelo roteiro Weinberger para a construção de seqüências argumentativas (roteiro 2) que afetam o curso do processo de pensamento presente numa discussão. Os resultados revelaram que rotulação (roteiro 1) supera o roteiro argumentativo (roteiro 2) em relação a profundidade do pensamento crítico global e do pensamento crítico durante o estágio de identificação de problemas e, em particular, na etapa relativa à integração de problemas. Além disso, pode ser considerado que alunos competentes em rotulação são mais engajados, críticos e orientados para a prática nas discussões.

Palavras-chave: Pensamento crítico, Resolução de problemas, Aprendizagem.

---

RESUMEN

El objetivo de este estudio es enfatizar el uso de dos guías diferentes como posibilidades para estructurar el discurso de estudiantes universitarios en grupos de discusión asíncrona y, consecuentemente, promover el aprendizaje de los mismos. Más especificamente, este estudio tiene el objetivo de enseñarle a alumnos a dar sus contribuciones por medio de la guía de De Bono's Thinking Hats (guía 1) y por la guía Weinberger para la construcción de secuencias de argumentos (guía 2) que afectan el curso del proceso de pensamiento que está presente en una discusión. Los resultados mostraron que rotulación (guía 1) supera la guía argumentativa (guía 2) en relación a la profundidad del pensamiento crítico global y del pensamiento crítico durante el periodo de identificación de problemas y, particularmente, en la etapa relativa a la integración de problemas. Además de eso, puede ser considerado que los alumnos competentes en rotulación son más comprometidos, críticos y orientados para la práctica en las discusiones.

Palabras clave: Pensamiento crítico, Resolución de problemas, Aprendizaje.

---

 

 

Introduction

Computer-supported collaborative learning (CSCL), and asynchronous discussion groups in particular, is very popular because of the positive effects on different aspects of learning. Empirical evidence however stresses the importance of engaging students in high-quality interaction as a prerequisite for supporting online learning. In this respect, scripts can be regarded as an approach to facilitate true collaborative learning. With the present study we want to shed light on the impact of different kind of scripts and the different aspects that should be taken into account when designing CSCL environments.

A script can be defined as a detailed and more explicit didactic contract between the teacher and the group of students regarding their mode of collaboration (Dillenbourg, 2002). Certain scripts, for instance, stimulate learners to construct specific arguments by providing them prompts on which they have to respond (Baker & Lund, 1997; Dillenbourg, 2002; Kollar, Fischer & Hess, 2003; Weinberger, 2003; Weinberger, Ertl, Fischer & Mandl, 2005). This approach is particularly interesting to specify, sequence, and eventually to allocate different learning activities to learners (Weinberger et al., 2005).

The concept of 'script' however encompasses a very broad range of methods, techniques, and approaches. In this respect it is difficult to speak about the overall efficacy of CSCL scripts (Dillenbourg, 2002). In this study, we compared the use of two different types of scripting tools and the impact of these scripts on the critical thinking processes reflected in the discussion. As part of the course 'Instructional Strategies', 57 third-year university students were engaged in asynchronous discussion groups debating different perspectives, possibilities, and limitations of "constructivism" during two weeks.

Two research conditions were distinguished. In the first condition (script 1), students were required to tag their messages by means of De Bono's (1991) Thinking Hats. Each note in the discussion had to be associated to one of a predefined set of six labels expressing different thinking types. The aim of requiring students to tag their posts is twofold. First, it obliges students to reflect upon the nature of their contribution and on how it will add to the ongoing discussion. Second, the labels improve the outline of the discussion and indicate the predominance or absence of one or more thinking types. Table 1 presents an overview of the six thinking types reflected in De Bono's thinking hats (1991).

In the second condition (script 1), the argumentation-visualization script based on Weinberger, Stegmann, and Fischer (2005) was used to support the construction of argumentation sequences. With this script, there is an imposed path of argument, counterargument, and integration through the discussion.

 

 

Oliver (2001) argues that critical thinking skills represent an important issue for education and that these skills are particularly important nowadays in order to make meaningful use of electronic information. Boxler (2002) considers critical thinking as a main tool that one must develop and use to enact social change. Although asynchronous discussion groups might support opportunities for engagement in various cognitive processes such as critical thinking, they do not guarantee it (Murphy, 2004). Few studies specifically focus on critical thinking. However, some aspects of critical thinking have been investigated in online asynchronous discussion environments. Empirical evidence stresses for instance the importance of engaging students in high-quality interaction as a prerequisite for supporting critical thinking and deep-level learning. Because there is little empirical evidence on whether and how computer-supported collaboration scripts specifically aimed to support critical thinking are really successful, the following research question was explored: Do messages of students who were required to tag their contributions to asynchronous discussion by means of De Bono's Thinking Hats (1991) differ from messages of students who were required to follow a script for the construction of argumentation sequences with regard to the overall depth of critical thinking, the depth of critical thinking for different categories and indicators, and the depth of critical thinking at the successive critical thinking stages distinguished by Garrison (1992).

Theoretical framework

Although most educators agree on the importance of critical thinking for learning, there is much disagreement about the exact meaning of the term "critical thinking" and there is no universal definition of critical thinking for education. In the present study, we agree with the definitions of Chance (1986) and Scriven and Paul (1992) who respectively define critical thinking as the ability to analyze facts, generate and organize ideas, defend opinions, make comparisons, draw inferences, evaluate arguments, and solve problems (Chance, 1986) and as the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/ or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action (Scriven & Paul, 1992).

A number of theorists have considered critical thinking as a problem-solving process (e.g., Brookfield, 1987; Garrison, 1992). Garrison (1992) more particularly identifies five phases of critical thinking. According to his theory, critical thinkers move through the stages of identifying a problem, defining it more clearly, exploring the problem and possible solutions, evaluating their applicability, and integrating this understanding with existing knowledge. The model employed to analyze the discourse in the present study is based on Garrison's model which is a dynamic cognitive one, similar to models of problem-solving used in cognitive psychology and artificial intelligence. Although Garrison initially developed it as a means of studying individual learning, it requires shared understanding with others and is therefore suitable for studying group learning as well. Underneath, the different stages are illustrated briefly.

Stage 1: Problem identification. Students start by identifying a problem and gather information on it. This information is made available for other students in the online discussion groups. It is in the interaction with others that the problem is identified.

Stage 2: Problem definition. In the problem definition phase students have to define problem boundaries, ends, and means. At this point students should bring in outside information from textbooks or own experience to clarify the problem. It requires the students to identify a position before putting arguments for and against it.

Stage 3: Problem exploration. The phase of problem exploration can be regarded as the creative generation of new ideas, since it is the most creative part of the critical thinking process. Here students have to explore the problem and possible solutions. They use both logical reasoning and creative thinking to extend their understanding beyond the basic problem definition (Newman et al., 1995).

Stage 4: Problem evaluation/applicability. In the fourth stage, students critically evaluate possible solutions, link ideas together, and try to find out if these proposed solutions can work in practice.

Stage 5: Problem integration. When students integrate the solutions into existing knowledge, they need to validate the solution within the group. This is the stage where the solutions are grounded in the real world. This requires feedback. In this respect, not only external feedback is necessary; but criticism from other group members is important as well.

 

Method

Sample and design

All students enrolled for the course 'Instructional Strategies' participated in the present study (N=57). The students were subdivided into 6 groups and each group was randomly assigned to one of the two research conditions. Students were either required to tag their contributions by means of De Bono's (1991) Thinking Hats (script 1) or to post messages following the argumentation-construction model of Weinberger, et al. (2005) (script 2).

With regard to the script 1 condition, the online discussion environment offered a checklist interpreting the different types of contributions advancing the discussion process. For each Thinking Hat, the students received a description of what the hat implies in terms of a discussion contribution. In the script 2 condition, we aimed to facilitate a specific argumentation sequence of argument-counterargument-integration (Weinberger, et al., 2005; Leitão, 2000). In this script, each first message of a discussion tread has to be labelled "argument". The answer to an argument should than be labelled as "counterargument" and a reply to a counterargument has to be labelled as "integration". The next message is again a "counterargument", then "integration" and so on. In this way, there is an imposed path throughout the discussion. If necessary, students can start a new thread, starting again with an argument.

Task environment and procedure

Students participated in an asynchronous discussion session of two weeks. Students were flexible as to time and place to work on the discussion assignment within this two-week frame. During the first face-to-face session of the course, the CSCL-environment was demonstrated. A typology functionality was included in the system. The typology and the different types, in this case, the different hats were defined and added to the forum. If students wanted to post a message to the forum, they first had to pick a type of the typology concerned, in this case this means choosing between red, white, green, yellow, black, or blue hat from a drop-down menu. When their message was contributed to the forum, the hat chosen appeared next to their message.

A number of strict rules were stated to define the expected student participation: participation in the discussion groups was a formal part of the course, successful participation implied that each student posted at least 5 messages, and the instructor followed the ongoing discussions and limited the interventions to structural feedback. To ensure a correct use of the labels students received a thorough training with regard to the differences between the labels. Moreover the information and description of the different hats was incorporated in the discussion environment and visible for the students at all times.

The nature of the discussion assignment was the same for all discussion groups in the study. The same learning goal, context, expectations, time requirements, and deliverables was put forward in all discussion group, regardless of the research condition the groups were in. The discussion assignment allowed learners to construct different arguments pro or contra "constructivism". An online column on EduSite (http://www.edusite.nl/edusite/ columns/) served as a starting point. Students were asked to read this text and to discuss the content from different perspectives. They were urged to consult the literature suggested in the course reader and to search for additional information to feed the discussion.

Data analysis

Content analysis was applied in order to study the critical thinking processes reflected in the discussions. More particularly, the content analysis scheme based on Newman, Webb, and Cochrane (1995) was used. This content analysis instrument is based on Garrison's (1992) five stages of critical thinking and Henri's (1992) cognitive skills. It identifies 10 critical thinking categories: relevance, importance, novelty, outside knowledge, ambiguities, linking ideas, justification, critical assessment, practical utility, and width of the discussion. For each category, a number of positive and negative indicators are formulated and most indicators are fairly obvious opposites (Newman et al., 1995). Within the framework of the present study all critical thinking categories and indicators distinguished by Newman et al. (1995) were adopted, except for the indicators referring to tutor postings. Taken into account the restricted time span of a two-week discussion we requested students to move on quickly to a more focused discussion. Therefore, the category 'Width of the discussion' received a deviated interpretation, assigning a positive connotation to focused discussions and a negative connotation to broad discussion contributions. In this respect, the category was renamed 'Focus of the discussion'. The indicators used in the present study are listed in Table 2.

 

 

For each of the 6 groups, the complete discussion transcripts were analyzed. This encompasses a total of 510 messages. According to Newman et al. (1995), phrases, sentences, paragraphs, as well as messages illustrating at least one of the indicators can serve as units of analysis. The authors only mark and count the obvious examples, and ignore less clear indicators. They do not report reliability data and according to Marra, Moore, and Klimczak (2004) calculating inter-reliability is not even possible given that the unit of analysis varies from phrases, to paragraphs, or to the entire posting. Therefore, in the present study the whole message was used as the unit of analysis. According to Rourke, Anderson, Garrison, and Archer (2001) this results in the objective identification of all units of analysis. Each message that obviously indicated critical or uncritical learning according to the indicators explained above was analyzed. Per category only one indicator could be chosen. This means that every message could receive a maximum of 10 codes.

Two coders coded the messages independently. Training was provided to all coders and included a thorough explanation of the coding process, written coding rules and guidelines, examples and non-examples and practice with sample data. Group discussion helped students to get acquainted with the particularities of the coding scheme and to reach mutual agreement about the coding category to be selected. Inter-rater reliability was calculated and found satisfactory for each category of critical thinking.cTo check the correspondence between the students' message label and the actual message content, a sample of the messages (5%) posted in the script 1 condition was analyzed. An agreement of almost 87% was found which indicates that students generally used the correct labels. The fact that there was a clear distinction between the different labels, can explain this high correspondence. Students were not likely to mix up between hats. Moreover, the training students received and the description of the hats which remained visible during the discussions strengthened this correct use of the hats.

An equally large sample of the messages posted in the script 2 condition was checked for correct use of the script. The analyses of the discussions revealed that in almost all cases students followed the script as they were instructed to play it. Only in 6% of our sample students did not completed the argument followed by counter-argument path.

 

Results and Discussion

As to the results, we will first focus on differences with regard to the overall depth of critical thinking. Secondly, the effect of the research conditions on the separate categories and indicators of critical thinking is studied. Finally, the findings are related to Garrison's stages of critical thinking (1992). Therefore, the depth of critical thinking at successive stages is examined.

As to the overall depth of critical thinking, a critical thinking ratio was calculated on the basis of the coded critical thinking indicators. This ratio varies from -1, indicating all uncritical thinking statements to +1, indicating all critical thinking statements. The results with regard to the overall depth of critical thinking in the discussion groups reveal more statements indicating critical thinking than the opposite in both research conditions, implying that there is evidence for critical thinking in both conditions. Analysis of variance, however, indicates that students tagging their contributions by means of the Thinking Hats surpassed the argumentation-visualization group significantly in the achieved general depth of critical thinking (mean depth of critical thinking = 0.41, respectively = 0.30) (F(1, 359)=14.25, p<0.001).

Apart from the overall depth of critical thinking, the content analysis scheme of Newman et al. (1995) makes it possible to study different components of critical thinking through the different categories and indicators. To enable more detailed statements with regard to the differential impact of both research conditions on students' critical thinking in the discussions, in a second step the global measure of overall depth of critical thinking was split up by analyzing the ratios for each critical thinking category and the incidence of the separate critical thinking indicators. Table 3 gives an overview of the critical thinking ratios for each category per research condition. Figure 1 shows the overall pattern, comparing both research conditions.

 

 

 

As can be seen in Figure 1, the patterns between both research conditions with regard to the different critical thinking categories are generally quite similar. Analyses of variance, however, indicate significantly higher critical thinking ratios for the discussion focus (F(1, 359)=29.28, p<0.001), for the critical assessment reflected in messages (F(1, 357)=10.52, p=0.001), and for discussing practical utility (F(1, 349)=15.19, p<0.001) in the condition in which students tag their messages by means of De Bono's (1991) Thinking Hats. Multinomial logistic regressions corroborate these findings. More specifically, it can be concluded that students who are required to reflect on and to tag the type of thinking in their contributions are 3.26 times more prone to engage in in-depth discussions than their peers in the argumentation-visualization condition who are involved in more general online discourse. Further, students in the script 1 condition are also 2 times more likely to include critical assessment of one's own or others contributions. Finally, these students tend in to bring 3.85 times more possible solutions to familiar situations and 2.14 times more discussions points regarding practical utility of new ideas.

In order to study the depth of critical thinking taking place in each of Garrison's stages of critical thinking (1992), in the third step of the analyses each indicator was related to the stage in which it is most expected. For example, it can be expected that new problem-related information is to be introduced in Garrison's stage of problem definition (stage 2). The critical thinking indicators were mapped to Garrison's five stages by means of the procedure reported by Newman et al. (1996) and presented in Table 2. Table 4 gives an overview of the mean critical thinking ratios for each of the stages.

 

 

In Figure 2, the calculated critical thinking ratios per stage are plotted for each research condition. Figure 2 indicates that the patterns of critical thinking during the successive stages identified by Garrison (1992) are quite similar for both research conditions. Analyses of variance, however, reveal significantly deeper critical thinking for the students in the script 1 condition at the stages of problem identification (F(1, 359)=14.25, p<0.001) and problem integration (F(1, 359)=4.41, p=0.036), indicating that students who are required to tag the type of thinking in their messages are doing better at identifying a problem by going to the core of the matter and at integrating the solutions emerging from the discussion into existing knowledge than the students asked to follow an argumentation path.

 

 

In conclusion, it can be argued that asking students to identify their thinking types by means of De Bono's Thinking Hats (1991) or to follow an argumentation path appears to be successful to foster profound critical thinking in asynchronous discussions. These results corroborate the assumption that scripts can be regarded as a way to facilitate true collaborative learning (Dillenbourg, 2002; Kollar, Fischer & Hesse, 2003; Weinberger, 2003). However, the labeling condition (script 1) surpasses the argumentation script (script 2) with regard to the overall depth of critical thinking in the discussion, and the critical thinking processes during the stages of problem identification and problem integration in particular. Further, it can be argued that students in the labeling condition are engaged in more focused, more critical, more and practically-oriented discussions. The fact that the results in the labeling condition exceed the results in the argumentation-visualisation condition is probably due to the fact that in the labeling condition, students are asked to step back and to reflect upon the ongoing discussion and on how to contribute to optimize the debate. Moreover, the labels visualize the possible predominance or absence of one or more thinking types. The six hats together encompass the successive stages of critical thinking. Since students were invited to apply a whole range of thinking hats, going through the successive stages of critical thinking was stimulated. In the argumentation-visualisation condition students were engaged in an identical assignment, however the imposed path of argument, counterargument, and integration through the discussion is more restricted and does not cover all the different stages and components of critical thinking. Moreover since this kind of script forces each contribution in a straitjacket we have to be aware of the danger of too rigid scripting (Dillenbourg, 2002).

Clearly the results for both scripting conditions are promising, however the results indicate that not all scripts fit in for every kind of computer supported collaboration. The scripts should be adapted depending on the specific goal and the discussion processes you try to stimulate.

 

References

Baker, M., & Lund, K. (1997). Promoting reflective interactions in a CSCL environment. Journal of Computer Assisted Learning, 13, 175-193.        [ Links ]

Boxler, H. (2002). Quest for the grail? Searching for Critical Thinking in Adult Education. Paper presented at the 2002 Adult Education Research Conference. North Carolina        [ Links ]

Brookfield, S. D. (1987) Developing Critical Thinkers. Challenging adults to explore alternative ways of thinking and acting. Milton Keynes: Open University Press.        [ Links ]

Chance, P. (1986). Thinking in the classroom: A survey of programs. New York: Teachers College, Columbia University        [ Links ]

De Bono E. (1991). Six Thinking Hats for schools, Resource book for Adult Educators. Logan (IA): USA Perfection Learning.        [ Links ]

Dillenbourg, P. (2002). Over-scripting CSCL: The risks of blending collaborative learning with instructional design. In P.A. Kirschner (Ed.), Three worlds of CSCL: Can we support CSCL? (pp. 61-91). Heerlen: Open Universiteit Nederland.         [ Links ]

Garrison, D. R. (1992). Critical thinking and self-directed learning in adult education: An analysis of responsibility and control issues. Adult Education Quarterly, 42, 136-148.        [ Links ]

Henri, F. (1992). Computer Conferencing and Content Analysis. In A. R. Kaye (Ed.), Collaborative Learning Through Computer Conferencing. The Najadan Papers (pp. 117-136). London: Springer-Verlag.        [ Links ]

Kollar, I., Fischer, F., & Hesse, F. (2003). Cooperation scripts for computer-supported collaborative learning. In B. Wasson, R. Baggetun, U. Hoppe & S. Ludvigsen (Eds.), Proceedings of the International Conference on Computer Support for Collaborative Learning - CSCL 2003, COMMUNITY EVENTS -Communication and Interaction (pp. 59-60). Bergen, Norway: InterMedia.        [ Links ]

Leitão, S. (2000). The potential of argument in knowledge building. Human development, 43, 332-360.        [ Links ]

Marra, R. M., Moore, J. L., & Klimczak, A. K. (2004). Content analysis of online discussion forums: A comparative analysis of protocols. Educational Technology Research Development, 52, 23-40.        [ Links ]

Murphy, E. (2004) An instrument to support thinking critically about critical thinking in online asynchronous discussions. Australasian Journal of Educational Technology, 20, 295-315.        [ Links ]

Newman, D. R., Webb, B., & Cochrane, C. (1995). A content analysis method to measure critical thinking in face-to-face and computer supported group learning. Interpersonal Computing and Technology, 3, 56-77.        [ Links ]

Oliver, R. (2001). Exploring the development of critical thinking skills through a Web-supported problem-based learning environment. In J. Stephenson (Ed.), Teaching and Learning Online: Pedagogies for New Technologies (pp. 98-111). VA: Kogan Page.        [ Links ]

Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (2001). Methodological issues in the content analysis of computer conference transcripts. International Journal of Artificial Intelligence in Education, 12, 8-22.        [ Links ]

Scriven, M., & Paul, R. (1992). Critical thinking defined. Paper presented at the Critical Thinking Conference, November 1992, Atlanta, GA.        [ Links ]

Weinberger, A. (2003). Scripts for computer-supported collaborative learning. Effects of soical and epistemic cooperation scripts on collaborative knowledge construction. München, Unpublished doctoral dissertation. Ludwig-Maximilians-Universität. Retrieved August 15, 2004, from http://edoc.ub.uni-muenchen.de/archive/00001120/01/ Weinberger_Armin.pdf        [ Links ]

Weinberger, A., Ertl, B., Fischer, F., & Mandl, H. (2005). Epistemic and social scripts in computer-supported collaborative learning. Instructional Science, 33, 1-30.        [ Links ]

Weinberger, A., Stegmann, K., & Fischer, F. (2005). Computer-supported collaborative learning in higher education: Scripts for argumentative knowledge construction in distributed groups. In T. Koschmann, D. Suthers & T. W. Chan (Eds.), Computer Supported Collaborative Learning 2005: The Next 10 Years (pp. 717-726). Mahwah, NJ: Lawrence Erlbaum.        [ Links ]

 

 

Correspondence to
E-mail: tammy.Schellens@UGent.be