Serviços Personalizados
Journal
artigo
texto em 
Inglês (pdf)
Artigo em XML
Referências do artigo
Tradução automáticaIndicadores
Compartilhar
Permalink
Revista Brasileira de Terapias Cognitivas
versão impressa ISSN 1808-5687versão On-line ISSN 1982-3746
Rev. bras.ter. cogn. vol.19 no.1 Rio de Janeiro jan./jun. 2023 Epub 01-Jul-2024
https://doi.org/10.5935/1808-5687.20230007
Empirical Research
Development of a Group CBT Competence Scale (GCBT-CS)
IUniversidade de São Paulo, Faculdade de Filosofia, Ciências e Letras de Ribeirão Preto - Ribeirão Preto - São Paulo - Brasil
The current work presents the process of development of the Group Cognitive-Behavioral Therapy Competency Scale (CS-GCBT) and initial evidence of its content validity, aiming to fill the gap in the scope of therapist assessment in this modality, which is in wide dissemination. The process involved four steps: designation of the theoretical framework, development of the preliminary version, analysis of items by seven expert judges and semantic analysis of items by the target population. The scale has currently of 18 items. The development of this scale combined theoretical principles, consultation of recognized instruments and experts, and consultation with the target population, having also gone through several phases of item refinement. Furthermore, it has the advantage of combining quantitative and qualitative methods. Thus, it is considered that the method used was satisfactory to guarantee good initial indicators of content validity. It is expected that, with the continuity of the validation process, the scale will contribute to the process of training therapists in Brazil.
Keywords: Psychotherapy, Group; Professional Competence; Cognitive Behavioral Therapy
Keywords: Psicoterapia de grupo; Competencia profesional; Terapia cognitivo-conductual
O presente trabalho tem como objetivo apresentar o processo de construção e evidências iniciais de validade de conteúdo da Escala de Competências em Terapia Cognitivo-comportamental em Grupos (EC-TCCG), visando suprir a lacuna no âmbito da avaliação do terapeuta nessa modalidade que se encontra em ampla disseminação. O processo envolveu quatro etapas: fundamentação teórica, construção da versão preliminar, análise dos itens por sete juízes especialistas e análise semântica dos itens pela população-alvo, sendo que o instrumento conta atualmente com 18 itens. A construção da escala conciliou pressupostos teóricos, consulta a instrumentos reconhecidos, especialistas e população-alvo, tendo passado, ainda, por várias fases de refinamento dos itens. Além disso, apresenta o diferencial de conciliar métodos quantitativos e qualitativos. Dessa forma, considera-se que o método utilizado foi satisfatório para garantir bons indicadores iniciais de validade de conteúdo. Com a continuidade do processo de validação, espera-se que o instrumento possa contribuir com o processo de formação de terapeutas no Brasil.
Palavras-chave: Psicoterapia de grupo; Competência profissional; Terapia cognitivocomportamental.
El objetivo de este trabajo es presentar el proceso de construcción y las evidencias iniciales de la validez de contenido de la Escala de Competencias en Terapia Cognitivo-conductual Grupal (EC-TCCG), con el fin de cubrir el vacío existente en la evaluación del terapeuta de dicha modalidad, que se encuentra en franca diseminación. El proceso constó de cuatro etapas: fundamentación teórica, construcción de la versión preliminar, análisis de los ítems por parte de siete jueces expertos y análisis semántico de los ítems por parte del público destinatario, siendo que el instrumento cuenta actualmente con dieciocho ítems. La construcción de la escala concilió presupuestos teóricos y la consulta de instrumentos reconocidos, expertos y público destinatario y también pasó por varias fases para refinar los ítems. Además, cuenta con la singularidad de conciliar métodos cuantitativos y cualitativos. Por lo tanto, se considera que el método utilizado ha sido satisfactorio para garantizar buenos indicadores iniciales de validez de contenido. Con la continuidad del proceso de validación, se espera que el EC-TCCG pueda contribuir al proceso de formación de terapeutas en Brasil.
With the emergence of Evidence-Based Practice in Psychology (EBPP), the professional field of psychology has entered a new era of evaluating education and training, standards, and practice guidelines based on the measurement of professional competencies using reference documents, guidelines, and competency models (Barlow, 2012). In this context, Cognitive-Behavioral Therapy (CBT), being an evidence-based practice (Dobson & Dobson, 2018), has increasingly considered competency assessment methods as central to research on processes and outcomes. The means of assessing competency in training programs most recommended by specialists has been examining therapist practice using competency measurement instruments completed by their supervisor to assess specific competencies or skills, based on the observation of live sessions or video recordings (Muse & McManus, 2016).
In Brazil and Latin America, it is known that such practices and publications on this topic are still scarce (Scotton et al., 2021), as well as EBPP itself (Melnik et al., 2019). Training in our region has been criticized for dissociating scientific training from practice, with our limited production being focused on supervision processes. The available literature highlights the lack of models and systematization for supervision and the lack of diversified formative pedagogical strategies, as well as the scarcity of empirical studies on the effectiveness of training programs and therapeutic outcomes (Barletta & Neufeld, 2020; Reis & Barbosa, 2018; Scotton et al., 2021).
The Cognitive Therapy Scale (CTS; Young & Beck, 1980) and the Cognitive Therapy Scale Revised (CTS-R; Blackburn et al., 2001) have been the most widely used instruments for evaluating cognitive-behavioral therapist sessions, as well as for validating other instruments for accessing competencies in specific disorders and populations. The CTS was developed by Young and Beck (1980) to assess therapist competency in implementing CBT protocols. It is an 11-item scale to be scored by an external evaluator and is divided into two subscales: General Skills and Specific Skills. Each item corresponds to a theoretical-practical assumption (e.g., agenda, feedback, use of techniques) of treatment delivery. However, research has not supported this division, revealing it to be a unidimensional instrument (Reis & Barbosa, 2018). In 2020, Moreno and DeSouza released a cross-cultural adaptation of the CTS for the Brazilian context, but a study with psychometric data for this version has not been published yet.
The CTS-R, on the other hand, was developed to overcome limitations of the CTS. It established statements for the items (rather than just presenting the competency to be assessed, as in the CTS) and a more specific scoring system, incorporating adherence, placing greater emphasis on emotional expression, reducing item overlap, and incorporating nonspecific elements related to therapeutic alliance (Blackburn et al., 2001; Muse & McManus, 2013; Reis & Barbosa, 2018). Unlike the CTS, the CTS-R already has a translated and adapted version for the Brazilian context with statistical studies and good psychometric characteristics (Reis & Barbosa, 2018). According to the authors of this version, factor analyses also indicated it to be a unidimensional instrument.
In addition to these instruments, in 2017, Muse et al. developed the Assessment of Core CBT Skills (ACCS), aiming to address the critiques toward both the CTS and the CTS-R (see Muse et al., 2017, for more information). This scale is based on the two preexisting scales to provide an assessment framework, but incorporated several strategies to address their limitations, essentially constituting a new instrument. Notably, it includes space after each item for narrative feedback, in addition to the numerical ratings. However, it is worth mentioning that this instrument does not yet have a translated and adapted version for the Brazilian context like the previous scales.
With the widespread adoption of CBT in various countries, group cognitive-behavioral therapy (GCBT) interventions have gained increasing prominence both in research and clinical practice over the last three decades in Brazil due to its efficacy and cost-effectiveness (Rangé et al., 2017). Similar to individual CBT practice, there are various guidelines and practical training recommendations available from competent international professional institutions for group intervention. However, these have not yet been integrated into psychology training programs in a systematic competency-based framework (Goigoechea & Kessler, 2018).
It is noteworthy that there is a shortage of research on therapists’ adherence and competency in group therapy, and there are several limitations regarding the application of existing scales to the group therapy environment. These scales, for example, do not measure therapist competency in managing aspects of group processes, among other equally important specific competencies for GCBT therapists (for more details on group therapist competencies in CBT, see Barlow, 2012; Pohl et al., 2017). The literature also indicates that a group therapist in CBT needs to master both the competencies typically required in individual therapy (e.g., working on behavioral activation for a patient with depression) and the aforementioned competencies related to group processes, namely, managing aspects arising from interaction among group members (Neufeld et al., 2017).
In this direction, researchers from Germany (Pohl et al., 2017) published a study on the development of an assessment instrument for GCBT therapists, the CTS-D-G (German Cognitive Therapy Scale for Group Therapy). This instrument aims to fill the gap in the evaluation of group therapists. The CTS-D-G is based on both the CTS and the CTS-R. In the CTS-D-G, the items were reformulated, and the instrument was expanded to include items specifically designed to assess therapist competency in group settings. Early statistical studies of the instrument indicated good psychometric characteristics (Pohl et al., 2017).
This German scale is currently the only instrument found to assess the competencies of group cognitive-behavioral therapists (and is limited to the German context). Given this scarcity, there is a call in the literature for the development of assessment measures for group therapist interventions as a next step in this field (Pohl et al., 2017). It is also worth noting that while instruments focused on evaluating CBT therapist competencies are already available for use in Brazil, the adoption of GCBT in the country makes it essential for instruments tailored to the group context to be available, ultimately aiming to ensure the quality of group interventions offered and to provide strategies and tools for therapist training.
To address this gap in the evaluation of group therapists in CBT, this study aims to present the construction and initial content validation of an instrument for this purpose. It also aims to describe the methodological process used, proposing a feasible approach to constructing high-quality instruments with an emphasis on theoretical foundation. The hypothesis is that the construction of this scale, based on careful theoretical and methodological principles and consultation of recognized therapist competency assessment instruments, will present good initial indicators of content validity.
METHOD
The present study employs a methodological design for instrument development. Initially, the authors intended to perform the translation and cross-cultural adaptation of the German Cognitive Therapy Scale for Group Therapy (CTS-D-G).
For this purpose, the translation process of the CTS-D-G was initiated. However, upon analyzing the data derived from the theoretical investigations of the first author, obtained from the other stages of this instrument’s procedure (described below), and considering the second author’s expertise in GCBT, a decision was made to construct a new instrument. This decision was based on significant gaps identified in the CTS-D-G concerning fundamental aspects of GCBT theory and practice. Nevertheless, it should be noted that, guided by the argument that international instruments are widely grounded in robust theoretical foundations, these instruments were used as an initial basis for the current scale.
Theoretical foundation
Several procedures were undertaken in the theoretical foundation stage, which is the primary step in constructing and validating content. Initially, a preliminary reading about the topic of essential competencies for cognitive-behavioral therapists was conducted to allow the researcher to become more deeply familiar with the subject. In this phase, there was an opportunity to consult with an expert in the field, a researcher specializing in therapist training processes, a faculty member and supervisor, with whom multiple consultations were held.
An integrative literature review was conducted of national and Latin American literature, guided by the question: “What are the essential competencies for clinical psychologists?” The review was conducted across the PsycINFO, LILACS, Scielo, and Pepsic databases, encompassing articles by Latin American authors, written in Portuguese, English, and Spanish, published within the last 12 years in the field of Psychology, addressing therapist competencies in general (without specifying theoretical approach) or cognitive-behavioral therapist competencies as the main subject (Scotton et al., 2021).
To delve further into the topic and investigate the specific competencies required for group therapists, an additional integrative literature review of national and Latin-American literature was planned, following the same procedures as the first review, only adapting keywords and criteria specifically to group therapists. However, no articles meeting the inclusion criteria were found. For this reason, a narrative literature review was conducted, including books, chapters, relevant international articles, and international competency guidelines recognized by major professional organizations in the field (Scotton & Neufeld, in press).
Furthermore, an additional empirical qualitative study was conducted, involving two focus groups. The first group included seven psychologists who were researchers and faculty members responsible for supervising undergraduate and postgraduate students in Brazil. They also supervised therapists in training in clinical internships for Psychology programs in states throughout the country. This group was presented with the following guiding question: “What competencies are necessary for a cognitive-behavioral therapist?” The second focus group included eight psychologists conducting their postgraduate studies (master’s and doctoral students) who were also supervisors for internships involving GCBT practice in undergraduate Psychology programs in Brazil. The guiding question for this group was adapted specifically for the practice of GCBT therapists (Scotton & Neufeld, under revision).
Construction of preliminary version
As mentioned above, the original goal was to translate and adapt the CTS-D-G. For this purpose, after obtaining authorization from the authors of the original instrument, two independent translators were contacted for the first stage. Following the International Test Commission (ITC, 2017) guidelines, both translators were native in the target language (Brazilian Portuguese), lived within Brazilian culture, were fluent in the source language (German), and were familiar with German culture, being able to identify language expressions and specificities. After obtaining two translated versions of the instrument, a committee of judges was formed, consisting of the authors and an expert judge in the field. This committee conducted a synthesis, compared the different translations and assessed their semantic, idiomatic, conceptual, linguistic, and contextual discrepancies with the goal of arriving at a unified version (ITC, 2017). It is important to note that the judge had technical knowledge in the instrument’s domain and familiarity with test construction principles. Additionally, the second author of this study also possessed these qualifications, as well as fluency in both the source and target languages and being a native speaker of the latter.
Upon analyzing the first version of the instrument, the authors decided to make significant alterations due to the gaps identified. Therefore, the construction of a new instrument started, building upon the synthesis of the CTS-D-G translations. For these modifications, besides the CTS-D-G, the Brazilian version of the CTS-R (Reis & Barbosa, 2018) and the ACCS (Muse et al., 2016) were used as references to form the main structure of the instrument (item descriptions and response formats), along with data from the theoretical foundation stage for creating new items.
A document was created synthesizing a comparison between CTS-R and CTS-D-G items (both regarding the competencies addressed per item and the phrasing of item descriptions and response scales) and the competencies listed in the reviews and focus groups. Additionally, the response scale used in both instruments was studied and adjusted based on the adopted literature. Thus, a second version of the instrument was developed.
Judges’ analysis
In this stage, the second version of the instrument was evaluated by seven psychologist judges, intentionally chosen based on their education and, most importantly, clinical experience in psychological assessment and/or professional practice involving supervision in GCBT. An online form was created, containing a brief introduction explaining the construction process, which considered the aforementioned instruments and the main modifications made. First, the judges were asked to assess the adequacy of rapport and the adequacy of the response scale, using a three-point Likert scale (adequate, partially adequate, inadequate). To evaluate the instrument items, the judges were asked to evaluate: a) the semantic/linguistic adequacy of the item; b) the adequacy of the content of the item statement, based on their knowledge of CBT and GCBT; c) the adequacy of the content of the response scale points for each item (adequacy of examples, adequacy with the proposed level), based on their knowledge of CBT and GCBT; and d) the adequacy of the proposed item change in relation to the base instruments (Brazilian version of the CTS-R and translated version of the CTS-D-G), using the same Likert scale for each instance. Additionally, a space for narrative feedback was provided at the end of each item for suggestions for improvement if the item was considered inadequate or partially adequate.
The evaluations of all judges were compiled into a spreadsheet and analyzed quantitatively and qualitatively. Quantitative analysis was conducted by calculating the Content Validity Index (CVI) for each item and the instrument as a whole, with values above 0.8 accepted as indicative of item quality for the judged aspect. Qualitative analysis was carried out by the authors based on the narrative feedback and discussions with judges, constituting a methodological triangulation. Items with CVI scores below 0.8 were modified based on the judges’ suggestions. Additionally, items meeting the quantitative criterion but receiving relevant modification suggestions from judges were also adjusted (Coluci et al., 2015).
Semantic analysis of items
The semantic analysis was conducted to verify the comprehensibility of all items among supervisors in GCBT, the target population of the instrument (Coluci et al., 2015). Seventeen psychologists participated in this stage, selected based on their experience in GCBT supervision within undergraduate Psychology programs across Brazil. These psychologists were contacted via email or messaging applications, through which they were informed about the research objectives and instructed about their role in the study. The instrument was sent to each participant, and they were asked to assess the rapport and each item, including responses, for comprehensibility. They were also requested to highlight any potential difficulties they perceived in case they were to use the instrument to evaluate a group intervention session. Supervisors could respond either in written form or orally. Any items that posed comprehension difficulties were modified, resulting in a fourth version of the instrument.
Ethical Procedures
This study is part of a larger project investigating clinical competencies in CBT in Brazil that has been approved by the Research Ethics Committee (CAAE: 91440818.0.0000.5407). All participants provided informed consent by signing a consent form. Ethical procedures were adopted in accordance with Resolution No. 510/2016 of the National Health Council for research involving human subjects.
RESULTS
Theoretical foundation
Through preliminary reading and consultations with the expert researcher, we engaged with significant empirical and review studies, books, chapters, and primarily the most utilized therapist competency guidelines up to this point. These guidelines represent syntheses guided by expert groups which, in turn, are based on reviews, manuals, and training materials and are peer-reviewed (Muse & McManus, 2016).
An integrative review of the literature also facilitated contact with Latin American studies published after the release of key guidelines (using year of publication as one inclusion criterion). Competencies outlined in these articles were categorized into four groups: analytical, instrumental, social, and self-awareness and self-reflection (see Scotton et al., 2021, for more information). Through narrative review, primary information on group therapist competencies were synthesized, highlighting the guideline based on Rodolfa’s cube adapted for groups (Barlow, 2012). This article emphasized competencies tied to group processes, such as therapist management of group roles and balancing member participation. This study systematized the main competencies to be considered (see Scotton & Neufeld, in press, for more information).
The conducted focus groups each lasted approximately 1 hour and 30 minutes, were recorded and transcribed, and data analysis involved content analysis. The competencies identified by participants were categorized in a manner similar to the review study (see Scotton & Neufeld, under revision, for more information).
Preliminary version construction
From the synthesis of information from the theoretical foundation phase and by comparing the instruments, the relevance of each item in the CTS-D-G was analyzed. After this process, this version included 18 items: Definition and adherence to the session agenda; Dealing with problems/questions/obstacles of intervention; Communication style; Pace and efficient use of time; Interpersonal effectiveness; Activation of participant resources; Assessment of action plans; Feedback; Guided discovery; Facilitation of adequate emotional expression; Identification of key cognitions; Identification of behaviors; Conceptual integration; Appropriate application of change techniques and strategies; Adaptation to needs; Proposing the action plan; Management and balance of group roles; and Management of therapeutic factors. Additionally, a final space was added for the supervisor to provide an overall assessment of the therapist.
The rapport was developed based on the Brazilian version of the CTS-R (as the CTS-D-G lacked rapport), with necessary adaptations for group practice. This version of the CTS-R also served as a reference for formulating each item statement, given its comprehensiveness and incorporation of the adherence component (Reis & Barbosa, 2018).
The response scale was also modified. In this version, the practice of defining all scale points was retained, integrating CST-R arguments that doing so enhances reliability and discriminative power. However, the number of points was adjusted. Considering the Dreyfus model of skill acquisition (Dreyfus, 2004), which comprises five stages (novice, advanced beginner, competent, proficient, and expert), the Likert scale points were adapted to align with each level’s description. This resulted in a six-point scale (one point per stage, plus a zero-point indicating absence of competency and adherence).
Judges’ analysis
Regarding the quantitative analysis, only one item had a CVI lower than 0.8 (Item 3 - “Communication Style” with a CVI of 0.71), specifically related to the adequacy of response scale content. It is worth noting that, aside from this item, only two others had CVIs between 0.8 and 0.89 (Items 4 - “Pace and Efficient use of time” and 12 - “Identification of Behaviors”), while all other items had CVIs above 0.95 in all evaluated aspects. Moreover, the total CVI for the instrument was 0.96.
The qualitative analysis of judges’ suggestions resulted in changes of diverse nature. Table 1 presents the different categories of changes made in this instrument version, along with examples of each.
Table 1 Changes in the stage of judge’s analysis.
| Category | Example |
|---|---|
| Adjustments to item wording to remove intersection with other items | Item 2 (dealing with problems/questions/obstacles of the intervention): removal of the expression “welcome and validate participants”, as it overlaps with item 5 (Interpersonal effectiveness) |
| Adjustments to the writing - changing words or expressions for greater understanding by the evaluator | Item 4: “time management is poor” for “the therapist makes no attempt to balance the participation of group members time” |
| Writing adjustments - greater language suitability | Item 5: exchanging “professional boundaries” for “ethical stance” |
| Adjustments to the wording of scale points for greater differentiation between them | Item 6: differentiation between points 4 and 5 - addition of the expression “Minimal problems (for example, misses some opportunities to give feedback)” in point 4 to differentiate from point 5, expert level |
| Adequacy of the description of each point on the scale to correspond with the level of the Dreyfus scale | Item 1: point 0 on the scale had the expression “highly inappropriate agenda”, but point zero on the scale presupposes non-adherence, therefore, this expression would be more coherent at point 1 on the scale, referring to a highly inappropriate performance |
| Adjustment in the wording for more coherence between item description and scale points | Item 3: the item description states that it should be assessed whether the speed of communication is appropriate for the participants' understanding, but the points on the scale did not refer to speed. This was added |
| Theoretical and technical adjustments | Item 9: adjustments to the wording to make it clear that the use of Socratic questioning is one of the techniques for guided discovery, and not the only one |
No new items were suggested by judges, nor were any existing items removed. Item 3, which had a CVI below 0.8, was modified for improved suitability. Importantly, the plausibility of suggestions was thoroughly examined by the authors of the article, forming an expert committee.
Semantic analysis
In this stage, further modifications were made to the instrument to enhance supervisors’ understanding and utilization. The changes implemented based on suggestions are detailed in Table 2.
Table 2 Adjustments based on semantic analysis.
| Category | Example |
|---|---|
| Rapport: adjustments to filling instructions/adding instructions | Changing the instruction: “Please mark an X at the level of competence at which you consider the therapist has achieved the central function” for: “Please enter an X at the level of competence (scale from 0 to 5) at which you consider the therapist has achieved, according to with the wdescription of the item (main characteristics)” |
| Adjustments to the writing for greater suitability in the language (change of expressions, removal of word repetition) | Item 15: changing the expression “microaggression” to “subtle expressions of hostility” |
| Correction of typing errors, punctuation marks, and expressions standardization | Standardize the terms “intermediate beliefs”, as different terms appeared, such as “conditional beliefs” |
| Change in graphic presentation of the item | Separation of the characteristics to be evaluated with “a)”, “b)”, “c)” |
| Incorporation of examples at scale points to facilitate assessment | Item 1 - added the expression “therapist’s lack of empathy regarding participants’ current concerns, resulting in rigidity in the agenda” at point 1 of the scale |
| Adequacy in the instructions of the items to clarify possible doubts at the time of the supervisor's evaluation | Item 5 - readjusted to specify observable behavioral responses from the therapist that may indicate interpersonal effectiveness, as previously there were examples of unobservable responses |
No items were removed during this stage. Participating supervisors provided alteration suggestions through comments within the instrument itself or via audio comments. All of them showed willingness to discuss these suggested changes. Similar to the previous stage, the plausibility of suggestions was assessed by the authors.
DISCUSSION
The present work described the construction of the Group Cognitive-Behavioral Therapy Competence Scale (GCBT-CS), aiming to establish a process with theoretical and methodological rigor, following recommendations from national and international literature (ITC, 2017; Coluci et al., 2015). This study focused on robust theoretical foundations for content validity, employing various resources suggested as useful and appropriate by the literature. Content evaluation is a crucial step in developing new measures, as it marks the initiation of mechanisms to connect abstract concepts with observable and measurable indicators (Alexandre & Coluci, 2011). According to the literature, the formulation of constitutive and operational definitions, based on a specific theoretical framework, has a direct impact on the quality of the test being developed (Andrade & Valentini, 2018).
With the aim of addressing the gap in therapist evaluation within GCBT in the best way, as previously mentioned, a decision was made between constructing a new instrument or adapting an existing one. The stance of different psychometricians on adaptation versus construction of tests is not uniform: while some lean towards construction, others advocate for adaptation. Those in favor of the latter have a strong argument supporting their view - the lack of theoretical development in our culture that could generate sufficient research for constructing a psychological measurement instrument. This is significant because defining the conceptual boundaries of the construct to be measured is the most complex phase in test construction (Fernández et al., 2010). Therefore, the most suitable option for this work would initially be adaptation, considering the notably more robust literature in the US and Europe.
However, due to reasons detailed in the Method section, the decision was made to construct a new instrument. Creating instruments also offers the advantage of considering specific aspects of the Brazilian context, given that construct equivalence and test administration and format can vary across cultures. Additionally, it allows freedom to make item adaptations, develop abbreviated versions, and adapt the instrument for alternative application formats without limitations (Fernández et al., 2010). This need for extensive modifications and adaptations in the CTS-D-G led to the shift towards an instrument construction methodology.
Nonetheless, the foundation of existing scales (the CTS-D-G and the Brazilian version of the CTS-R) remained influential in this instrument’s creation. The argument rested on the fact that these scales are based on widely used instruments for evaluating therapist training programs (the CTS and the CTS-R). These, in turn, were built from important competency guidelines. Furthermore, as per Coluci et al. (2015), using existing instruments is useful as their items have already been tested for psychometric qualities.
Initially intending to translate and adapt the German instrument, the early stages of this process were followed according to literature guidelines (Borsa et al., 2012). These stages encompassed translation by at least two independent bilingual translators, synthesis of translated versions, and expert evaluation of the synthesis. These steps provided clearer insights into the instrument, which were considered in constructing the present scale.
The theoretical foundation relied on several in-depth procedures, aiming to utilize different methods indicated by the literature as useful and essential for the operational definition of the construct. Consulting experts in the field offers the advantage of incorporating the latest knowledge on the subject, while literature searches on national and international databases serve as a primary resource in measurement instrument development research (Alexandre & Coluci, 2011; Coluci et al., 2015). Reviews and consultations with specialists provided immersion into the topic of therapist competencies in CBT and GCBT, exposing the main publications and widely used guidelines in therapist training excellence programs. These endeavors also highlighted the scarcity of studies and tools in Brazil and Latin America. Further publications stemming from this theoretical foundation process are also aimed at beginning to address this gap (Scotton et al., 2021; Scotton & Neufeld, in press; Scotton & Neufeld, under revision). According to Pasquali (2017), employing recognized theories and guidelines is essential for constructing content-valid measurement instruments.
Another important resource and excellent source of items, as per Coluci et al. (2015), is the experience of the target population, with focus groups being a frequently used strategy. The feedback from supervisors in the two conducted focus groups effectively complemented the information acquired in previous steps. Therefore, the GCBT-CS merged deductive (from literature reviews and existing scales) and inductive (from target population feedback) methods in item generation, in accordance with literature recommendations (Selau et al., 2020).
The synthesis of these theoretical foundation steps furnished a theoretical framework that underpinned the creation of new items in this scale, addressing themes not covered by the original scales that were used as a base (the CTS-D-G and the CTS-R). These themes, covered by the items related to flexibility and adaptation to needs (item 15), managing roles of difficult group participants and role balance (item 17), and managing therapeutic factors (item 18), are crucial to GCBT practice. These items covered aspects of the group process, which has been identified by the literature as relatively overlooked in therapist training for GCBT (Rangé et al., 2017).
Adjusting the response scale aimed to better align scores with Dreyfus’ (2004) model of skill acquisition, transitioning from a seven-point Likert scale to a six-point one. A study comparing psychometric properties and normality in Likert scales found no differences between scales of differing points, odd and even, in terms of mean, standard deviation, item-item correlation, total item correlation, reliability, exploratory factor analysis, or factor loading. In this study, it is worth noting that the six-point scale tends to follow a normal distribution, which can be advantageous psychometrically (Leung, 2011).
The inclusion of a qualitative feedback field was based on the assumption that it allows evaluators to utilize the exemplary behaviors provided in the scale and session-specific material to provide examples of strengths, areas for improvement, and strategies for further development. In this manner, it plays a pivotal role in the continuous development of competent and reflective professionals and is well-received by those undergoing assessment. It enables them to assess and reflect on their own performance (Muse et al., 2016).
The selection of judges is crucial to enhance the content validity of the instrument (Pasquali, 2017). The GCBT-CS benefited from the expertise of seven judges specialized in the construct area and experienced in instrument development. Their comprehensive theoretical and practical background was essential in making this a decisive phase for the improvement of the GCBT-CS. Thus, it is considered that there was a combination of theory and practical experiences in the instrument’s construction, which justified further theoretical and technical changes in this stage.
Semantic analysis aimed to ensure the target population’s comprehension when responding to the instrument, in line with literature recommendations. Authors state that comprehension difficulties can directly impact an instrument’s content validity (Coluci et al., 2015; Selau et al., 2020). This stage allowed for significant adjustments to enhance respondent understanding, such as incorporating examples into response scale points, refining item wording, and revising completion instructions to clarify potential uncertainties in responding, as well as adjusting the presentation format. These results underscore the significance of this stage, as it facilitated the enhancement of the instrument’s face validity while also contributing to its content validity.
Within the systematized stages of instrument construction - theoretical, empirical, and analytical (Pasquali, 2017) - this work focused on the theoretical procedure for creating the GCBT-CS. Obtaining validity evidence for instruments is the most important and fundamental parameter in their development and evaluation. However, it is important to note that not all tests available for assessment in Brazil are well-constructed, with some authors indicating that poor theoretical delimitation of constructs remains a significant issue in Psychology (Andrade & Valentini, 2018).
Therefore, to address this gap, the construction of the present scale balanced theoretical assumptions, consultation with recognized instruments and specialists, and input from the target population. It underwent several phases of item refinement. Moreover, it has the advantage of combining quantitative and qualitative methods. Given that instrument construction is intensive and complex (Selau et al., 2020), it is understood that all stages were fundamental for the construction of the GCBT-CS, as they contributed to substantial improvements to the initial version of the instrument. With the work of the judges and the semantic analysis, the theoretical procedures are completed, which encompass the explanation of the theory that underpins the competencies of the therapist in GCBT, as well as the behavioral representation of this construct (Pasquali, 2017).
It is crucial to emphasize that, to become a valid and usable instrument, the current version of the instrument will undergo further stages in the instrument construction process. The next step involves a pilot application, where supervisors will use the instrument to evaluate a GCBT session, either through live observation or video recording. Subsequent studies will then investigate item adequacy, grouping into dimensions, discrimination of therapist competency levels, and validity and reliability evidence through exploratory and confirmatory statistical analyses (Pasquali, 2017).
In conclusion, it is noteworthy that the theme of therapist competencies is still in its infancy in Brazil. While the international context has more substantial studies on this topic, the situation is similar in the context of therapist competencies in GCBT, with a scarcity of publications, available instruments, and systematic incorporation of guidelines into GCBT training programs. This study’s goal was to present an initial version of a scale that aims to contribute to the training process of group therapists, with the intention of anchoring this process in evidence-based practice and ensuring the training and quality of the intervention they provide.
REFERÊNCIAS
Alexandre, N. M. C., & Coluci, M. Z. O. (2011). Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciência & Saúde Coletiva, 16(7), 3061-3068. [ Links ]
Andrade, J. M. D., & Valentini, F. (2018). Guidelines for the construction of psychological tests: Regulation CFP No: 009/2018 in highlight. Psicologia: Ciência e Profissão, 38(SPE), 28-39. [ Links ]
Barletta, J. B., & Neufeld, C. B. (2020). Novos rumos da supervisão clínica em terapia cognitivo-comportamental: Conceitos, modelos e estratégias baseadas em evidências. In Federação Brasileira de Terapias Cognitivas, C. B. Neufeld, E. M. O. Falcone & B. P. Rangé. (Orgs.). PROCOGNITIVA Programa de Atualização em Terapia Cognitivo-Comportamental: Ciclo 7. (pp. 9-49). Artmed Panamericana. (Sistema de Educação Continuada a Distância, v. 3). [ Links ]
Barlow, S. H. (2012). An application of the competency model to groupspecialty practice. Professional Psychology: Research and Practice, 43(5), 442-451. [ Links ]
Blackburn, I. M., James, I. A., Milne, D. L., Baker, C., Standart, S., Garland, A., & Reichelt, F. K. (2001). The revised cognitive therapy scale (CTS-R): Psychometric properties. Behavioural and Cognitive Psychotherapy, 29(4), 431-446. [ Links ]
Borsa, J. C., Damásio, B. F., & Bandeira, D. (2012). Adaptação e validação de instrumentos psicológicos entre culturas: Algumas considerações. Revista Paidéia, 22(53): 423-432. [ Links ]
Coluci, M. Z. O., Alexandre, N. M. C., & Milani, D. (2015). Construção de instrumentos de medida na área da saúde. Ciência & Saúde Coletiva, 20(3), 925-936. [ Links ]
Conselho Nacional de Saúde. Resolução nº 510, de 07 de abril de 2016. https://conselho.saude.gov.br/resolucoes/2016/Reso510.pdf [ Links ]
Dreyfus, S. E. (2004). The five-stage model of adult skill acquisition. Bulletin of Science, Technology & Society, 24(3), 177-181. [ Links ]
Dobson, D., & Dobson, K. S. (2018). Evidence-based practice of cognitivebehavioral therapy. Guilford. [ Links ]
Fernández, A., Pérez, E., Alderete, A. M., Richaud, M. C., & Liporace, M. F. (2010). ¿Construir o adaptar tests psicológicos?: Diferentes respuestas a una cuestión controvertida. Revista Evaluar, 10(1), 60-74. [ Links ]
Goicoechea, J., & Kessler, L. E. (2018). Competency-based training in interpersonal, process-oriented group therapy: An innovative university partnership. Training and Education in Professional Psychology, 12(1), 46-53. [ Links ]
International Test Commission. (2017). ITC guidelines for translating and adapting tests (second edition). International Journal of Testing, 18(2), 101-134. https://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf [ Links ]
Leung, S. O. (2011). A comparison of psychometric properties and normality in 4-, 5-, 6-, and 11-point Likert scales. Journal of Social Service Research, 37(4), 412-421. [ Links ]
Melnik, T., Meyer, S. B., & Sampaio, M. I. C. (2019). Relato de experiência docente: A primeira disciplina no Brasil sobre a prática da psicologia baseada em evidências ministrada no Instituto de Psicologia da Universidade de São Paulo. Psicologia: Teoria e Pesquisa, 35, 1-5. [ Links ]
Moreno, A. L., & DeSousa, D. A. (2020). Adaptação transcultural da cognitive therapy rating scale (escala de avaliação em terapia cognitivo-comportamental) para o contexto brasileiro. Revista Brasileira de Terapias Cognitivas, 16(2), 98-105. [ Links ]
Muse, K., & McManus, F. (2013). A systematic review of methods for assessing competence in cognitive-behavioural therapy. Clinical psychology review, 33(3), 484-499. [ Links ]
Muse, K., & McManus, F. (2016). Expert insight into the assessment of competence in cognitive-behavioural therapy: A qualitative exploration of experts’ experiences, opinions and recommendations. Clinical Psychology & Psychotherapy, 23(3), 246-259. [ Links ]
Muse, K., McManus, F., Rakovshik, S., & Thwaites, R. (2017). Development and psychometric evaluation of the Assessment of Core CBT Skills (ACCS): An observation-based tool for assessing cognitive behavioral therapy competence. Psychological Assessment, 29(5), 542-555. [ Links ]
Neufeld, C. B., Maltoni, J., Ivatiuk, A. L. & Rangé, B. (2017). Aspectos técnicos e o processo em TCCG. In C. B. Neufeld, & B. Rangé (Orgs.), Terapia cognitivo-comportamental em grupos: Das evidências à prática (pp. 33-56). Artmed. [ Links ]
Pasquali, L. (2017). Validade dos testes. Examen: Política, Gestão e Avaliação da Educação, 1(1), 36-36. [ Links ]
Pohl, N., Albus, C., Herpertz, S., & Petrak, F. (2017). Assessment of therapist competence in group therapy: Pilot study to validate the German Cognitive Therapy Scale for Group Therapy (CTS-DG). Zeitschrift für Klinische Psychologie und Psychotherapie, 46(4), 271-283. [ Links ]
Rangé, B., Pavan-Cândido, C., & Neufeld, C. B. (2017). Breve histórico das terapias em grupo e da TCCG. In C. B. Neufeld, & B. Rangé (Orgs.), Terapia cognitivo-comportamental em grupos: Das evidências à prática (pp. 17-32). Artmed. [ Links ]
Reis, G. A., & Barbosa, A. J. G. (2018). Formação de terapeutas cognitivo-comportamentais: Um estudo sobre o estado da arte. Revista Brasileira de Terapia Comportamental e Cognitiva, 20(1), 72-85. [ Links ]
Scotton, I. L., Barletta, J. B., & Neufeld, C. B. (2021). Competências essenciais ao terapeuta cognitivo-comportamental. Psico USF, 26(1), 141-152. [ Links ]
Scotton, I. L., & Neufeld, C. B. (No prelo [a]). Competências do psicoterapeuta de grupos: uma revisão narrativa dentro do movimento da psicologia baseada em evidências. [ Links ]
Scotton, I. L., & Neufeld, C. B. (No prelo [b]). Competências do terapeuta cognitivo-comportamental individual e de grupos: pesquisa qualitativa. [ Links ]
Selau, T., Silva, M. A. D., & Bandeira, D. R. (2020). Construção e evidências de validade de conteúdo da Escala de Funcionamento Adaptativo para Deficiência Intelectual (EFA-DI). Avaliação Psicológica, 19(3), 333-341. [ Links ]
Young, J. E., & Beck, A. T. (1980). Cognitive therapy scale: Rating manual. unpublished manuscript, center for cognitive therapy. https://cdn.ymaws.com/www.academyofct.org/resource/collection/24743CF7351E-4335-9E93-83F26EF675A3/CTRS_Manual.pdf. [ Links ]
Received: August 17, 2022; Accepted: August 12, 2023
Este é um artigo publicado em acesso aberto (Open Access) sob a licença Creative Commons Attribution, que permite uso, distribuição e reprodução em qualquer meio, sem restrições desde que o trabalho original seja corretamente citado.
