REVIEW ARTICLE

 

Quatitative systematic review of mental health services from the users' perspective^*

 

 

Éllen Cristina Ricci^I,II; Mariana Barbosa Pereira^I,II; Leidy Janeth Erazo^I,II; Rosana Teresa Onocko-Campos^I; Erotildes Maria Leal^III

^IUniversidade Estadual de Campinas, Faculdade de Ciências Médicas, Campinas, SP, Brazil
^IIBolsista da Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES), Brazil
^IIIUniversidade Federal do Rio de Janeiro, Faculdade de Medicina, Rio de Janeiro, RJ, Brazil

Corresponding Author

 

 

---

ABSTRACT

OBJECTIVE: to identify, based on qualitative systematic literature review, Brazilian national and international publications in mental health area, addressing service's evaluation on their users perspective.
METHOD: a qualitative systematic literature review, which followed PRISMA orientations and criteria. For that, two international platforms that include a significant number of databases were used, with “mental health” and “evaluation” descriptors and 2004 to 2016 period.
RESULTS: 4,735 full articles were initially collected, with final eligibility of 137 for integral text reading, of which 44 were included in the final analysis. Research studies on mental health services evaluation with users' participation are present in Brazil, but still in reduced number when compared to countries like England, Australia and USA.
CONCLUSION: in Brazil, as well as in other countries, the ways by which users participate in services evaluations are still a challenge. Mental health service users' protagonist is insufficient. In Brazil, there is also a challenge to structure public services systematic evaluations, incorporating diversity of evaluative methods and stakeholders' participation.

Descriptors: Health Evaluation; Mental Health; Systematic Review; Patient Participation.

---

 

 

Introduction

Psychiatric reform projects in different countries emerged in the context of criticism of asylum institutions after World War II. These were criticized for their low therapeutic efficacy, high maintenance costs, the violence with which they treat people and the social exclusion they generate^(1-2).

This process resulted in initiatives aimed at the humanization of psychiatric hospitals, reflections on the quality of treatment and the creation of mental health services in the community⁽³⁾. In Brazil, this process is called the Brazilian Psychiatric Reform, which encompasses and articulates the technical assistance, political, legal, conceptual and socio-cultural fields⁽⁴⁾.

However, despite almost 30 years of implementing community mental health services in Brazil and 17 years of Law 10,216, which provides for the rights and protection of people with mental disorders, which systematically systematized, organized and implemented services^(5-6), the country still lacks service evaluation indicators⁽⁷⁾.

There is a shortage of Brazilian studies with mixed methods that describe evaluation and results processes, which contemplate the Psychosocial Care Network (Rede de Atenção Psicossocial, RAPS) mainly from the users' perspective⁽⁸⁾. In Brazil, the field of evaluation is incipient in general and mental health has particularities that make consensus around evaluative parameters a constant methodological challenge^(7-8).

The World Health Organization^(9-10) has produced technical reports and analyzes to support common objectives for sectorial mental health policies, seeking to establish consensus and create indicators. There is a worldwide trend towards the users' incorporation, based on their experience of falling ill and the relationships with the treatment spaces, in the evaluation processes of the services and assistance they have provided.

We understand that the users' participation in evaluations about health services is fundamental for obtaining results that represent them, in addition to influencing public policies that are effective and functional for the group of people served, valuing the knowledge that comes from the lived experience, thinking about ways and strategies to enhance the dialogue⁽¹¹⁾.

Therefore, the current challenge is to evaluate the network of territorial services and the treatment orientation, which values the experience of people in their daily lives and their recovery processes, as the challenge of the new services is to bring the user to the center of the scene of their treatment^(12-14).

Thus, the results found by two studies^(7-8) suggest that in Brazil, there are few studies that propose to evaluate mental health services from the perspective and with the participation of its users, when compared to the international scenario^(7-8).

Our interest at this point was to reveal how this phenomenon behaves both on the national and international scene through a qualitative systematic review on the users' participation of mental health services in their evaluation and we raised the following questions:

1. What is the national and international scenario for evaluating mental health services with the participation of the users of the system? and;

2. What is the form of user participation during the process of evaluating mental health services?

The objective of this work was to identify Brazilian and international national publications in the area of mental health, addressing the evaluation of services, highlighting the forms of user participation.

 

Method

To achieve the objective proposed in this work, we carried out a systematic qualitative review following the organizational guidelines of PRISMA reviews^(15-16). The qualitative systematic bibliographic review provides for a synthesis of studies containing objectives, materials and methods that are clearly explained and conducted according to an explicit, reproducible and rigorous method for identifying texts, making a critical assessment and synthesizing relevant studies⁽¹⁷⁾.

We use two international platforms that include a significant number of databases in order to get closer to the largest number of productions in the area.

The first platform to be used was the Virtual Health Library - VHL. The choice of this source is due to its wide and consolidated use in the field of health, especially in Brazil and in most Latin American countries. The keywords used were the following: EVALUATION and MENTAL HEALTH, and the filters: full texts available, country of affiliation, Brazil, and period, from 2004 to 2016.

For the international review, the platform used was Web of Science, due to its interdisciplinary profile, to its breadth of indexing scope and for being endorsed by Unicamp, the university where this review took place. The following keywords were used: EVALUATION and MENTAL HEALTH. Using the following filters: full text available, English, Portuguese and Spanish languages, and year of publication from 2004 to 2016.

We formulate the following text inclusion criteria: evaluation of a service or specialized program in mental health made by users, from different aspects such as satisfaction with the service, narratives of experiences for evaluation purposes, participation in the construction of the evaluation process as an interviewer and methods of analysis.

The text exclusion criteria were: services that were not exclusive to mental health, children's mental health services, specific interventions that did not evaluate the service as a whole, validation of instruments, users did not participate in the evaluation, and other interest groups that did not include users.

After the first stage of this review, we set up the data analysis team, composed of three researchers in the field of mental health and public health. This is necessary during the process of selection, exclusion and analysis of the content of the selected texts, to avoid analysis bias, since the selected data underwent double evaluation between different members of the team to be eligible or excluded. If there was disagreement between two members regarding the inclusion or not of the text, a third broker would check and discuss with the team until reaching a consensus^(16,18-19).

From the platforms we found a total of 4,735 complete articles. It can immediately be seen in the analysis in relation to the title, that the reviewers selected 394 articles out of the 4,735. This was mainly because the titles already described the exclusion criteria listed earlier because they did not contribute to the objective of this study. Most articles were about medication evaluation processes, some specific practices and quality of life after brief interventions, without characterizing service evaluation. We also found a large number of surveys that heard only part of the interested parties, mostly service workers.

The 394 selected texts were systematized based on the following information: name of the evaluator, inclusion or exclusion evaluation, article title, year, keywords, authors, methods used in relation to user participation, type of service evaluated, country where the research was carried out, techniques used, found results, and observations relevant and electronic address of the text.

Then reading the abstracts reduced our sample to 137 articles. For this reason, it was possible to identify articles that only described and evaluated part of the good practices incorporated into the services and treatments provided. It was at this stage that most Brazilian and United States articles were excluded, as they did not address the objectives of this review.

Keeping the methodology, two evaluators analyzed the 137 articles in full and issued their judgment on whether or not to be included in the final analysis process. At this stage, there were no differences in most articles, with only 3 of them needing the third assessment. Thereby, we come to the 44 articles that clarify the objectives of this article.

Figure 1 illustrates the steps and numerical findings of the literature review in a flowchart format:

 

 

We know that in the Brazilian context the users' participation as collaborators and leaders in evaluation and research is incipient, and internationally only a few countries have adopted such practices^(11,20). Therefore, from the studied literature, we built categories to analyze the methods used in relation to user participation, considering four possibilities subdivided as follows^(11,20):

A1: The user's perspective is taken in 'third person'; users are subjects of research and the methods applied are mainly questionnaires and other closed instruments;

A2: The user's perspective is taken in 'first person'; interviews and or narratives of user experiences with services;

B1: User perspective with their participation as an interviewer and/or evaluator in part of the process together with other non-user evaluators;

B2: User perspective with their participation in all phases of the evaluation, in the construction of the method and question to be evaluated that may or may not count on other actors.

Therefore, the articles selected for analysis were tabulated according to the categories described above. Throughout the course we use the Excel^® for organizations and tabulation of the findings and synthesis of the sample and the Mendeley^® it was used as a support and organization tool for the final material.

 

Results and Discussion

Table 1 shows the distributions of the 44 articles selected for the final analysis by year of publication, country of study, type of service and method of user participation.

 

 

It is noticed that there is no homogeneous distribution and so little growing over the years, since in 2009 we have the largest number of articles found, followed by the years 2006 and 2008, with 21 articles produced in these three periods.

In relation to the countries that have the highest production of articles on service evaluation that incorporate users in the process, we highlight England, Australia, United States followed by Brazil and Canada.

It is worth mentioning that England, Australia and Canada are countries with an evaluative culture of services, with national public health systems, which go beyond epidemiological data and public management reports, thus dedicating part of the evaluation studies from the perspective of the users/clients of the system^(11,14). Satisfaction surveys on access, treatment and conditions of services are common, using participatory methods and tools that include the different actors involved.

The United States, on the other hand, has a tradition of evaluating state-funded programs, since a good part of mental health depends on this type of financing to serve the population and there is no single national system^(11,14). We excluded several articles that were limited to just one practice, evaluation of drug responses, or to a specific audience identified by their diagnosis receiving timely treatment.

Brazil appears with six articles, which, compared to the countries previously mentioned, seems to be a good result, since the production of general evaluation of services stands out, considering the importance of the users in this process. The evaluation of services in Brazil is still a major challenge^(14,21). We know that there is a great effort by Brazilian researchers to create instruments, techniques, indicators and guides to good practices for the global evaluation of services with the participation of the users and other actors, but the practical results of systematic evaluations are not yet in the form of indexed articles on the studied bases.

As we expected, there is a great investment in understanding how users and other actors evaluate specialized services (25 of the 44 articles found), to the detriment of hospitals (psychiatric, university and nursing hospitals), considering the international movement, which includes Brazil, from reformulation of the mental health care model.

Specialized services are related to the treatment of people with severe mental disorder, alcohol and other drugs, housing or specific population (e. g., war veterans in the United States have an exclusive network). In Primary Care, they incorporate prevention and call center work. It is worth noting that we found two articles that evaluated two or more types of services concurrently (service network) and included users.

Most of the findings were concentrated in categories A1 (18 articles) and A2 (16 articles) for evaluation. The articles in A1 were characterized by interviews, questionnaires that sought the most objective assessment of the services, considering little or nothing of the users' subjective aspects. Those who use already validated instruments are more expressive in international experiences.

A2 brought more open opinion interviews, experiences and evaluations that could highlight their experience and how they evaluated the process. With regard to the methodological approach by which service users participate as research subjects (B1), it seems that research has invested more in the narrative experiences of these people to evaluate services, with users interviewing users.

It is worth mentioning that we did not find any article that approached B2, and four articles used more than one method for evaluations.

After this descriptive presentation of the quantitative and bibliometric data, we highlight the qualitative analysis of the most recent articles with methods A1, A2, B1 and those that use more than one method for the participation and users' evaluation.

United States and England had 4 articles each selected in A1. We highlight a study⁽²²⁾ evaluation of mental health programs aimed at war veterans in the USA, using a quantitative cohort method, using telephone interviews as tools, with internationally validated instruments, with emphasis on the RSA (recovery self-assessment). The findings indicate that people with comorbidities evaluated the care and treatment programs better than people with less complex conditions. However, with regard to person-centered practices and personal recovery, the program was rated less by all participants in all age groups, thus highlighting the challenge of these programs to develop improvements in these aspects to ensure quality of life of war veterans⁽²²⁾.

We found three Brazilian articles in A1 category, in the years 2006, 2009 and 2012. These are studies that have worked only with users, with instruments or closed interviews, using quantitative analysis methods, two evaluating Psychosocial Care Center (Centro de Atenção Psicossocial, CAPS) and an outpatient clinic. In general, users were satisfied with the treatment and the new services provided^(23-25).

All the articles that we included in the A1 model started from exclusively quantitative methodologies and with corresponding analyzes through data measurement programs^(26-28). What varied was the following: ways for collecting data: multiple choice interviews or application of instruments with Likert-type scales^(22,29-32); the sample size and the number of participants (with more than 300 respondents, including workers, users and family members)^(27,33-35); the data collection technique, such as telephone interviews, self-administered instruments sent by letters, e-mails or arranged at the reception desk^{(22,26,30,36)}.

This format of evaluative service research is able to capture and measure a larger number of data, generating a more general analysis of how the services have been providing care and how the actors involved evaluate the treatment and the consequences of this work. These assessments refer to measurements called 1^st generation evaluative⁽³⁷⁾, in which the evaluator is a technician who must know how to build and use the instruments, and that any research/evaluation variable must be measured statistically⁽³⁷⁾.

In the A2 category we highlight a Swedish article⁽³⁸⁾ which studied only users with severe and persistent disorders, based on interviews about housing and living, what we usually call in Brazil as Therapeutic Home Services (THSs). The phenomenological-based study sought individual and collective consensus on the narratives produced, building a value judgment on living in THSs. It was possible to highlight positive points such as rest, security and privacy space, with support from other residents and staff; but also negative points like dependency, passivity and oppression by residents and staff. The study concludes jointly with the users that in order to have a less institutionalized housing, support must be directed towards the rehabilitation and recovery of all⁽³⁸⁾.

The other three Brazilian articles are in the A2 category^(39-41). All evaluated the CAPS, from users and other actors, using focus groups, open interviews and one of them used SATIS - BR scale⁽⁴⁰⁾. Data analysis was performed using quantitative and qualitative methods, using the hermeneutic cycle in the process of collecting and interpreting data^(39,41).

In the three articles, the CAPS were well evaluated in relation to the therapeutic practices employed, the reception of the crisis and the free treatment. Negative points are highlighted by the lack of inputs, measurements, precarious building infrastructure and difficulty in integrating with other points in the social assistance network. They indicate the permanent evaluation of services, as well as the importance of promoting citizenship, social and family participation processes as a support and citizenship network, in guaranteeing the users' rights^(39-41).

We highlight the work⁽⁴⁰⁾ that evaluated a CAPS-type service in Rio Grande do Sul, using a qualitative method, with focus group techniques and hermeneutic analysis in the construction of consensus. Workers, users and family members participated, who were able to discuss positive points and challenges for the services, which touch the contractual power of the subjects in their daily treatment.

In the A2 category, most articles worked using qualitative methods with techniques, such as: semi-structured interviews, narratives and focus groups⁽⁴²⁾. Only two articles were based on mixed methods^(43-44). It is worth mentioning that during the analysis of the data, the authors reinforce their theoretical frameworks for this, with emphasis on phenomenology and historical materialism, which we do not find in articles classified as A1.

The A2 category can fit the 2^nd and 3^rd generation assessment⁽³⁷⁾: the description (2^nd), in which the focus is process, and not only in the measurement of results; the judgment (3^rd), in which the appraiser also judges and issues opinions. Clearly, the articles sought the experience of people with the services and treatments received and how the different actors proposed changes in the process, but the analyzes still translate into narratives interpreted by the researchers⁽³⁷⁾.

In the B1 category, the four Australian articles^(45-48) caught our attention^(45-48) showing other forms of service evaluation that encourage greater participation by the involved actors.

One of the works⁽⁴⁵⁾ studied the users' perception on the services provided at different points in the Australian mental health network, with students and research users as interviewers. The interviews were semi-structured with a focus on discussing users' perception on the services. In general, users highlighted the lack of information during access to services, especially in hospitals, which made empowerment in the face of the situation being difficult. Still in relation to the hospital, they highlighted the negative effects of medications and the little dialogue with the doctors. The article also highlighted that health service evaluations should have the users' point of view at the center of the discussion, in order to contribute to more user-oriented practices and their recovery⁽⁴⁵⁾.

Another important difference was in the way users were involved in the research (interviewing researchers), which facilitated access to other users surveyed, with deeper reports during the interviews than those made by the students, indicating the relevance of this approach for assessing services. This research group argues that users can and should participate in other aspects of the research, seeking the depth of the evaluation process, generating data more reliable to the realities experienced, including questions that are really relevant for the users^(45-46).

The other two works^(47-48) discuss the evaluation process of primary care services in mental health in Australia, using qualitative methods with research users in focus groups, interviews and building consensual narratives. The strengths of the results indicated for the evaluation of treatment received and supportive therapies and social connection. In relation to social connection, the importance of respect and support from peers was discussed as service offers for the recovery of the people. For some users, peer support was considered the most important and fundamental for their recovery and good evaluation of the services that indicated^(47-48). They also highlighted as important resources for the recovery and positive evaluation of services, drug treatment, plans and crisis management and spiritual therapies^(47-48).

Users discussed and brought negative ratings that deserve to be highlighted: the lack of employees and geographical barriers in Australian rural areas, unable to access services, with a delay of up to 4 days; lack of confidence in the teams and lack of systematization in the practice of peer support in these regions^(47-48). In large centers, the complaint was the feeling of being treated as numbers and symptoms and not as natural persons. They also highlighted the importance of expanding users' participation in the evaluation processes, from development to the analysis of more effective strategies for improving the services^(47-48).

The B1 category is perhaps the closest to 4^th generation evaluations⁽³⁷⁾ through the methodological assumptions of the constructivist paradigm, in which the claims, concerns and questions of the interest groups serve as the organizational focus of the research and the information necessary for the analyzes. Thus, the actors or interest groups are involved or potentially affected by the service and by the eventual consequences of the evaluation process⁽³⁷⁾.

It is important to emphasize that users when participating in research tend to propose relevant questions, identify flaws in the research, are good indicators of the validity of certain protocols and instruments, offer interesting interpretations and follow the results⁽⁴⁹⁾. In Brazil, the literature on users' participation in academic research is limited, with experiences of inserting users in research being scarce and recent⁽⁵⁰⁾.

Countries like Australia and New Zealand have incorporated the participation of the users of Mental Health in research as their right. Research with more horizontal relationships between academic researchers and communities enables the professional development of academics, they tend to have greater credibility and user involvement in the adhesion of other peers to the research^(51-52).

In relation to articles on evaluations that used various methods and ways of inserting users in the process, we highlight a study carried out in England⁽⁵³⁾, which used quantitative methods and statistical analysis, as well as qualitative, with interviews, which were carried out by university students and users of the system, who had undergone hospitalization experiences. The challenges raised in the hospitals' evaluation process were the following: duties and dispositions of employees, ambience and communication, mainly related to rules and less to therapeutic processes, such as welcoming listening by the professionals. The users in the present study attached great importance to justice, being heard, having activities and being treated with respect⁽⁵³⁾.

Articles with multiple methods, with different forms of participation by users and other actors appear in more recent literature, starting in 2010, in countries with a strong tradition in systems assessment (England and Canada)^(53-55) and references in the reforms of the mental health care model (Italy)⁽⁵⁶⁾.

This presents itself to us as a major advance in evaluative culture, seeking to bridge the gap between quantitative Cartesian models and phenomenological, materialistic, historical and critical models. We consider it essential to use multiple methods, with the participation of as many stakeholders as possible, in different ways, for an evaluation of services that is truly committed to the affirmation of successful practices, as well as to the transformation of projects that do not serve the public at issue.

In the international articles, there is a greater number and investment in evaluation processes, even so the users' voice appears in lesser number in relation to workers and managers. The vast majority of articles evaluated only some conditions or practices of the services and the satisfaction of receiving that specific attention. Global assessments considering the service in its territorial and network complexity, with multiple offers and with diversification of methods and actors for the production of more consistent results are restricted. We highlight England, Australia, Canada, USA and Brazil with productions aimed at more global evaluations.

We know that there are other articles on the evaluation of mental health services in Brazil, however our finding corroborates with other reviews on the subject with different objectives^(8,57-58), in the sense that few studies give voice to users and, when they do, they only evaluate some practices and not the service as a whole.

Limitations

We expected to find more international articles valuing global assessments with all the involved actors. The fact that our analysis only contemplates productions that describe service evaluation and not specific practices, as well as the keywords that the authors used in the articles did not include those used by us in the search (mental health and evaluation), may have influenced in our survey. Although we believe that the descriptors were quite comprehensive, they may not have captured any relevant article both nationally and internationally.

 

Conclusion

This article presented a qualitative systematic review of the literature on the evaluation of mental health services, highlighting those that incorporated user participation, identifying Brazilian and international national publications in the mental health area.

The quantitative data found showed us that Brazil has research on the evaluation of mental health services, but still in smaller numbers, in comparison with other countries like England, Australia and USA.

According to this review, the methodological approach of incorporating the user's perspective, that is, the way users are involved in studies, both in Brazil and in other countries, is still a challenge. There is little investment and efforts in building shared and effective research with the users' role.

We concluded from the results and analysis of our review on the importance of advancing in more robust assessments of mental health services, which highlight the voice and the users' participation, as they are still fewer in the world; and in Brazil, there is also the challenge of structuring systematic evaluations of public services, with a multiplicity of evaluation methods and involving all actors.

We considered that systematic review studies are always important tools for reflection to support research, illuminate phenomena and demonstrate how other researchers have studied the themes.

 

References

1. Amarante P. Loucos pela vida: a trajetória da reforma psiquiátrica no Brasil. 3ª ed. Rio de Janeiro (RJ): Fiocruz; 1998. 136 p.         [ Links ]

2. Barros DD. Jardins de Abel: Desconstrução do Manicômio de Trieste. São Paulo (SP): Lemos; 1994. 158 p.         [ Links ]

3. Mângia EF, Nicácio F. Terapia ocupacional em saúde mental: tendências principais e desafios contemporâneos. In: De Carlo MMRP, Bartalotti CC. organizadores. Terapia Ocupacional no Brasil: fundamentos e perspectivas. São Paulo (SP): Plexus; 2001. p. 63-80.         [ Links ]

4. Lancetti A, Amarante P. Saúde mental e saúde coletiva. In: Campos GWS, Minayo MC, Akerman M, Drumond M, Carvalho YM, organizadores. Tratado de Saúde Coletiva. São Paulo (SP): Hucitec; 2006. p. 615-34.         [ Links ]

5. Ministério da Saúde. Reforma Psiquiátrica e política de Saúde Mental no Brasil [Internet]. Brasília (DF): Ministério da Saúde; 2005. [Acesso 16 ago 2018]. Disponível em: http://bvsms.saude.gov.br/bvs/publicacoes/Relatorio15_anos_Caracas.pdf        [ Links ]

6. Lei Federal Nº. 10216, de 6 de abril de 2001 (BR). Proteção e os direitos das pessoas portadoras de transtornos mentais e redireciona o modelo assistencial em saúde mental. [Internet]. Brasília (DF): Ministério da Saúde; 2001. [Acesso 16 ago 2018]. Disponível em: http://www.planalto.gov.br/ccivil_03/leis/leis_2001/l10216.htm        [ Links ]

7. Trapé TL. Rede de atenção à saúde mental: Estudo comparado Brasil - Catalunha [Internet]. [tese]. Campinas (SP): Universidade Estadual de Campinas; 2015. [Acesso 16 ago 2018]. Disponível em: http://repositorio.unicamp.br/handle/REPOSIP/313041        [ Links ]

8. Dantas C de R, Oda AMGR. Cartografia das pesquisas avaliativas de serviços de saúde mental no Brasil (2004-2013). Physis. [Internet]. 2014; 24(4):1127-79. [Acesso 23 Jan 2018];Disponívelem:http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0103-73312014000401127&lng=pt&tlng=pt        [ Links ]

9. WHO. Mental Health - Action Plan 2013-2020 [Internet]. Geneva (SW); World Health Organization; 2013. [Acesso Jan 10 2018]. Available at: http://apps.who.int/iris/bitstream/10665/89966/1/9789241506021_eng.pdf?ua=1        [ Links ]

10. WHO. Atlas Mental Health [Internet]. Geneva (SW); World Health Organization; 2014. [cited Jan 10 2018]. Available at: http://www.who.int/mental_health/        [ Links ]

11. Dimov T, Ricci EC. A pesquisa acadêmica como atividade humana: participação de usuários da saúde mental e as contribuições da terapia ocupacional. Cad Ter Ocup UFSCar. [Internet]. 2016;24(3):651-8. [Acesso 23 jan 2018]. Disponível em: http://doi.editoracubo.com.br/10.4322/0104-4931.ctoEN0674        [ Links ]

12. Onocko Campos R. Clínica: a palavra negada - sobre as práticas clínicas nos serviços substitutivos de Saúde Mental. Saúde em Debate. [Internet]. 2000;25(58):98-111. [Acesso 14 fev 2018]. Disponível em: http://www.ensp.fiocruz.br/parcerias/cebes/cebes.html        [ Links ]

13. Scriven M. Metodologia da Avaliação. In: Instituto Fonte, organizador. Introdução à avaliação de programas sociais. São Paulo (SP): Instituto Fonte; 2003. p. 8-33.         [ Links ]

14. Furtado JP. Avaliação de Programas e Serviços. In: Campos GWS, Minayo MC, Akerman M, Drumond, M, Carvalho, YM, organizadores. Tratado de saúde coletiva. 2º ed São Paulo (SP): Hucitec; 2009. p. 715-39.         [ Links ]

15. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. [Internet]. 2009;6(7). [cited Nov 1 2017]. Available from: https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1000100        [ Links ]

16. Moher D, Liberati A, Tetzlaff J, Altman DG, Group TP. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. [Internet]. 2009;6(7). [cited Nov 1 2017]. Available from: http://dx.plos.org/10.1371/journal.pmed.1000097        [ Links ]

17. Lopes ALM, Fracolli LA. Revisão sistemática de literatura e metassíntese qualitativa: considerações sobre sua aplicação na pesquisa em enfermagem. Texto Contexto Enferm. [Internet]. 2008;17(4):771-8. [Acesso 21 nov 2017]. Disponível em: http://www.redalyc.org/articulo.oa?id=71411240019        [ Links ]

18. Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH, Cameron C, et al The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: Checklist and explanations. Ann Intern Med. [Internet]. 2015;162(11):777-84. [cited Nov 21 2017]. Available from: http://annals.org/aim/fullarticle/2299856        [ Links ]

19. Moher D, Liberati A, Tetzlaff J, Altman D. Principais itens para relatar Revisões sistemáticas e Meta-análises: A recomendação PRISMA*. Epidemiol Serv Saude. [Internet]. 2015;24(2):335-42. [Acesso 16 ago 2017]. Disponível em: www.prisma-statement.org.         [ Links ]

20. Rose D. Having a diagnosis is a qualification for the job. BMJ. [Internet]. 2003;326(7402):1331. [cited Feb 4 2018]. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12805192        [ Links ]

21. Campos GW de S. Produção de Conhecimento, avaliação de políticas públicas em saúde mental:notas reflexivas. In: Onocko-Campos R, Furtado JP, Passos E, Benevides R, organizadores. Pesquisa Avaliativa em Saúde Mental - desenho participativo e efeitos da narratividade. Hucite: São Paulo (SP): Aderaldo & Rothschild Editores; 2008. p. 97-102.         [ Links ]

22. Blonigen DM, Bui L, Harris AHS, Hepner KA, Kivlahan DR. Perceptions of behavioral health care among veterans with substance use disorders: Results from a national evaluation of mental health services in the Veterans Health Administration. J Subst Abuse Treat. [Internet]. 2014;47(2):122-9. [cited Aug 31 2018]. Available from: https://www.sciencedirect.com/science/article/pii/S0740547214000725        [ Links ]

23. Silva MA da, Bandeira M, Scalon JD, Quaglia MAC. Satisfação dos pacientes com os serviços de saúde mental: a percepção de mudanças como preditora. J Bras Psiquiatr. [Internet]. 2012;61(2):64-71. [Acesso 4 set 2018]. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0047-20852012000200002&lng=pt&tlng=pt        [ Links ]

24. Heckert U, Teixeira LS, Trindade A de S. Avaliação da satisfação dos usuários do Centro Regional de Referência em Saúde Mental (CRRESAM) da região central de Juiz de Fora, MG. HU Rev. [Internet]. 2006;32(1):15-20. [Acesso 4 set 2018]. Disponível em: https://hurevista.ufjf.emnuvens.com.br/hurevista/article/view/6/3        [ Links ]

25. Tomasi E, Facchini LA, Piccini RX, Thumé E, Silva RA da, Gonçalves H, et al. Efetividade dos centros de atenção psicossocial no cuidado a portadores de sofrimento psíquico em cidade de porte médio do Sul do Brasil: uma análise estratificada. Cad Saúde Pública. [Internet]. 2010;26(4):807-15. [Acesso 4 set 2018]. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-311X2010000400022&lng=pt&tlng=pt        [ Links ]

26. Shamir D, Szor H, Melamed Y. Dropout, early termination and detachment from a public psychiatric clinic. Psychiatr Danub. [Internet]. 2010;22(1):46-50. [cited Aug 31 2018] Available from: https://hrcak.srce.hr/index.php?show=clanak&id_clanak_jezik=74997        [ Links ]

27. Whittington R, Bowers L, Nolan P, Simpson A, Neil L. Approval Ratings of Inpatient Coercive Interventions in a National Sample of Mental Health Service Users and Staff in England. Psychiatr Serv [Internet]. 2009;60(6):792-8. [cited Aug 31 2018]. Available from: http://psychiatryonline.org/doi/abs/10.1176/ps.2009.60.6.792        [ Links ]

28. Remnik Y, Melamed Y, Swartz M, Elizur A, Barak Y. Patients' Satisfaction with Psychiatric Inpatient Care. Isr J Psychiatry Relat Sci. [Internet]. 2004;41(3):208-12. [cited Aug 31 2018]. Available from: https://search.proquest.com/docview/236928012/fulltextPDF/C6E7A788E3054D36PQ/1?accountid=8113        [ Links ]

29. Ruggeri M, Salvi G, Perwanger V, Phelan M, Pellegrini N, Parabiaghi A. Satisfaction with community and hospital-based emergency services amongst severely mentally ill service users. Soc Psychiatry Psychiatr Epidemiol. [Internet]. 2006;41(4):302-9. [cited Aug 31 2018]. Available at: http://link.springer.com/10.1007/s00127-006-0030-x        [ Links ]

30. Mavrogiorgou P, Siebers F, Juckel G, Kienast T. Patient satisfaction with specialized mental health service for obsessive-compulsive disorder. Ann Gen Psychiatry. [Internet]. 2013;12(1):41. [cited Aug 31 2018]. Available from: http://annals-general-psychiatry.biomedcentral.com/articles/10.1186/1744-859X-12-41        [ Links ]

31. Lundqvist L-O, Ahlström G, Wilde-Larsson B, Schröder A. The patient's view of quality in psychiatric outpatient care. J Psychiatr Ment Health Nurs. [Internet]. 2012;19(7):629-37 [cited Aug 31 2018]. Available from: http://doi.wiley.com/10.1111/j.1365-2850.2012.01899.x        [ Links ]

32. Tierney KR, Kane CF. Promoting Wellness and Recovery for Persons With Serious Mental Illness: A Program Evaluation. Arch Psychiatr Nurs. [Internet]. 2011;25(2):77-89. [cited Aug 31 2018]. Available from: https://www.sciencedirect.com/science/article/pii/S0883941710000853        [ Links ]

33. Day CA, Demirkol A, Tynan M, Curry K, Hines S, Lintzeris N, et al. Individual versus team-based case-management for clients of opioid treatment services: An initial evaluation of what clients prefer. Drug Alcohol Rev. [Internet]. 2012;31(4):499-506. [cited Aug 31 2018]. Available from: http://doi.wiley.com/10.1111/j.1465-3362.2011.00347.x        [ Links ]

34. Gould MS, Munfakh JLH, Kleinman M, Lake AM. National Suicide Prevention Lifeline: Enhancing Mental Health Care for Suicidal Individuals and Other People in Crisis. Suicide Life Threat Behav. [Internet]. 2012;42(1):22-35 [cited Aug 31 2018]. Available from: http://doi.wiley.com/10.1111/j.1943-278X.2011.00068.x        [ Links ]

35. Solomon P, Stanhope V. Recovery: Expanding the Vision of Evidence-Based Practice. Br Treat Cris Interv. [Internet]. 2004;4:311-21 [cited Jan 17 2018]. Available from:https://www.researchgate.net/profile/Victoria_Stanhope/publication/31362986_Recovery_Expanding_the_Vision_of_Evidence-Based_Practice/links/02e7e525ed47e049f9000000.pdf        [ Links ]

36. Hill S, Turner N, Barry S, O'Callaghan E. Client satisfaction among outpatients attending an Irish community mental health service. Ir J Psychol Med. [Internet]. 2009;26(03):127-30. [cited Aug 31 2018]. Available from: https://www.cambridge.org/core/product/identifier/S0790966700000422/type/journal_article        [ Links ]

37. Guba EG, Lincoln YS. Fourth generation evaluation [Internet]. SAGE Publications, organizador. United States of America (USA): Sage Publications; 1989. 294 p. [cited Sep 1 2018]. Available from: https://books.google.com.br/books?hl=en&lr=&id=k_zxEUst46UC&oi=fnd&pg=PA7&dq=GUBA,+E.%3B+LINCOLN,+Y.+Effective+evalution.&ots=__5avnhTEZ&sig=XTFwtZujRrD5xn9QfbijR9jIbWE&re
dir_esc=y#v=onepage&q=GUBA%2CE.%3B LINCOLN%2C Y.Effective evalution.&f=f        [ Links ]

38. Bengtsson-Tops A, Ericsson U, Ehliasson K. Living in supportive housing for people with serious mental illness: A paradoxical everyday life. Int J Ment Health Nurs. [Internet]. 2014;23(5):409-18. [cited Aug 31 2018]. Available from: http://doi.wiley.com/10.1111/inm.12072        [ Links ]

39. Schneider JF, Wagner M, Prado L. Avaliação de um centro de atenção psicossocial brasileiro. Cienc y Enferm. [Internet]. 2009;15(3):91-100 [citado 04 Sep 2018]. Disponível em: https://scielo.conicyt.cl/pdf/cienf/v15n3/art_10.pdf        [ Links ]

40. Kantorski LP, Jardim V da R, Wetzel C, Olschowsky A, Schneider JF, Heck RM, et al. Satisfação dos usuários dos centros de atenção psicossocial da região Sul do Brasil. Rev Saúde Pública. [Internet]. 2009;43(1):29-35. [Acesso 4 set 2018]. Disponível em: https://www.scielosp.org/pdf/rsp/2009.v43suppl1/29-35/pt        [ Links ]

41. Surjus L, Onocko-Campos R. A avaliação dos usuários sobre os Centros de Atenção Psicossocial (CAPS) de Campinas, SP. Rev Latinoam Psicopatol Fundam. [Internet]. 2011;14(1):122-33. [Acesso 4 set 2018]. Disponível em: http://www.redalyc.org/html/2330/233018495009/        [ Links ]

42. Restall G, Strutt C. Participation in planning and evaluating mental health services: Building capacity. Psychiatr Rehabil J. [Internet]. 2008;31(3):234-8. [cited Aug 31 2018]. Available from: http://doi.apa.org/getdoi.cfm?doi=10.2975/31.3.2008.234.238        [ Links ]

43. Rogers A, Oliver D, Bower P, Lovell K, Richards D. Peoples' understandings of a primary care-based mental health self-help clinic. Patient Educ Couns. [Internet]. 2004;53(1):41-6. [cited Aug 31 2018]. Available from: https://www.sciencedirect.com/science/article/pii/S0738399103001149        [ Links ]

44. Hodge S, Barr W, Göpfert M, Hellin K, Horne A, Kirkcaldy A. Qualitative findings from a mixed methods evaluation of once-weekly therapeutic community day services for people with personality disorder. J Ment Heal. [Internet]. 2010;19(1):43-51. [cited Aug 31 2018]. Available from: http://www.tandfonline.com/doi/full/10.3109/09638230903469152        [ Links ]

45. Malins G, Oades L, Viney L, Aspden S. What's in a service? Consumers' views of Australian mental health services. Psychiatr Rehabil J. [Internet]. 2006;29(3):197-204. [cited Aug 31 2018]. Available from: http://doi.apa.org/getdoi.cfm?doi=10.2975/29.2006.197.204        [ Links ]

46. Happell B, Roper C. Consumer Participation in Mental Health Research: Articulating a Model to Guide Practice. Australas Psychiatry. [Internet]. 2007;15(3):237-41. [cited Feb 4 2018]. Available from: http://journals.sagepub.com/doi/10.1080/10398560701320113        [ Links ]

47. Happell B. Determining the effectiveness of mental health services from a consumer perspective: Part 1: Enhancing recovery. Int J Ment Health Nurs. [Internet]. 2008;17(2):116-22. [cited Feb 4 2018]. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1447-0349.2008.00519.x        [ Links ]

48. Happell B. Determining the effectiveness of mental health services from a consumer perspective: Part 2: Barriers to recovery and principles for evaluation. Int J Ment Health Nurs. [Internet]. 2008;17(2):123-30. [cited Feb 4 2018]. Available from: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1447-0349.2008.00520.x        [ Links ]

49. Ning L, Weavell W, Woodhouse S. Consumer participation in research. Psychiatric Disability Services of Victoria. Victoria; 2010. [cited Sep 21 2017]. Available from: http://mheco.org.au/attachments/NewParadigm_Au-tumn2010_Consumer_Research.pdf        [ Links ]

50. Furtado JP, Campos RO. Participação, produção de conhecimento e pesquisa avaliativa: a inserção de diferentes atores em uma investigação em saúde mental. Cad Saúde Pública. [Internet]. 2008;24(11):2671-80. [Acesso 5 set 2018]. Disponível em: http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0102-311X2008001100022&lng=pt&tlng=pt        [ Links ]

51. Presotto RF. Participação de Usuários de Serviços de Saúde Mental em Pesquisas: Um olhar a partir dos conceitos de Empowerment e Recovery [Internet]. [Dissertação] Campinas (SP). Universidade Estadual de Campinas; 2013. [Acesso 12 out 2017]. Disponível em: http://repositorio.unicamp.br/bitstream/REPOSIP/311879/1/Presotto_RodrigoFernando_M.pdf        [ Links ]

52. Nascimento NS do, Nogueira A, Presotto RF, Angelini CR, Enes G da ST, Bilbao EV, et al. Cad Bras Saúde Mental. [Internet]. 2017;9(01):271-81. [Acesso 4 Fev 2018]. Disponível em: http://stat.ijkem.incubadora.ufsc.br/index.php/cbsm/article/view/4691        [ Links ]

53. Stewart D, Burrow H, Duckworth A, Dhillon J, Fife S, Kelly S, et al. Thematic analysis of psychiatric patients' perceptions of nursing staff. Int J Ment Health Nurs. [Internet]. 2015;24(1):82-90. [cited Aug 31 2018]. Available at: http://doi.wiley.com/10.1111/inm.12107        [ Links ]

54. Taylor S, Abbott S, Hardy S. The INFORM Project: A Service User-Led Research Endeavor. Arch Psychiatr Nurs. [Internet]. 2012;26(6):448-56. [cited Aug 31 2018]. Available from: https://www.sciencedirect.com/science/article/pii/S0883941712000337        [ Links ]

55. Tempier R, Hepp SL, Duncan CR, Rohr B, Hachey K, Mosier K. Patient-Centered Care in Affective, Non-Affective, and Schizoaffective Groups: Patients' Opinions and Attitudes. Commun Ment Health J. [Internet]. 2010;46(5):452-60. [cited Aug 31 2018]. Available from: http://link.springer.com/10.1007/s10597-010-9316-2        [ Links ]

56. Barbato A, Bajoni A, Rapisarda F, D'Anza V, De Luca LF, Inglese C, et al. Quality Assessment of Mental Health Care by People with Severe Mental Disorders: A Participatory Research Project. Community Ment Health J. [Internet]. 2014;50(4):402-8. [cited Aug 31 2018]. Available from: http://link.springer.com/10.1007/s10597-013-9667-6        [ Links ]

57. Wetzel C, Kantorski LP. Avaliação de serviços em saúde mental no contexto da reforma psiquiátrica. Texto Contexto Enferm. [Internet]. 2004;13(4):593-8. [Acesso 4 set 2018]. Disponível em: http://www.redalyc.org/html/714/71413412/        [ Links ]

58. Silva N dos S, Melo JM, Esperidião E. Avaliação dos serviços de assistência em saúde mental brasileiros: revisão integrativa da literatura. Rev Min Enferm. [Internet]. 2012;16(2):280-8. [Acesso 5 set 2018]. Disponível em: http://www.reme.org.br/artigo/detalhes/530        [ Links ]

 

 

Autor correspondente:
Éllen Cristina Ricci
E-mail: ellenricci@gmail.com

Received: Jul 12^th 2019
Accepted: Jan 31^st 2020

 

 

Authors' Contributions: Research conception and design: Éllen Cristina Ricci, Mariana Barbosa Pereira, Leidy Janeth Erazo, Rosana Teresa Onocko-Campos and Erotildes Maria Leal. Data collection: Éllen Cristina Ricci, Mariana Barbosa Pereira and Leidy Janeth Erazo. Data analysis and interpretation: Éllen Cristina Ricci, Mariana Barbosa Pereira, Leidy Janeth Erazo and Erotildes Maria Leal. Manuscript writing: Éllen Cristina Ricci, Mariana Barbosa Pereira, Leidy Janeth Erazo, Rosana Teresa Onocko-Campos and Erotildes Maria Leal. Critical revision of the manuscript: Éllen Cristina Ricci, Mariana Barbosa Pereira, Leidy Janeth Erazo, Rosana Teresa Onocko-Campos and Erotildes Maria Leal.
All authors approved the final version of the text.
Conflict of interest: The authors have stated that there are no conflicts of interest.
* Paper extracted from doctoral dissertation “Evaluation of mental health services in relation to recovery practices in the users' perspective: translation, cross-cultural adaptation and instrument validation of the Recovery Self- Assessment (RSA-R).”, presented to Faculdade de Ciências Médicas, Unicamp 2019, Campinas, SP, Brazil.