Publications
>
Article

Vickybot, a chatbot for anxiety-depressive symptoms and work-related burnout in primary care and healthcare professionals: development, feasibility, and potential effectiveness studies

Publicated to:Journal Of Medical Internet Research. 25 e43293- - 2023-04-03 25(), DOI: 10.2196/43293

Authors: Anmella, Gerard; Sanabra, Miriam; Prime-Tous, Mireia; Segu, Xavier; Cavero, Myriam; Morilla, Ivette; Grande, Iria; Ruiz, Victoria; Mas, Ariadna; Martin-Villalba, Ines; Caballo, Alejandro; Esteva, Julia-Parisad; Rodriguez-Rey, Arturo; Piazza, Flavia; Valdesoiro, Francisco Jose; Rodriguez-Torrella, Claudia; Espinosa, Marta; Virgili, Giulia; Sorroche, Carlota; Ruiz, Alicia; Solanes, Aleix; Radua, Joaquim; Also, Maria Antonieta; Murgui, Sandra; Sans-Corrales, Mireia; Young, Allan H; Vicens, Victor; Blanch, Jordi; Caballeria, Elsa; Lopez-Pelayo, Hugo; Lopez, Clara; Olive, Victoria; Pujol, Laura; Quesada, Sebastiana; Sole, Brisa; Torrent, Carla; Martinez-Aran, Anabel; Guarch, Joana; Navines, Ricard; Murru, Andrea; Fico, Giovanna; de Prisco, Michele; Oliva, Vicenzo; Amoretti, Silvia; Pio-Carrino, Casimiro; Fernandez-Canseco, Maria; Villegas, Marta; Vieta, Eduard; Hidalgo-Mazzei, Diego

Affiliations

Abi Global Hlth, Dublin, Ireland - Author
Addict Unit Hosp Clin Barcelona, Dept Psychiat, Grup Recerca Add Clin, Barcelona, Spain - Author
Barcelona Supercomp Ctr BSC, Text Min Technol Hlth Domain, Barcelona, Spain - Author
Barcelona Supercomputing Center (BSC). Text Mining Technologies in the Health Domain, Barcelona, ES. - Author
CAP Borrell, Consorci Atencio Primaria Salut Barcelona Esquerr, Barcelona, Spain - Author
CAP Borrell, Consorci d'Atenció primaria de Salut Barcelona Esquerra (CAPSBE), Barcelona, ES. - Author
CAP Casanova, Consorci Atenció primaria de Salut Barcelona Esquerra (CAPSBE), Barcelona, ES. - Author
CAP Casanova, Consorci Atencio Primaria Salut Barcelona Esquerr, Barcelona, Spain - Author
Centre for Affective Disorders, Institute of Psychiatry, Psychology & Neuroscience, King's College, London, GB. - Author
Chief Medical Officer and co-founder of Abi Global Health, Barcelona, ES. - Author
European Assoc Psychosomat Med, Oregon, OR USA - Author
Hosp Clin Barcelona, Inst Neurosci, Dept Psychiat & Psychol, Barcelona, Spain - Author
Hosp Clin Barcelona, Occupat Hlth Dept, Barcelona, Spain - Author
Hospital Clínic de Barcelona, Department of Psychiatry and Psychology, Institute of Neuroscience, C. de Villarroel, 170, 08036, Barcelona, ES. - Author
Hospital Clínic de Barcelona, Occupational Health Department, barcelona, ES. - Author
Inst Invest Biomed August Pi i Sunyer IDIBAPS, Bipolar & Depress Disorders Unit, Digital Innovat Grp, Barcelona, Spain - Author
Inst Invest Biomed August Pi i Sunyer IDIBAPS, Imaging Mood and Anxiety Related Disorders Grp IM, Barcelona, Spain - Author
Inst Neurosci UBNeuro, Barcelona, Spain - Author
Inst Salud Carlos III, Biomed Res Networking Ctr Consortium Mental Hlth, Madrid, Spain - Author
Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, ES. - Author
Karolinska Inst, Ctr Psychiat Res, Dept Clin Neurosci, Stockholm, Sweden - Author
Kings Coll London, Inst Psychiat Psychol & Neurosci, Ctr Affect Disorders, London, England - Author
Kings Coll London, Inst Psychiat Psychol & Neurosci, Dept Psychosis Studies, Early Psychosis Intervent & Clin Detect EP Lab, London, England - Author
Red Invest Atenc Primaria Adicc RIAPAd, Barcelona, Spain - Author
T1.01 - Recerca transversal en atenció primària. Institut d'Investigacions Biomèdiques August Pi i Sunyer - Author
Univ Barcelona UB, Sch Med & Hlth Sci, Dept Med, Barcelona, Spain - Author
See more

Abstract

A significant proportion of people attending Primary Care (PC) have anxiety-depressive symptoms and work-related burnout compounded by a lack of resources to meet their needs. The COVID-19 pandemic has exacerbated this problem and digital tools have been proposed as a solution.We present the development, feasibility, and potential effectiveness studies of Vickybot, a chatbot aimed at screening, monitoring, and reducing anxiety-depressive symptoms and work-related burnout in PC patients and healthcare workers.User-centered development strategies were adopted. Main functions included self-assessments, psychological modules, and emergency alerts. Healthy controls (HCs) tested Vickybot for reliability. (1) Simulation: HCs used Vickybot for 2 weeks to simulate different clinical situations. (2) Feasibility and effectiveness study: People consulting PC or healthcare workers with mental health problems were offered to use Vickybot for one month. Self-assessments for anxiety (GAD-7) and depression (PHQ-9) symptoms, and work-related burnout (based on the Maslach Burnout Inventory) were administered at baseline and every two weeks. Feasibility was determined from both subjective and objective user-engagement Indicators (UEIs). Potential effectiveness was measured using paired t-tests or the Wilcoxon signed-rank test for the change in self-assessment scores.40 HCs tested Vickybot simultaneously, and data was transmitted and registered reliably. (1) Simulation: 17 HCs (73% female; mean age=36.5±9.7) received 98.8% of the expected modules according to each simulation. Suicidal alerts were correctly received. (2) Feasibility and potential effectiveness study: 34 patients (15 from PC and 19 healthcare workers; 77% female; mean age=35.3±10.1) completed the first self-assessments, with 34 (100%) presenting anxiety symptoms, 32 (94%) depressive symptoms, and 22 (64.7%) work-related burnout. Nine (26.5%) patients completed the second self-assessments after two weeks of use. No significant differences were found between the scores of the first and second self-assessments for anxiety [t(8) = 1.000, P = .34] or depressive [t(8) = .40, P = .70] symptoms. However, work-related burnout scores were moderately reduced (Z = -2.07, P = 0.038, r = .32). There was a non-significant trend towards higher reduction in anxiety-depressive symptoms and work-related burnout with greater use of the chatbot. Three patients (8.8%) activated the suicide alert, and the research team intervened promptly with successful outcomes. Vickybot showed high subjective-UEIs, but low objective-UEIs (completion, adherence, compliance, and engagement). Feasibility was moderate.The chatbot was useful in screening for the presence and severity of anxiety and depressive symptoms, and detecting suicidal risk. Potential effectiveness was shown in reducing work-related burnout, but not anxiety or depressive symptoms. Subjective perceptions of use contrasted with low objective-use metrics. Our results are promising but suggest the need to adapt and enhance the smartphone-based solution in order to improve engagement. Consensus on how to report UEIs and validate digital solutions, especially for chatbots, are required.

Keywords
Adjustment disorderAdultAmitriptylineAntidepressant agentAnxietyAnxiety disorderArticleBenzodiazepine derivativeBurnoutBurnout, professionalChatbotClinical articleClinical evaluationCognitive behavioral therapyControlled studyCovid-19DepressionDialectical behavior therapyDigitalEating disorderFeasibility studiesFeasibility studyFemaleHealth care personnelHealth care workersHealth personnelHopelessnessHumanHumansLow drug doseMaleMaslach burnout inventoryMental diseaseMindfulnessMirtazapinePandemicPandemicsPatient engagementPatient health questionnaire 9Personality disorderPrestoPrimary carePrimary care digital support tool in mental healthPrimary health carePrimary medical careProfessional burnoutPsychologyReliabilityReproducibilityReproducibility of resultsSatisfactionScreeningSelf esteemSelf evaluationSmartphoneSuicidal ideationSuicideSymptomTrazodoneUsability

Quality index

Bibliometric impact. Analysis of the contribution and dissemination channel

The work has been published in the journal Journal Of Medical Internet Research due to its progression and the good impact it has achieved in recent years, according to the agency WoS (JCR), it has become a reference in its field. In the year of publication of the work, 2023, it was in position 10/174, thus managing to position itself as a Q1 (Primer Cuartil), in the category Health Care Sciences & Services. Notably, the journal is positioned above the 90th percentile.

From a relative perspective, and based on the normalized impact indicator calculated from World Citations provided by WoS (ESI, Clarivate), it yields a value for the citation normalization relative to the expected citation rate of: 5.95. This indicates that, compared to works in the same discipline and in the same year of publication, it ranks as a work cited above average. (source consulted: ESI Nov 14, 2024)

This information is reinforced by other indicators of the same type, which, although dynamic over time and dependent on the set of average global citations at the time of their calculation, consistently position the work at some point among the top 50% most cited in its field:

  • Weighted Average of Normalized Impact by the Scopus agency: 4.41 (source consulted: FECYT Feb 2024)
  • Field Citation Ratio (FCR) from Dimensions: 34.68 (source consulted: Dimensions May 2025)

Specifically, and according to different indexing agencies, this work has accumulated citations as of 2025-05-11, the following number of citations:

  • WoS: 21
  • Scopus: 30
  • Europe PMC: 7
  • OpenCitations: 22
Impact and social visibility

From the perspective of influence or social adoption, and based on metrics associated with mentions and interactions provided by agencies specializing in calculating the so-called "Alternative or Social Metrics," we can highlight as of 2025-05-11:

  • The use, from an academic perspective evidenced by the Altmetric agency indicator referring to aggregations made by the personal bibliographic manager Mendeley, gives us a total of: 197.
  • The use of this contribution in bookmarks, code forks, additions to favorite lists for recurrent reading, as well as general views, indicates that someone is using the publication as a basis for their current work. This may be a notable indicator of future more formal and academic citations. This claim is supported by the result of the "Capture" indicator, which yields a total of: 197 (PlumX).

With a more dissemination-oriented intent and targeting more general audiences, we can observe other more global scores such as:

  • The Total Score from Altmetric: 20.6.
  • The number of mentions on the social network Facebook: 1 (Altmetric).
  • The number of mentions on the social network X (formerly Twitter): 18 (Altmetric).

It is essential to present evidence supporting full alignment with institutional principles and guidelines on Open Science and the Conservation and Dissemination of Intellectual Heritage. A clear example of this is:

  • The work has been submitted to a journal whose editorial policy allows open Open Access publication.
Leadership analysis of institutional authors

This work has been carried out with international collaboration, specifically with researchers from: Sweden; United Kingdom; United States of America.

There is a significant leadership presence as some of the institution’s authors appear as the first or last signer, detailed as follows: First Author (Anmella Diaz, Gerard) and Last Author (Navinés De la Cruz, Ricard).

the author responsible for correspondence tasks has been Hidalgo Mazzei, Diego Alberto.