Mental health is a multifaceted concept with fuzzy boundaries.1–10 Constructs subsumed by ‘mental health’ include ‘mental illness’, ‘mental well-being’,11 12 ‘psychological well-being’13 and ‘mental wellness’.14 15 In the context of this conceptual complexity, a vast array of literature reports on empirical studies aiming to measure aspects of mental health using various self-report instruments. Several critics have noted that there is a remarkable lack of argument concerning the choice of self-report measurement instruments for the wide variety of study aims.16–21 The relatively new domain of digital data collection (collecting such self-report data through digital technologies such as online websites, mobile applications and self-service health kiosks) is not exempt from this criticism,22 23 with many studies apparently treating selection of a self-report mental health measure as uncontentious.
The overarching aim of the present review is to advance understanding of the measurement of the multifaceted construct ‘mental health’ by conducting (to our knowledge) the first audit of instruments focusing on digital delivery and repeated measurement of mental health in the general adult population. Such instruments are becoming more common and more important across the landscape of mental health and well-being in digital monitoring,24 digital assessments,25 digital phenotyping,26 self-management27 and so on in research and practice. The outcomes from the present review will expand on and update previous reviews conducted on mental health instruments (eg, Breedvelt et al and Beidas et al 28 29) and ultimately be used to inform the development of a new digital mental health instrument for monitoring purposes in the general population. This new digital instrument will be developed using item response theory30 31 and adopting the Patient-Reported Outcomes Measurement Information System (PROMIS) instrument development and validation scientific standards.32 Once the digital instrument is developed, it will be integrated into our commercial partner’s health kiosk and online portal,33 complementing their physical well-being measurements. The present review is guided by the question: What is the optimal operationalisation of mental health in the context of digital delivery of assessment and repeated measurement in the general population?
This review will take a broad definition of ‘mental health’ to target the general adult population, which covers the entire spectrum of mental health phenomena. Informed by the complete state model of mental health2 34 (see figure 1), this broad definition of mental health encompasses two correlated but separate dimensions: positive mental health (eg, flourishing, satisfaction of life, hedonic/emotional well-being, psychological well-being, social well-being) and mental illness (eg, anxiety, depression, post-traumatic stress disorder, psychosis, schizophrenia). We recognise that there are subgroups within the general population with different health characteristics such as individuals who have depression and individuals who are living with chronic illnesses (eg, diabetes, cancer). Depression is prevalent in the general adult population in Australia,35 and we expect our new instrument will measure it. By adopting the broad definition of mental health in guiding the search strategy, this review can improve the relevancy of the systematic review and increase the ecological validity of the new digital instrument for the general population, including subgroups with different health characteristics.
Complete state model of mental health (Keyes69). Source: adapted from Teng et al. 70
One consequence of the present study focusing on repeated measure instruments is a slightly expanded approach to the investigation of instruments’ format (eg, survey length, response options) and psychometric properties. Indeed, there are considerable differences between the desirable properties of instruments that are used for longitudinal measures (repeated) and instruments that are used for cross-sectional (one-off) measures.36 For example, format-wise, a small number of items (between 1 and 40 items) and multiple response options (eg, 7-point Likert scales) are recommended for an instrument used for repeated measures. In contrast, a large number of items (between 20 and 100 items) and a smaller number of response options (eg, binary or at most three options) are recommended for an instrument used for cross-sectional (one-off) measures.36 Similarly, as an instrument is repeatedly administered in a longitudinal study, this instrument’s ability to detect meaningful changes (shifts) over time is important compared with an instrument used for one-off measurement.23 37 Furthermore, as most of the general public in Australia (the target population of our new instrument) are not diagnosed with mental disorders,38 the existence of floor and ceiling effects in an instrument could jeopardise its overall utility.
The focus on the digital delivery of instruments in this review also raises novel questions about the influence of digital format on the instruments’ psychometric properties. Specifically, the historical development of an instrument, that is, whether an instrument is developed with digital delivery in mind or adapted from an original instrument previously designed for other modalities such as pen-and-paper or face-to-face/telephone interview. While past literature has established the feasibility and acceptability of using digital measurements as part of the monitoring routines in psychiatric treatments,25 39 complemented by an abundance of mobile apps that enable symptoms monitoring,40 questions remain on how the digital delivery/modality may influence the psychometric properties of an instrument compared with their equivalent conventional counterparts (eg, pen-and-paper). Although the comparison between different modalities is out of scope in this review, the sole focus on digital instruments in this review will provide comparable insights into their structure, format, psychometric properties and usage and measurement aims.
A key methodological challenge this review must plan for is the abundance of self-report mental health instruments in common use (research and practice) due to the multifaceted nature of the mental health concept and its heterogeneity. To address this challenge, this review will reference prominent conceptual frameworks and taxonomies in the mental health domain to guide the search strategy and eligible criteria (see Methods and Analysis section). The broad mental health constructs measured by current instruments could span from symptoms of mental disorders28 41 to mental well-being.15
Instruments that measure the symptoms of mental disorders are commonly based on two widely accepted taxonomies—the Diagnostic and Statistical Manual of Mental Disorders (DSM-542) and the International Classification of Diseases (ICD-1143), such as the 9-item Patient Health Questionnaire44 measuring the severity of depression symptoms and the 7-item General Anxiety Disorder Questionnaire45 screening symptoms of the Generalised Anxiety Disorder. In contrast, instruments that measure mental well-being vary depending on the constructs,28 ranging from mental states (eg, happiness, emotional well-being, psychological well-being, social well-being), cognitive evaluation (life satisfaction, life meaning), protective factors (eg, resilience, optimism, hope, compassion) and risk factors (eg, stress, sleep quality, traumatic experience). For example, the 5-item Satisfaction with Life Scale46 measures global evaluation of one’s life and the 8-item Flourishing Scale47 measures perceived success in one’s life areas (eg, self-esteem, purpose and optimism). The abundance of mental health self-report instruments also reveals another phenomenon, in which one or more instruments can measure a construct. For example, at least 11 self-report instruments are available to assess the severity of depression with varying degrees of measurement precision, range and target population.48 Similarly, at least 92 self-report instruments are available to measure anxiety.49 To navigate and manage the complexity among this large pool of mental health instruments and constructs, this protocol will put together a defensible framework (described in the ‘Methods and analysis’ section), referencing prominent conceptual frameworks and taxonomies to guide the search strategy and eligibility criteria.
This review also aims to generate insights into the relationship between mental health measures and mental health constructs, as exemplified by the studies identified here. Specifically, we will, where possible, extract from text in empirical studies the constructs authors were intending to measure with a given instrument, and compare them to the constructs the instrument developers were intending to capture (as described in the text of the original validation article). This comparison may highlight potential validity mismatches between the phenomena that instruments were developed to measure, the phenomena measured by empirical studies and participants’ views of the constructs.17 Although studying this mismatch is out of scope for the present review, we will extract constructs that instruments were used to measure empirically in identified studies, as well as constructs that the instruments were originally designed to measure. The preliminary analysis and data extracted about this phenomenon in this review could provide the groundwork for future reviews and potentially increase the awareness of instrument selection for future studies.
Previous and ongoing systematic review efforts have investigated instruments used in non-clinical adult populations for mental disorder diagnosis,41 symptoms screening28 and mental well-being or similar constructs (eg, subjective well-being)21 50–53 grounded in different conceptual frameworks. A preliminary search on PROSPERO (conducted 5 November 2021) also revealed several similar reviews of public mental health instruments (completed or ongoing reviews), such as54 on public mental health outcome measures in the UK and15 on mental wellness of adolescents.
To the best of our knowledge, there is no review (or protocol) that shares the specific focus of the present review, that is, digital self-report mental health instruments for repeated measurements in the general adult population.
The main objective of this review is to systematically identify past empirical studies in which mental health in the general population was measured (1) using digital self-report instruments, and (2) in repeated measures designs. We will extract from identified empirical studies information about (1) the structure, (2) format, (3) psychometric properties, (4) frequency of use and (5) the construct instruments were used to measure in these studies, as well as the constructs the instruments were designed to measure in the original publication. The review will study these instruments from different perspectives through five research questions:
This protocol adheres to the Preferred Reporting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) 2015 guidelines.55 56
No patient involved.
This review will include studies that match the inclusion criteria in the PICOT format57 defined below.
All studies involving community dwellers in the general adult population will be included. Adulthood will be defined as 18 and above, and studies with samples including adults aged 18 or above, for example, a study that recruits participants (young adults) aged between 16 and 25 will be included. In addition, studies involving subpopulation groups, for example, grouped by occupational or sociodemographical, will also be included. Studies of clinical populations, including those with physical and mental health conditions will be included, as long as the sample population is living in the community.
Studies that exclusively target infants, children, adolescents and individuals not residing in the community (eg, inpatients, prisoners, military personnel during deployment) will be excluded.
Interventions are not a focus of this review. All peer-reviewed empirical studies reporting on the administration of digital (online websites, mobile apps, health kiosks) self-report and self-administered mental health instruments in English at more than one time point will be included. This covers empirical studies measuring mental health constructs of the same individuals (within-person) or the same groups (within-group) over time. Study designs are likely to include population-based longitudinal studies, repeated cross-sectional studies, multiwaves surveys, cohort studies (retrospective/prospective), case-control studies, mixed-method studies, scale evaluation studies, quantitative randomised and non-randomised controlled trials (pre-post measurements after any intervention/treatment, eg, psychological, medication).
Studies that administered a mental health instrument that is non-English (including translated instruments from English) or at one time point only (eg, single-wave cross-sectional survey, screening participants) will be excluded. Secondary analyses of previously collected surveys, feasibility, pilot, proof-of-concept, exploratory studies, qualitative studies, case studies and protocol for research studies or review protocols will be excluded.
Included empirical studies must be published in English, in the format of peer-reviewed journal articles. Review articles (systematic review, literature review, scoping review, integrative review, meta-analyses) will be excluded. Studies that were not peer-reviewed, not published in English or published as preprints, case reports, opinions, conceptual or theoretical discussion articles will also be excluded.
Included studies must have at least one self-report digital instrument in English measuring mental health at more than one time point (except studies that administered a single wave of a national survey that had been administered in the past). Screening instruments are eligible. Studies that used self-report instruments as third-party observations such as proxy-report instruments (eg, parent’s self-report on a child’s behaviours) will be excluded.
The mental health constructs of interest in this review are guided by frameworks defined in the section ‘Conceptual Frameworks and Taxonomies’ (see figure 2 and online supplemental appendix 1). As shown in figure 2, syndromes (mental disorders) defined in Forbes et al 58 guided by HiTOP are included. Personality disorders were excluded because they are closely related to personality, which is a trait that is generally considered stable over time in adults.59 In this review, we are interested in state-like constructs and their level of change across time. Furthermore, personality disorders could manifest as symptoms of other psychopathology such as anxiety and depression, which will be included in this review.58 60 We also included psychological stress because stress is commonly being recognised to precipitate anxiety and depression and it has also been found as a separate factor while analysing anxiety and depression scales.61 All these syndromes will form the search terms in this review.