BackGround
Selecting and prioritising determinants for an occupation-based intervention
The emergence of mental illness during adolescence is known to affect daily functioning, specifically disrupting daily activities, routines, and patterns (Parsonage, 2016; McGorry & Mei, 2018) with potentially long lasting consequences for the individual, their family and society (Patton & Temmerman, 2016; Patton et al. , 2016). Conversely, a narrative review of leisure activities and a multi-level theoretical framework of mechanisms of action suggests the relationship between mental health and activity choices is actually bi-directional (Fancourtet al. , 2021). Neuroscience provides further insight illuminating potential mechanisms by which engaging in activity within one’s environment affects adolescent brain development and may present an opportunity for intervention (Larsen & Luna, 2018).
Novel interventions are needed to address rising levels of adolescent mental health difficulties internationally (Mei et al., 2020). Reviews highlight the limitations of current approaches (Das et al, 2016). Developing interventions using an occupational therapy approach to improve mental health shows potential (Kirsh et al ., 2019). Although there is limited research supporting this approach in adolescent populations (Parsonage-Harrison et al ., 2022). Occupational therapy approaches incorporate a focus on the person, and their daily activities (known as occupations) in the context of their environment (Creek, 2006). The evidence-base for using activity in adolescent populations to improve mental health remains problematic (Parsonage-Harrison et al ., 2022 & Das et al 2016). Effective intervention development requires the identification, selection and prioritisation of determinants or factors affecting behaviour change to improve health outcomes (Bartholomew-Eldredge et al. , 2016).
The onset, nature, and subsequent course of mental health difficulties may be improved if multiple inter-related personal, social and environmental determinants are addressed (Viner et al. , 2012; McGorry et al. , 2014; Patel et al. , 2018; Mei et al. , 2020), minimising the disruption to an individual’s life. The value of addressing determinants at sub-clinical symptom threshold levels before severe functional impairments emerge is strongly advocated internationally, but remains a challenge (McGorry, & Mei, 2018). Knowledge of the effects of these many determinants on the emerging and early stages of mental health difficulties is limited (Cairns et al. , 2015; Bale et al. , 2020). Earlier qualitative work identified determinants connected to adolescents’ choices about the activities they do (Parsonage et al., 2020). The work, highlighting a process of considering time factors, appraising values and priorities, interaction with the situational context and an exploration of skills and occupational repertoire, that through experience shapes the development of an adolescent’s future self (Parsonage et al., 2020). Given the potentially modifiable nature of many of these determinants, knowing which are realistic to attempt to change and have greatest influence on health outcomes, is important for intervention development.
A wealth of experiential knowledge based on using activity to improve adolescent mental health exists internationally in the form of clinically practicing occupational therapists and researchers, that can help to inform intervention development for adolescent populations. Multiple methods exist to identify and prioritise determinants when developing interventions, we adopted Intervention Mapping framework for intervention development, which advocates a systematic consultation of the literature and a wide variety of stakeholders at all stages of the development process (Bartholomew-Eldredge et al. , 2016). Stakeholder involvement helps maintain focus on issues of concern; ensures intervention acceptability to the target population; increases expertise on the project; and improves external validity (Bartholomew-Eldredge et al. , 2016). Involvement reduces researcher bias towards certain topics or ideas and can highlight ideas the researcher may not otherwise have thought of (Bartholomew-Eldredge et al. , 2016).
This paper reports on a novel Delphi study conducted with an expert stakeholder group of occupational therapists and researchers working with adolescents or related researching topics. The study was undertaken to select and prioritise the determinants connected with what activities or occupations young people choose to do, in their daily lives, that influence their mental health. To the best of the author’s knowledge, no study has previously been conducted with occupational therapists and researchers, to prioritise occupation or activity focused determinants related to adolescent choice that may affect or influence mental health.
Aim
To establish an expert consensus view of which occupational determinants should be prioritised within the development of an occupation therapy-based intervention for adolescents with emerging mental health difficulties.
Ethics
The study received approval from Oxford Brookes University Research Ethical Committee (UREC no.191347).
Method: The Delphi method and seeking consensous
An electronic two round Delphi survey method was chosen, designed to establish an expert ‘consensus of opinion’ evolving from individual experts’ anonymised judgements, disclosed through multiple iterative rounds of questionnaires (Keeney et al. , 2001; Dimitrijevićet al. , 2012; McPherson et al. , 2018; Sossa et al. , 2019). The method is suited to addressing practice-related problems where human judgement is required to solve complex problems (Powell, 2003; Steurer, 2011; Dimitrijević et al. , 2012; Donohoe et al. , 2012) and has previously been used to prioritise determinants important to adolescent mental health (Cairns et al. , 2015; Baleet al. , 2020). This method enables the inclusion of participants from a broad range of geographical areas (McPherson et al. , 2018), and makes the distribution, collection and analysis of data cost-effective and time-efficient (Dimitrijević et al. , 2012; Donohoe et al. , 2012), all of which were important for this study.
Recruitment and selection of the expert panel
The representativeness of the expert panel is important and the selection of experts is influenced by the information the researcher wants to gather (Steurer, 2011). We set the following criteria for our expert panel; Participants must hold a qualification as an occupational therapist and have experience of working with adolescents, or be a researcher, working with adolescents with an occupation focus. These criteria were checked by potential participant’s responses to self-report and verification questions. The research team identified potential participants through specialist groups and the peer reviewed literature. Each potential panel member received an email invitation to participate. We opted for a minimum of twenty participants, reflecting the typical numbers used in the Delphi studies literature, and in light of the lack of formal recommendations in the literature (Keeney et al. , 2001; Dimitrijević et al. , 2012).
Questionnaire development
The Delphi study was structured in two parts. The first part consisted of an information sheet followed by seven consent related questions and questions designed to check about the expert panel members experience. The second part was formed of six questions informed by the intervention mapping framework. Each of the 59 occupation related determinants identified in relation to adolescents’ mental health through three earlier studies (Parsonage et al., 2020, Parsonage-Harrisonet al. , 2022, Parsonage, 2022) were organised under the appropriate question heading. As suggested by Dimitrijević et al., (2012) to ensure reliability, the questionnaire was piloted. The questionnaire format was developed for distribution using Qualtrix XM (Qualtrix, 2005), then piloted by three researchers before being distributed via email following amendments. The development of round two followed the same process.
Delphi rounds
We conducted a modified two round Delphi study, replacing the open-questions round typically used in round one of a Delphi with a ranking question round (Keeney et al ., 2011). Participants were given the option to add additional qualitative information in round one. Three rounds were originally planned but due to a moderately high-level agreement after the second-round, coinciding with the beginning of Covid pandemic, the research team agreed a third round was not required and should not be conducted to avoid unnecessary burden on clinicians.
Based on their professional opinion, participants completing the first Delphi round were given the option to add determinants before ranking them according to which they considered had the greatest impact on mental health. Following the closure of round one, the data was exported from Qualtrix XM (Qualtrix 2005) into Microsoft Excel (Microsoft Corporation, 2016). In round two, participants received a summary of their responses, and a summary of the whole panel’s results. The rationale was to provide the participant with an opportunity to reflect on their choices (McPherson et al. , 2018), and encourage a response to round two (Murphy M.K. et al. , 1998; Powell, 2003).
Achieving consensus
Delphi studies aim to achieve a consensus opinion, defined as the general agreement arrived at (McPherson et al. , 2018).Considerable variability exists in how consensus is both defined and achieved (Bowles, 1999). We chose a frequently used ranking system (Powell, 2003), using a weighted points system to reflect the number of times an item was selected and its position in the ranking, resulting in a total score. This total score was used to rank and identify the consensus. A further non-parametric assessment, Kendall’s W coefficient of concordance, was used to consider the extent of agreement between those rating each round (Sossa et al. , 2019) The following divisions can help to provide a benchmark for considering levels of agreement (Landis and Koch, 1977): poor agreement = less than 0.20, Fair agreement = 0.21 to 0.40, Moderate agreement = 0.41 to 0.60, good agreement = 0.61 to 0.80 and very good agreement = 0.81 to 1.00.
Results
Twenty people agreed to participate as panel members. Two blank responses were excluded. One participant submitted a partial and a completed response, only the completed questionnaire was analysed. A computer error effecting consent questions was identified, so the research team sent an additional email to 11 of the 17 respondents to confirm full consent. This resulted in at total of 15 consenting expert panel members in round one, who were invited to take part in round two. The second Delphi round received 13 responses.
Of the fifteen panel members in round one, four self-identified as a researcher and twelve as state registered occupational therapists working with adolescents. Thirteen participants reported at least five years of work experience, while six indicated they had over 10 years’ experience. All but one panel member agreed with the statement that in their professional opinion the way adolescents spend their time affects their wellbeing. Responses from round one added a further 30 determinants, to the original 59 determinants previously identified (see Figure 1 for details). All of the determinants included in the Delphi are available in Appendix 1.
Add here Figure 1
In question one, round one item scores ranged from 36 to 175 and 18 to 154 in round two. The most frequently selected determinants relating to what adolescents do, that affects their mental health, were: ‘types of activity’ (n154) and ‘balance of activity’ (n137). These two determinants achieved the highest level of agreement (31%) in round one and increased in round two to 90% and 60% respectively. The item ranked third was the ‘pressure to conform’ (n130) but the level of agreement decreased from 27% to 20% between rounds (See Table 1 for details).