BackGround
Selecting
and prioritising determinants for an occupation-based intervention
The emergence of mental illness during adolescence is known to affect
daily functioning, specifically disrupting daily activities, routines,
and patterns (Parsonage, 2016; McGorry & Mei, 2018) with potentially
long lasting consequences for the individual, their family and society
(Patton & Temmerman, 2016; Patton et al. , 2016). Conversely, a
narrative review of leisure activities and a multi-level theoretical
framework of mechanisms of action suggests the relationship between
mental health and activity choices is actually bi-directional (Fancourtet al. , 2021). Neuroscience provides further insight illuminating
potential mechanisms by which engaging in activity within one’s
environment affects adolescent brain development and may present an
opportunity for intervention (Larsen & Luna, 2018).
Novel interventions are needed to address rising levels of adolescent
mental health difficulties internationally (Mei et al., 2020).
Reviews highlight the limitations of current approaches (Das et al,
2016). Developing interventions using an occupational therapy approach
to improve mental health shows potential (Kirsh et al ., 2019).
Although there is limited research supporting this approach in
adolescent populations (Parsonage-Harrison et al ., 2022).
Occupational therapy approaches incorporate a focus on the person, and
their daily activities (known as occupations) in the context of their
environment (Creek, 2006). The evidence-base for using activity in
adolescent populations to improve mental health remains problematic
(Parsonage-Harrison et
al ., 2022 & Das et al 2016). Effective intervention development
requires the identification, selection and prioritisation of
determinants or factors affecting behaviour change to improve health
outcomes (Bartholomew-Eldredge et al. , 2016).
The onset, nature, and subsequent course of mental health difficulties
may be improved if multiple inter-related personal, social and
environmental determinants are addressed (Viner et al. , 2012;
McGorry et al. , 2014; Patel et al. , 2018; Mei et
al. , 2020), minimising the disruption to an individual’s life. The
value of addressing determinants at sub-clinical symptom threshold
levels before severe functional impairments emerge is strongly advocated
internationally, but remains a challenge (McGorry, & Mei, 2018).
Knowledge of the effects of these many determinants on the emerging and
early stages of mental health difficulties is limited (Cairns et
al. , 2015; Bale et al. , 2020). Earlier qualitative work
identified determinants connected to adolescents’ choices about the
activities they do (Parsonage et al., 2020). The work, highlighting a
process of considering time factors, appraising values and priorities,
interaction with the situational context and an exploration of skills
and occupational repertoire, that through experience shapes the
development of an adolescent’s future self (Parsonage et al., 2020).
Given the potentially modifiable nature of many of these determinants,
knowing which are realistic to attempt to change and have greatest
influence on health outcomes, is important for intervention development.
A wealth of experiential knowledge based on using activity to improve
adolescent mental health exists internationally in the form of
clinically practicing occupational therapists and researchers, that can
help to inform intervention development for adolescent populations.
Multiple methods exist to identify and prioritise determinants when
developing interventions, we adopted Intervention Mapping framework for
intervention development, which advocates a systematic consultation of
the literature and a wide variety of stakeholders at all stages of the
development process (Bartholomew-Eldredge et al. , 2016).
Stakeholder involvement helps maintain focus on issues of concern;
ensures intervention acceptability to the target population; increases
expertise on the project; and improves external validity
(Bartholomew-Eldredge et
al. , 2016). Involvement reduces researcher bias towards certain topics
or ideas and can highlight ideas the researcher may not otherwise have
thought of (Bartholomew-Eldredge et al. , 2016).
This paper reports on a novel Delphi study conducted with an expert
stakeholder group of occupational therapists and researchers working
with adolescents or related researching topics. The study was undertaken
to select and prioritise the determinants connected with what activities
or occupations young people choose to do, in their daily lives, that
influence their mental health. To the best of the author’s knowledge, no
study has previously been conducted with occupational therapists and
researchers, to prioritise occupation or activity focused determinants
related to adolescent choice that may affect or influence mental health.
Aim
To establish an expert consensus view of which occupational determinants
should be prioritised within the development of an occupation
therapy-based intervention for adolescents with emerging mental health
difficulties.
Ethics
The study received approval from Oxford Brookes University Research
Ethical Committee (UREC
no.191347).
Method: The Delphi method
and seeking consensous
An electronic two round Delphi survey method was chosen, designed to
establish an expert ‘consensus of opinion’ evolving from individual
experts’ anonymised judgements, disclosed through multiple iterative
rounds of questionnaires (Keeney et al. , 2001; Dimitrijevićet al. , 2012; McPherson et al. , 2018; Sossa et al. ,
2019). The method is suited to addressing practice-related problems
where human judgement is required to solve complex problems (Powell,
2003; Steurer, 2011; Dimitrijević et al. , 2012; Donohoe et
al. , 2012) and has previously been used to prioritise determinants
important to adolescent mental health (Cairns et al. , 2015; Baleet al. , 2020). This method enables the inclusion of participants
from a broad range of geographical areas (McPherson et al. ,
2018), and makes the distribution, collection and analysis of data
cost-effective and time-efficient (Dimitrijević et al. , 2012;
Donohoe et al. , 2012), all of which were important for this
study.
Recruitment
and selection of the expert panel
The representativeness of the expert panel is important and the
selection of experts is influenced by the information the researcher
wants to gather (Steurer, 2011). We set the following criteria for our
expert panel; Participants must hold a qualification as an occupational
therapist and have experience of working with adolescents, or be a
researcher, working with adolescents with an occupation focus. These
criteria were checked by potential participant’s responses to
self-report and verification questions. The research team identified
potential participants through specialist groups and the peer reviewed
literature. Each potential panel member received an email invitation to
participate. We opted for a minimum of twenty participants, reflecting
the typical numbers used in the Delphi studies literature, and in light
of the lack of formal recommendations in the literature (Keeney et
al. , 2001; Dimitrijević et al. , 2012).
Questionnaire
development
The Delphi study was structured in two parts. The first part consisted
of an information sheet followed by seven consent related questions and
questions designed to check about the expert panel members experience.
The second part was formed of six questions informed by the intervention
mapping framework. Each of the 59 occupation related determinants
identified in relation to adolescents’ mental health through three
earlier studies (Parsonage et al., 2020, Parsonage-Harrisonet al. , 2022, Parsonage, 2022) were organised under the
appropriate question heading. As suggested by Dimitrijević et
al., (2012) to ensure reliability, the questionnaire was piloted. The
questionnaire format was developed for distribution using Qualtrix XM
(Qualtrix, 2005), then piloted by three researchers before being
distributed via email following amendments. The development of round two
followed the same process.
Delphi
rounds
We conducted a modified two round Delphi study, replacing the
open-questions round typically used in round one of a Delphi with a
ranking question round (Keeney et al ., 2011). Participants were
given the option to add additional qualitative information in round one.
Three rounds were originally planned but due to a moderately high-level
agreement after the second-round, coinciding with the beginning of Covid
pandemic, the research team agreed a third round was not required and
should not be conducted to avoid unnecessary burden on clinicians.
Based on their professional opinion, participants completing the first
Delphi round were given the option to add determinants before ranking
them according to which they considered had the greatest impact on
mental health. Following the closure of round one, the data was exported
from Qualtrix XM (Qualtrix 2005)
into Microsoft Excel (Microsoft Corporation, 2016). In round two,
participants received a summary of their responses, and a summary of the
whole panel’s results. The rationale was to provide the participant with
an opportunity to reflect on their choices (McPherson et al. ,
2018), and encourage a response to round two (Murphy M.K. et al. ,
1998; Powell, 2003).
Achieving
consensus
Delphi studies aim to achieve a consensus opinion, defined as the
general agreement arrived at (McPherson et al. ,
2018).Considerable variability exists in how consensus is both defined
and achieved (Bowles, 1999). We chose a frequently used ranking system
(Powell, 2003), using a weighted points system to reflect the number of
times an item was selected and its position in the ranking, resulting in
a total score. This total score was used to rank and identify the
consensus. A further non-parametric assessment, Kendall’s W coefficient
of concordance, was used to consider the extent of agreement between
those rating each round (Sossa et al. , 2019) The following
divisions can help to provide a benchmark for considering levels of
agreement (Landis and Koch, 1977): poor agreement = less than 0.20, Fair
agreement = 0.21 to 0.40, Moderate agreement = 0.41 to 0.60, good
agreement = 0.61 to 0.80 and very good agreement = 0.81 to 1.00.
Results
Twenty people agreed to participate as panel members. Two blank
responses were excluded. One participant submitted a partial and a
completed response, only the completed questionnaire was analysed. A
computer error effecting consent questions was identified, so the
research team sent an additional email to 11 of the 17 respondents to
confirm full consent. This resulted in at total of 15 consenting expert
panel members in round one, who were invited to take part in round two.
The second Delphi round received 13 responses.
Of the fifteen panel members in round one, four self-identified as a
researcher and twelve as state registered occupational therapists
working with adolescents. Thirteen participants reported at least five
years of work experience, while six indicated they had over 10 years’
experience. All but one panel member agreed with the statement that in
their professional opinion the way adolescents spend their time affects
their wellbeing. Responses from
round one added a further 30 determinants, to the original 59
determinants previously identified (see Figure 1 for details). All of
the determinants included in the Delphi are available in Appendix 1.
Add
here Figure 1
In question one, round one item scores ranged from 36 to 175 and 18 to
154 in round two. The most frequently selected determinants relating to
what adolescents do, that affects their mental health, were: ‘types of
activity’ (n154) and ‘balance of activity’ (n137). These two
determinants achieved the highest level of agreement (31%) in round one
and increased in round two to 90% and 60% respectively. The item
ranked third was the ‘pressure to conform’ (n130) but the level of
agreement decreased from 27% to 20% between rounds (See Table 1 for
details).