ABSTRACT
Introduction Using the ‘positive health’ perspective has emerged in general healthcare. Conceptual similarities exist with the ‘recovery’ perspective in mental healthcare. Both concepts are multidimensional and focus on capability. The My Positive Health (MPH) and Individual Recovery Outcomes Counter (I.ROC) tools were developed for dialogues. These tools might be useful for quantitively measuring the positive health construct for monitoring and scientific purposes as well. We aimed to investigate this.
Method An observational cross-sectional study was conducted in a representative general Dutch population (the LISS panel) to investigate factor structures and internal consistency from the 42-items MPH and 12-items I.ROC. After randomly splitting the dataset, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were applied. Spearman correlation coefficient between both tools’ total scores was calculated.
Results 2,457 participants completed the questionnaires. A six-factor structure was extracted for MPH (PH42) and a two-factor structure for I.ROC (I.ROC12). Explained variances were 68.1% and 56.1%, respectively. CFA resulted in good fit indices. Cronbach’s alphas were between 0.74 to 0.97 (PH42) and 0.73 to 0.87 (I.ROC12). Correlation between the total scores was 0.77.
Conclusion Both PH42 and I.ROC12 are useful to quantitatively measure positive health aspects which can be summarised in sum scores in a general population. The dimensions found in this study and the corresponding item division differed from the dimensions of the original dialogue tools. Further research is recommended focussing on item reduction for PH42, factor structure of I.ROC and assessment of construct validity (in a general population) in more depth.
INTRODUCTION
In 2011, Huber et al. introduced a new, dynamic perspective on health: “health as the ability to adapt and self- manage in the face of social, physical, and emotional challenges” (Huber et al., 2011). This new perspective differs from the original WHO definition of health (The World Health Organization (WHO), 2006). It focusses on someone’s capability instead of incapability, which means that people with disabilities or chronic diseases are no longer automatically seen as ‘not healthy’ (Huber et al., 2011; van Druten et al., 2022). Huber’s perspective laid the foundations for positive health, in which health is regarded as a broad construct and as an asset to living a meaningful life (Huber et al., 2016).
In the last decades, comparable focus shifts have taken place in mental healthcare for the recovery perspective. It was broadened including not only clinical recovery (reducing symptoms), but also social recovery (the role(s) someone fulfils in society), and personal recovery (living a satisfying and meaningful life) (Anthony, 1993; Davidson & Roe, 2007; Leamy et al., 2011). Taking a closer look into positive health and recovery concepts shows that both perspectives focus on a comparable multidimensional construct of health including the ability to adapt to changing conditions (Anthony, 1993; Davidson & Roe, 2007; Huber et al., 2011; Leamy et al., 2011).
To apply these perspectives of health in daily living and practice dialogue tools were developed. The My Positive Health (MPH) dialogue tool was developed to help citizens, patients and healthcare professionals to guide their conversation, to self-reflect and to encourage the person in question to make personalised health improvements. MPH consists of 42 items about health based on information emerged from a large study including interviews about what health is to citizens, patients and other stakeholders (Huber et al., 2016). It visually presents health in six dimensions: bodily functions, mental wellbeing, meaningfulness, quality of life, social and societal participation and daily functioning (Huber et al., 2016; Institute for positive health, n.d.). For recovery, the Individual Recovery Outcomes Counter (I.ROC) dialogue tool was developed, specifically for mental health related consultations (Ion et al., 2013; Monger et al., 2013). I.ROC also aims to support the conversation about broad health, self-reflection, and to make personalised recovery improvements (Rudd et al., 2020). The I.ROC consists of 12 questions visually presented in 4 domains; home, opportunity, people and empowerment (Ion et al., 2013; Monger et al., 2013). Looking at the underlying aspects representing health in the MPH and I.ROC dialogue tools it can be concluded that the content of these dialogue tools is also almost the same. Both are used as self-reported questionnaires in preparation for, or as guide during, the conversation with healthcare professionals.
To facilitate the evaluation of working with this positive health approach comprehensible quantitative outcome measures are needed. The MPH dialogue tool and its dimensions were not developed to measure positive health but to facilitate the conversation about positive health (Huber et al., 2016). Earlier research about measuring positive health revealed concerns about relevance, comprehensiveness and comprehensibility of the positive health model (Prinsen & Terwee, 2019). Subsequently, a shortened measurement scale, consisting of 17 out of the 42 items of the dialogue tool (PH17) was developed (Van Vliet et al., 2021). Although fit indices were adequate (Doornenbal et al., 2021), caution should be taken when generalising these results, because the original study of van Vliet et al. (Van Vliet et al., 2021) in which the PH17 was developed was based on research among citizens in just one part of the Netherlands, an eastern Dutch region, and the response rate was very low (25%) (Doornenbal et al., 2021; Van Vliet et al., 2021). In contrast to the MPH, the I.ROC is already further developed for measuring purposes. Studies for the I.ROC as measurement tool among different mental healthcare populations showed good internal consistency, test-retest reliability and concurrent validity, but with differing factor structures (Beckers et al., 2022; Dickens et al., 2017; Monger et al., 2013; Roze et al., 2020; Rudd, 2018; Rudd et al., 2020; Sportel et al., 2023). However, neither for MPH or I.ROC research at measurement properties has been conducted in a representative general population as yet.
Because of the similarities between both tools, it was hypothesised that they might both serve as a measurement tool for positive health. As a first step to develop such a generic measurement tool for positive health this study assessed measurement properties including factor structures and internal consistencies of both the MPH and I.ROC dialogue tools among a large Dutch cohort representative for a general population. It was aimed to identify if these dialogue tools measure multidimensional aspects of positive health which can be summarised in sum scores. Additionally, coherence between the MPH and the I.ROC was assessed.
METHODS
Study design
An observational cross-sectional research design was used to evaluate factor structures and internal consistency of the MPH and I.ROC. Correlation between the MPH and the I.ROC dialogue tools was calculated in a representative cohort from the general Dutch population, the Longitudinal Internet Studies for the Social sciences (LISS) panel.
Participants
The LISS panel (http://www.lissdata.nl) is a non-commercial online longitudinal research panel, operational since 2007 and includes a representative sample of approximately 7,000 individuals from approximately 5,000 households from the Dutch general population. The panel includes both healthy people and people with health conditions, all living at home. The panel was carefully composed based on a true probability distribution drawn from the population register by Statistics Netherlands (Scherpenzeel & Das, 2010). To become a LISS panel member, at least one person in the household should master the Dutch language. To minimise selection bias, households were provided with a computer and internet connection if they could not otherwise participate.
Data collection
LISS panel members complete monthly online questionnaires and are paid for each completed questionnaire. Response rates for this panel are high. For example, 5,714 (83.6%) LISS panel members completed a LISS core study module about health in November 2020 (https://www.lissdata.nl/research/liss-core-study). For this study, the selected study population was asked to additionally fill out the MPH and I.ROC dialogue tool questionnaires. Rule of thumb for sample size calculation is four to ten respondents per item of a questionnaire with a minimum of 100 patients according to de Vet et al. (2018). A study population of 2,500 respondents is adequate to assess factor structures of the data and their goodness of fit in two separate groups (n=1250). To achieve the required number (n=2500) counting on a response rate of 75%, 3,246 adults (≥18 years, one per household) of the LISS panel were randomly selected.
Dialogue tools MPH and I.ROC
The MPH dialogue tool consists of 42 statements scored on an 11-point scale ranging from 0 ‘totally disagree’ to 10 ‘totally agree’. The I.ROC dialogue tool consists of 12 items on a six-point scale from 1 ‘never’ to 6 ‘all the time’. Higher scores represent better health (see Supplementary tables 1 and 2). In this study both dialogue tools are used as self-reported questionnaires.
Evaluation of measurement properties/analytical plan
First, exploratory factor analysis (EFA) was used to explore factor structures and generate sum scores per factor. For this purpose, the extraction method principal component analysis (PCA) was applied. PCA (also called exploratory factor analysis) is a statistical technique used for analysing the interrelations among a set of variables (or questions in the questionnaire) and for explaining these interrelations in terms of a reduced number of variables, called factors (de Vet et al., 2018; Floyd & Widaman, 1995; Velicer & Jackson, 1990). Next, to assess if the extracted factor structure also holds in another group, confirmatory factor analysis (CFA) was applied in another part of the sample. The dataset was randomly split; group 1 for EFA and group 2 for CFA. Last, per factor internal consistency was explored, and coherence between the total scores of both tools was assessed. Statistical analyses were performed using software package IBM SPSS version 27, except for CFA which was analysed with IBM SPSS Amos version 27.
Exploratory factor analysis
Step 1: associations between all items were assessed with Kaiser-Meyer-Olkin (KMO) and Bartlett’s test correlation in data group 1; these should be >0.5 with significance level <0.05 to have sufficient correlations for factor analysis. Items that do not correlate with any other item (<0.2) were discussed for its content. Item correlations of >0.9 might measure the same thing. Items with low or high correlations (<0.2 and >0.9) could be retained or deleted, depending on the qualitive content discussion about the importance for the construct to be measured.
Step 2: the number of factors was identified with Oblimin with Kaiser Normalisation rotation, because items and factors were not expected to be independent from each other. The number of factors was determined with a criterion eigenvalue >1 and inspection of the scree plot for confirmation. Highest factor loadings of items were used as guidance to divide the items over the factors with a minimum threshold of 0.32 (21). Factor loadings of >0.71 are considered excellent, >0.63 very good, >0.55 good and >0.45 fair (Tabachnick & Fidell, 2007). Since our aim was not to reduce items but to assess structures and to generate sum scores, we chose the relatively low cut-off point for factor loadings of >0.32 for including items (Tabachnick & Fidell, 2007).
Step 3: item content and factor structure were discussed by the research team. The items were assigned to the factors on which they loaded the highest, unless an item conceptually fitted better in another factor according to the research team; then they were relocated.
Step 4: inter-item correlations per factor were assessed; values should be between 0.2 and 0.5. Inter-item correlations greater than 0.7 suggest that they may measure the same thing (de Vet et al., 2018).
Confirmatory factor analysis
Step 5: Extracted factor structures were assessed for goodness of fit using CFA. Maximum likelihood (ML) was used as estimation method. Goodness of fit indices included chi-square (X2). A non-significant X2 is desirable, however in a large sample, the X2 is usually significant. Comparative fit index (CFI) and Root mean square error of approximation (RMSEA) were therefore also used. CFI values between 0.90 and 0.95 are indicators of acceptable fit and >0.95 indicates superior model fit (Byrne, 2010; Hu & Bentler, 1999). RMSEA values <0.05 represent good fit, 0.05-0.08 acceptable fit, >0.08 medium fit and >0.1 poor fit (Byrne, 2010; Hu & Bentler, 1999).
Internal consistency
Step 6: Cronbach’s alphas were calculated to further explore internal consistency; values of factors between 0.7 and 0.9 with a minimum of 3 items were considered good (de Vet et al., 2018).
Total scores and sum scores per factor
Step 7: To describe the distribution of the data, mean, median, standard deviation, minimum, maximum, skewness and kurtosis (< -1 and < 1) of the factor scores and total scores were calculated. If ≥ 15% of the respondents scored lowest possible scores this suggests that floor effects exists (de Vet et al., 2018). If ≥ 15% of the respondents scored highest possible scores this suggests ceiling effects (de Vet et al., 2018).
Correlation between total scores
Step 8: Correlation between the total scores of MPH and I.ROC was calculated with Spearman’s correlation coefficient to assess if coherence exists between the tools. A value >0.4 suggests moderate to strong correlation (de Vet et al., 2018).
Ethical considerations
The study was conducted in accordance with current public regulations, laws, and the principles of the Declaration of Helsinki. Ethical approval was given by the METC Brabant (Tilburg, the Netherlands, study number NW2024-15). Informed consent was given by each participant to be included as a LISS panel member and for each monthly questionnaire (Scherpenzeel & Das, 2010). The General Data Protection Regulation (GDPR) was followed. All data from the LISS panel are anonymised and open access available for researchers and policymakers upon request.
RESULTS
Characteristics of the respondents
The response rate was high (76%) with 777 respondents not responding. Twelve respondents did not fill out the questionnaires completely and this small number was excluded, leaving 2,457 respondents for the analyses.
Characteristics of the included respondents are shown in Table 1. The response group (n=2,457) was randomly split in group 1 for EFA (n=1,199) and group 2 for CFA (n=1,258) as described in the Methods section.
My Positive Health (MPH)
Exploratory factor analysis (EFA)
Step 1: KMO and Bartlett’s test was statistically significant (0.97; p <=.01). Item correlations showed that item 3 (having complaints or pain) of the MPH dimension Bodily Functions (BF) did not correlate (<0.2) with 15 items from the MPH dimensions Mental Wellbeing (MW), Meaningfulness (MF), Quality of Life (QL), Social and societal Participation (SP) and Daily Functioning (DF): MW10, MW13, MF21, QL27, QL28, SP29, SP31, SP32, SP33, SP35, DF36, DF37, DF40, DF41, DF42 (for content of these items see Table 2). Based on relevance of the content and its correlation with other items, item BF3 was not deleted.
Step 2: the extraction method PCA and Oblimin rotation revealed a six-factor structure with an eigenvalue >1 explaining 68.1% of the variance. Factor loadings ranged from 0.36 to 0.94. After careful consideration within the research team, the factors were named: ‘Acceptance, meaningfulness and satisfaction with life’, ‘Physical health and functioning’, ‘Self-management’, ‘Social network and societal roles’, ‘Personal development’ and ‘Cognition’. For details of factors, items and factor loadings see Table 2.
Step 3: the research team discussed item distribution among the factors on its content; no changes were made based on the discussion.
Step 4: inter-item correlations showed that 48.1% of the correlations within the factor ‘Acceptance, meaningfulness and satisfaction with life’, 14.3% within the factor ‘Physical health and functioning’, 4.8% within the factor ‘Self-management’, 25% within the factor ‘Social network and societal roles’, none within the factor ‘Personal development’ and 33.3% within the factor ‘Cognition were >0.7 (For more details see Supplementary tables 3 to 8.).
Confirmatory factor analysis
Step 5: the extracted six-factor structure had an acceptable fit in CFA with significant X2 (5290.328; degrees of freedom 790; p <= 0.01), CFI of 0.90 and RMSEA of 0.067 with a 90% confidence interval of 0.066 to 0.069.
Internal consistency of MPH
Step 6: Cronbach’s alphas of the extracted factors ranged from 0.74 to 0.97; 0.969 (Acceptance, meaningfulness and satisfaction with life), 0.866 (Physical health and functioning), 0.860 (Self-management), 0.923 (Social network and societal roles), 0.742 (Personal development) and 0.801 (Cognition).
Sum scores per factor and total score of MPH
Step 7: Results are shown in Table 3. No floor or ceiling effect was present on the factors or total score; no more than 8.3% of the respondents scored the highest possible score and none of the respondents scored the lowest possible score (data not shown).
Individual Recovery Outcomes Counter (I.ROC)
Exploratory factor analysis
Step 1: KMO and Bartlett’s test was significant (0.92; p <=.01). Item correlations showed that only item 8 (Social network) did not correlate (<0.2) with items 2 (life skills) and 3 (safety and comfort). Based on relevance of the content and its correlation with other items, item PE8 was not deleted.
Step 2: the extraction method PCA with Oblimin rotation revealed a two-factor structure with an eigenvalue >1 explaining 56.1% of the variance. Factor loadings ranged from 0.37 to 0.87. After careful consideration within the research team, the factors were named: ‘Wellbeing, control, network and meaningfulness’ and ‘Health, safety and abilities’ (Table 4).
Step 3: the research team concurred on item distribution for 11 items. Item 5 (exercise and activity) loaded almost equally on both factors. This item was relocated to the factor ‘Health, safety and abilities’ following discussion about the content of the I.ROC dimensions and related items. For details of factors, items and factor loadings see Table 4.
Step 4: for both factors all inter-item correlations were <0.7 (see Supplementary tables 9 and 10).
Confirmatory factor analysis
Step 5: the two-factor structure extracted from EFA had an acceptable fit in CFA with significant X2 (321.503; degrees of freedom 47; p <= 0.01), CFI of 0.96 and RMSEA of 0.068 with a 90% confidence interval of 0.061 to 0.075.
Internal consistency of I.ROC
Step 6: Cronbach’s alphas of the extracted factors were 0.87 (Wellbeing, control, network and meaningfulness) and 0.73 (Health, safety and abilities).
Sum scores per factor and total scores of I.ROC
Step 7: Results are shown in Table 5. No more than 9.3% of the respondents scored the highest score on one of the factors or the total score and no respondents scored the lowest score (data not shown).
Correlation between total scores of MPH and I.ROC
Step 8: Spearman correlation coefficient between total scores of MPH and I.ROC was 0.77 and significant (P<=0.01).
DISCUSSION
This study assessed whether the original items of the dialogue tools MPH and I.ROC can be used for measuring multiple aspects of positive health summarised in sum scores among a general (Dutch) population. For the MPH tool, factor analysis resulted in a six-factor structure including all 42 items and for the I.ROC tool in a two-factor structure including all 12 items. For both tools, CFA confirmed acceptable fit to the data. Internal consistency was good, except for two of the extracted factors of the MPH tool in which internal consistency was too high (>0.9). The total scores of the tools were highly correlated (0.77). To differentiate the original dialogue tools MPH and I.ROC from the measurement tools, these measurement tools (questionnaires) are further called the PH42 and I.ROC12. These results imply important points for consideration as described below.
It is important to realise that different methodological approaches are used to identify dimensions depending on the aim of the instrument; i.e., whether it is meant to be a dialogue tool or a quantitative measurement tool. These different aims may lead to different results. The study of Huber et al. (2016) intended to get more insight in what health meant for different stakeholder groups (patients, practitioners, citizens and policymakers) using a qualitative research design. The indicators of health that emerged from this study led to the operationalisation of the concept of positive health (Huber et al., 2016). The concept was further elaborated and an MPH dialogue tool including six dimensions was developed, aiming to support the conversation about someone’s health in daily practice from a positive health perspective as well as to support self-evaluation (Huber et al., 2016). The aim of our study was to evaluate the items of the MPH and I.ROC dialogue tools for quantitative measuring purposes among a general population by assessing their factor structure and generating factor sum scores. In addition, the aim was not to reduce items, as long as the items did not hamper a clear interpretation. Therefore, the more conservative cut-off point for factor loadings of >0.32 (Tabachnick & Fidell, 2007) instead of the >0.5 cut-off point aiming to reduce items (de Vet et al., 2018), was chosen. Compared to the MPH dialogue tool, we identified differences in the division of items over the dimensions of the investigated PH42 measurement tool. Both the MPH dialogue tool and the PH42 measurement tool correspond to the positive health construct. The results showed that the measurement tool is still multidimensional while no items (and thus aspects of positive health) were deleted. Similar issues were seen for the I.ROC dialogue tool. Initially, the four dimensions of the I.ROC dialogue tool were also developed for visual presentation and to guide the conversation (Monger et al., 2012). In the current study differences in the division of items over two dimensions of the I.ROC12 measurement tool were identified. For quantitative measuring purposes the dimensions retrieved from our study are the best fit.
The factor structure found for I.ROC12 in our general population did not correspond with the results from earlier research aiming to develop a measurement tool based on the I.ROC for a mental healthcare population. While our study resulted in two factors, these studies resulted in one up to three factors (Beckers et al., 2022; Dickens et al., 2017; Monger et al., 2013; Rudd et al., 2020). These differences can most likely be explained by the differences in the study populations, because perceptions and values of aspects of health may differ between a mental healthcare population and the general population (van Druten et al., 2022). Furthermore, cross-cultural measurement invariance between our Dutch population and the Scottish populations, in which the I.ROC was developed and first validated, may be present indicating that comparable groups of respondents among different cultures score differently on a questionnaire (de Vet et al., 2018). Also, while this study was being performed, the COVID-19 pandemic struck, which can have led to different scores particularly on the questions about social network and activities. However, the PH17 which was developed pre-COVID showed a factor structure comparable to the PH42 (Van Vliet et al., 2021). In contrast to the I.ROC12, for the PH42, no research into factor structures in other than a general population has been conducted so far. If the PH42 and I.ROC12 will be used in other populations, more research is needed to assess if the found factor structures for the PH42 and the I.ROC12 also hold.
Although the PH42 seems useful for measurement purposes, the results imply that future research into item reduction is advisable. Two factors ‘Acceptance, meaningfulness and satisfaction with life’ and ‘Social network and societal roles’ had high internal consistency (>0.9) meaning that a redundancy of items is present; there was also a high amount of inter-item correlations (>0.7) in these factors (48.7% and 25%), meaning that some items might measure the same thing. Furthermore, it should be stressed that PH42 includes items that loaded substantially (>0.3) on more than one factor (6/42). These items were judged not to hamper a clear interpretation of the factor and were therefore retained. These results imply added value of item reduction for the PH42 to develop a clarified, comprehensible and comprehensive measurement tool.
We conclude the PH42 to be the first choice for quantitively measuring the multidimensional concept of positive health in a general population. The I.ROC12 – although developed for a different purpose (recovery) – is a potential alternative. We found a high coherence between the total scores of the tools supporting that they measure a comparable construct. The I.ROC12 is shorter, however, the PH42 gives more information and the explained variance of the PH42 is higher. The results of our study offer some support for the use of the PH17 (Van Vliet et al., 2021) as a shorter alternative: all except one item of the dimensions of the PH17 showed overlap with the dimensions of the PH42. However, Van Vliet et al. (2021) did not apply the preferred methodological approach for item reduction as published by de Vet et al. (2018), when developing the PH17.
This approach implies item reduction based on repeated factor analyses after removal of each item including judgement of factor loadings and inter-item correlations, content discussion, and maintaining acceptable internal consistencies. As a next step in the validation process of both the PH42 and I.ROC12 in a general population, assessment of construct validity in more depth is recommended.
This study has some limitations. A minor limitation of the selection procedure was that at least one member of the household had to master the Dutch language, whereby migrants were relatively underrepresented. Also, the members of the participating households had to be able to use a computer, which may have led to some selection bias. However, this study is robust in terms of its large sample size, the high response rate and the representativeness of the general Dutch population (Scherpenzeel & Das, 2010).
CONCLUSION
The results of this study suggest that both the PH42 and the I.ROC12 are useful to quantitatively measure aspects of positive health which can be summarised in sum scores in a general population. For both tools, the dimensions and item division differed from the original dialogue tools. This means that for quantitative measurement purposes the dimensions found in our study should be used instead of the dimensions of the dialogue tools. The PH42 is the first choice, however the I.ROC12 is an adequate alternative. For the PH42, further research into item reduction is recommended. For both tools, future research should focus on assessing construct validity (in a general population) in more depth.
Data Availability
All data from the LISS panel are anonymised and available upon request for researchers and policymakers. For more information see: https://www.lissdata.nl/access-data.
Footnotes
STATEMENTS AND DECLARATIONS
Funding: The study was funded by the Jeroen Bosch Hospital, ‘s-Hertogenbosch, the Netherlands.
Conflicts of interests: MvV co-developed the MPG and worked at the Institute for Positive Health at the start of this study. BR previously worked at Penumbra Scotland, where the I.ROC was developed.
Ethical approval: The study was conducted in accordance with current public regulations, laws, and the principles of the Declaration of Helsinki. Informed consent was given by each participant to be included as a LISS-panel member.
Data availability: All data from the LISS panel are anonymised and available upon request for researchers and policymakers. For more information see: https://www.lissdata.nl/access-data.
REFERENCES
