Abstract
Purpose The 6-item Vision-related Quality of life and Limitations questionnaire (VQL-6) is a screening instrument that signals a need for additional low vision care in patients with chronic ophthalmic diseases. This study aims to (1) determine the optimal scoring method for the VQL-6 and (2) evaluate its discriminative capacity for clinical use.
Methods Patients with glaucoma and retina– and cornea-related disorders completed the VQL-6 and participated in an interview designed to distinguish between patients with and without a need of additional care. For the two subscales (general health and quality of life, HQOL, and vision-related limitations, VL) we compared a scoring model with equal item weights to a cross-validated model using regression weights. The optimal scoring method and discriminative capacity of the subscales were determined using receiver operating characteristics (ROC) analysis, with the interview outcome as gold standard. Sensitivities were evaluated at specificities of 90 and 95%.
Results In 297 interview assessments, 96 patients (32%) appeared to need additional care. Both scoring models demonstrated very similar area under the ROC curves (AUC). The selected equal weight model yielded an AUC of 0.91 (95% confidence interval 0.87-0.94) for VL and 0.71 (0.65-0.77) for HQOL. For VL, sensitivity was 72 and 64% at 90 and 95% specificity, respectively. Corresponding HQOL sensitivities were 34 and 17%.
Conclusions The subscale VL shows excellent discriminative capacity and outperformed HQOL. Future studies should explore the feasibility of the VQL-6 in clinical practice.
Introduction
Visual impairments significantly impact an individual’s quality of life, affecting daily activities and psychological well-being.1–5 Additional low vision care, such as prescription and training of visual aids, visual rehabilitation, or medical social work, can improve overall quality of life and daily functioning in people with visual impairments.6,7 Yet, it is reported in literature that people are often referred too late to low vision services.8–12
While quality of life is of key interest in low vision care settings, the main focus in ophthalmic clinics is on the assessment of visual functions for clinical decision making and referrals. These measures, however, may overlook the nonvisual factors that influence quality of life and rehabilitative needs.13–15 Moreover, visual rehabilitation is seen as a last resort by ophthalmologists,16 and quality of life is infrequently discussed during consultations.17 Consequently, reduced (vision-related) quality of life and a need for additional low vision care in ophthalmic patients may be overlooked, delaying timely referrals.
Questionnaires may be a useful tool to improve patient-clinician communication,18 aiding in early detection of reduced quality of life and improving timely referrals.19,20 However, existing questionnaires often lack validation for screening purposes or are too time-consuming in clinical practice. To address this gap, we developed the 6-item Vision related Quality of Life and Limitations questionnaire (VQL-6). The VQL-6 is a short questionnaire that measures general health and quality of life, and vision-related limitations, aiming to signal a need for additional low vision care (referred to as ‘additional care’ from hereon) in patients with chronic ophthalmic diseases. In previous work, the VQL-6 demonstrated good psychometric properties in a large Dutch population-based sample and in patients with glaucoma and macular degeneration.21 Factor analyses demonstrated a robust two-factor structure, resulting in two subscales: general health and quality of life (HQOL), and vision-related limitations (VL).
The aim of the present study is to (1) determine the optimal scoring method of the VQL-6 and (2) subsequently evaluate its discriminative capacity for clinical use. For this purpose, patients with glaucoma and retina– and cornea-related disorders completed the VQL-6 and took part in a semi-structured interview. The interview was designed to distinguish between patients with and without a need of additional care. We determined the optimal scoring method and the discriminative capacity of the VQL-6 subscales in terms of Receiver Operating Characteristics (ROC) analysis, with the interview outcome as gold standard.
Methods
Study population
Participants were consecutive, regular visitors of the outpatient department of Ophthalmology of the University Medical Center Groningen (UMCG) with a scheduled appointment at the cornea, glaucoma, or anti-VEGF injection consultation hour. Patients were eligible to participate if they were 18 years or older, had Dutch as their primary language, and were diagnosed with one or more of the following chronic conditions: glaucoma or retina-or cornea-related disease. The disease had to persist for at least one year and participants had to be under active treatment of an ophthalmologist. We excluded patients who had undergone ophthalmic surgery within the three months preceding their appointment, in order to avoid confounding by temporary changes in visual function.
The ethics board of the UMCG approved the study protocol (#201800249). All participants provided written informed consent. The study followed the tenets of the Declaration of Helsinki.
Data collection
Eligible patients were asked by the researcher (VLD) to complete the VQL-6 in the waiting room immediately before or after their scheduled appointment. If patients faced challenges due to low vision or other physical limitations, assistance could be offered, e.g., by the person who accompanied the patient. All participants were called by the same researcher within a week’s notice to partake in the interview. The telephone interview took approximately 15 minutes and was recorded with an audio recorder. Responses of the patients, along with additional comments from the researcher, were documented on an interview template. Patient characteristics were collected from their medical files, including age, gender, ophthalmic diagnosis, visual acuity, and perimetry results.
Materials
The 6-item Vision-related Quality of life and Limitations (VQL-6)
Details of the VQL-6 have been described before.21 The VQL-6 comprises six items (see Supplementary S1 for the Dutch questionnaire; see Supplementary S2 for an English translation). On item 1, 2, and 3, respondents are asked to indicate, on a scale of 0 to 10, their (1) general health, (2) general quality of life, and (3) the extent of experienced limitations by their visual impairment. The subsequent items 4, 5, and 6 measured whether the respondent, because of their visual impairment, (4) worries about the future, (5) feels like he/she accomplishes less, and (6) feels dependent on others. Items 4, 5, and 6 were scored on a 5-point Likert scale with answer options ‘never’, ‘rarely’, ‘sometimes’, ‘often’, and ‘always. The VQL-6 generates one subscale related to general health and quality of life (HQOL; item 1 and item 2), and a vision-specific subscale measuring the extent of vision-related limitations (VL; item 3 to item 6). Respondents were instructed to answer the questions based on their situation in the past month while considering their use of glasses or contact lenses, if applicable. For analysis, all VQL-6 items were converted to a 0 to 10 scale (by transforming the 5-point Likert scale answer options to 0, 2.5, 5, 7.5, 10). In addition, all items were transformed so that a higher score represents better functioning to enable a fair comparison between items.
Interview
A semi-structured interview with 7 open-ended questions was constructed to assess whether a patient may need additional care and served as a gold standard for the evaluation of the VLQ-6. The first six interview questions were formed by reconstructing the six closed-ended questions of the VQL-6 into open-ended interview questions. Similar to the VQL-6, respondents were instructed to answer the questions based on their situation in the past month. Based on the responses, a comprehensive insight was attained into the general health and quality of life, and the extent of the vision-related limitations experienced by the patients. Interview Question 7 was designed to assess whether the patients themselves indicated a need for additional care. The possibilities (including low vision aids, visual rehabilitation, medical social work, training, and practical advice) were explained to the patients so that they could form an informed opinion. The interview outcome had three scoring options (need for additional care/no need for additional care/uncertain). The three scoring options were later dichotomized into ‘need for additional care’ or ‘no need for additional care’ for analyses, as described below. Scoring of the interview was based on the researcher’s evaluation of the patient’s responses to the first six interview questions. The patient’s self-perceived need for care was also taken into account. For all interview questions, patients were asked to explain their answer and give examples.
To ensure reliability of the interview, we assessed the interrater reliability of the main interview outcome by having a clinical expert (PP, a professional who performs intake interviews at a visual rehabilitation center) score a random sample of 60 interview audio recordings (with scoring options ‘need for additional care’/’no need for additional care’/’uncertain’), covering 20 cornea patients, 20 retina patients, and 20 glaucoma patients. Cases within this sample that were scored differently by the researcher and clinical expert were discussed until a consensus was reached. In addition, to dichotomize the main interview outcome into ‘need for additional care’ or ‘no need for additional care’, audio recordings of all cases of the total sample that were categorized by the researcher as ‘uncertain’ on Question 7 were discussed between the researcher and clinical expert and reevaluated.
Data analysis
Data analysis was performed using RStudio (version 2023.12.0).22 Data was checked for normality and linearity; if the assumptions were violated, non-parametric tests were performed. A p-value of 0.05 or less was considered statistically significant.
Descriptives
To evaluate results across different ophthalmic conditions, all data was categorized into four ophthalmic conditions: retina (e.g., macular degeneration, diabetic retinopathy, or retinal vein occlusion), glaucoma, cornea (e.g., keratoconus, corneal infection, corneal dystrophy), and comorbid. The comorbid category included patients with a combined retina, cornea, or glaucoma diagnosis. We did not consider other ocular comorbidities, such as cataract, amblyopia or myopia, in our categorization, that is, for example, a glaucoma patient with myopia was classified in the glaucoma category. The participant characteristics were described with mean and standard deviation (SD). For patients with visual field test results available (the glaucoma patients, including those within the comorbid group), the integrated visual field (IVF) score was calculated from the monocular visual field test results extracted from the HFA printouts, using the ‘best sensitivity’ method.23,24 Age, gender, main interview outcomes, and VQL-6 subscale sum scores were compared between the patient groups using Kruskal-Wallis test and chi-square (χ²) tests.
For the interview, descriptive data was generated for the interview outcome. Cohen’s kappa with linear weighting was established as a measure of interrater reliability of the interview (applied to the 3×3 table with the options ‘need for care’/’no need for care’/’uncertain’). Values < 0 indicate agreement worse than chance, 0–0.20 none to slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1.00 almost perfect agreement.25 VQL-6 subscale scores between participants with and without a need for additional care based on the interview were compared with Wilcoxon rank sum tests.
Scoring method and discriminative capacity
To determine the optimal scoring method of the VQL-6 subscales, we compared two models. First, we evaluated an equal weight model, which was determined on the a priori assumption that all items have equal weight in predicting a need for additional care, signifying simple sum scores for each subscale. Second, we evaluated a cross-validated fitted model – a competing model that was fitted to our dataset – where each item was weighted according to its contribution in predicting a need for additional care. For the fitted model, we first performed a binary logistic regression for both subscales, with the subscale item scores as independent variables and the interview outcome as a dependent variable. Weighted subscale scores were calculated using the Beta values of the regression formula giving weight to each item. ROC analyses were performed for the equal weight model and the fitted model, with the interview outcome as gold standard, employing the pROC package in R.26 Considering the fitted model was built and tested in the same dataset, we performed a 10-fold cross validation using the ‘caret’ package in R,27 to estimate how well the ROC model would perform in an independent dataset28 Mean and standard deviation of the 10 evaluations represented the generalized performance and standard error of the fitted model.
To evaluate which model performed best, the AUC values of the equal weight model and the fitted model were compared for significant differences, using bootstrap tests for correlated AUC values.29 Within each model, we determined the best performing VQL-6 subscale by comparing AUC values employing DeLong’s test.30,31 AUC values were interpreted according to the following criteria: AUC < 0.7 low, 0.7 to 0.9 acceptable, and ≥ 0.90 excellent.32 Using the best performing model, sensitivities were evaluated at predefined specificity values of 90% and 95%.
Finally, we performed ROC analyses to evaluate the capacity of visual functions in predicting a need for additional care. AUC values were calculated and compared with the VQL-6 subscales, for visual acuity (in logMAR) of the better eye for all patients and for HFA MD of the better eye and the integrated visual field (IVF) for the glaucoma patients and comorbid patients with glaucoma.
Results
Descriptives
Table 1 presents the participant characteristics. In total, 322 patients agreed to participate. One participant was excluded based on missing VQL-6 responses. In addition, 24 participants were excluded as they did not complete the interview for various reasons, including no registered phone number, did not pick up the phone, or not being able to partake due to bad hearing or poor cognitive capabilities. After exclusion, 297 participants were left for data analysis. Included patients were categorized according to their ophthalmic diagnosis, resulting in 96 retina, 60 glaucoma, 81 cornea, and 60 comorbid patients. There was a significant difference between patient groups with respect to age (p <.001). No significant difference was found for gender (p = 0.51).
Table 2 shows the median sum scores of the VQL-6 subscales, stratified per interview outcome and per patient group. Of all 297 patients, 96 (32%) were evaluated to be in need of additional care based on the interview. Using the data from Table 2 multiple comparisons were made, both between interview outcomes (horizontally) and between different patient groups (vertically). Significant differences were found for both subscales per interview outcome, with patients with a need for additional care scoring significantly lower on HQOL and VL than patients without a need for additional care (both p < 0.001). This was the case for all patients together and for the individual patient groups separately (all p < 0.001). Between the patient groups, no significant differences were found in the HQOL scores (p = 0.23), but a significant difference was found in the VL scores (p = 0.008). Post-hoc Dunn tests demonstrated significantly lower VL scores for cornea patients, compared to glaucoma (p = 0.04) and retina (p = 0.04) patients. Additionally, no significant differences were found in the main interview outcome between the patient groups (p = 0.33). Therefore, the patient groups were merged, and all additional analyses were performed on the group as a whole.
Scoring method and discriminative capacity
In our total sample, the binary logistic regression analysis of HQOL and VL resulted in the following classification equations:
and
Weighted subscale scores were calculated and a 10-fold cross validation was performed to calculate the generalized performance of the fitted model. Figure 1 shows the ROC curves and AUC values of the equal weight model and the fitted model, for both subscales. For comparison, we added an ROC curve and AUC value for the visual acuity of the better eye. HQOL showed acceptable discriminative capacity in both models (AUC equal weight [95% CI] = 0.71 [0.65 – 0.77]; AUC fitted [95% CI] = 0.71 [0.65 – 0.77]). VL demonstrated excellent discriminative capacity (AUC equal weight [95% CI] = 0.91 [0.87 – 0.94]; AUC fitted [95% CI] = 0.92 [0.89 –0.95]). A comparison between subscales demonstrated a significantly higher AUC value of VL compared to HQOL, using the equal weight model (Z = 6.45, p < .001) and the fitted model (Z = 6.86, p < .001). Between the equal weight and the fitted model, no significant differences were found for HQOL (D = 0.13, p = 0.90) and VL (D = 0.46, p = 0.65). Therefore, following parsimony, we employed the equal weight model for the evaluation of sensitivity and specificity, and the subsequent analysis.
Table 3 shows sensitivities and specificities (specificity starting from 70%) for potential cutoff sum scores of the VL subscale. At a predefined specificity level closest to 90% (91%), a sensitivity of 72% was achieved, with a cutoff sum score of ≤ 17.75. At a specificity level closest to 95% (95.5%), a sensitivity of 64% was found, with a cutoff sum score of ≤ 15.75. For HQOL sensitivity levels of 34% and 17% were found for specificities of 90 or 95%, respectively. Given this poor performance of HQOL, we did not tabulate potential cutoff scores.
As shown in Figure 1, logMAR (of the better eye) demonstrated an AUC of 0.76 (95% CI: 0.70 – 0.82). VL demonstrated a significantly higher AUC value (D = – 4.07, p < .001) compared to logMAR. No significant difference was found between logMAR and the AUC value HQOL (D = 1.32, p = 0.19). Figure 2 depicts the ROC curves and AUC values of both subscales, compared to HFA MD of the better eye and the integrated visual field (IVF) for the glaucoma patients and comorbid patients with glaucoma. HFA MD (of the better eye) yielded an AUC of 0.77 (95% CI: 0.63 – 0.90), which was not significantly different from HQOL (Z = 0.79, p = 0.43) and significantly lower than VL (Z = –2.50, p = 0.01). Integrated visual field (IVF) measures demonstrated an AUC of 0.81 (95% CI: 0.68 – 0.95), with no significant difference compared to HQOL (Z = 1.75, p = 0.08) and VL (Z = –1.95, p = 0.052).
Discussion
This study determined the optimal scoring method and discriminative capacity of the VQL-6 subscales in patients with chronic ophthalmic diseases. Through interview assessments, 32% of the patients were evaluated to be in need of additional care. A scoring model with equal weight items and a cross-validated fitted model using regression weights demonstrated a very similar AUC for predicting a need for additional care. Following parsimony, we selected the equal weight model, and the VL subscale showed excellent capacity in predicting a need for additional care. VL outperformed the HQOL subscale, which showed acceptable discriminative capacity, and visual acuity and visual field measures, which revealed good discriminative capacity.
We opted for a scoring method employing equal weighted items instead of weighted items for calculating subscale scores, as both methods resulted in a very similar discriminative capacity of the VQL-6. These results agree with prior studies that have compared equal weighted sum scores with weighted scores derived from item response theory methods or factor analyses.33–35 Similarly, studies have demonstrated a comparable psychometric basis of sum scores compared to weighted scores, when the dimensional structure of items is verified.33,36,37
We evaluated sensitivity at clinically meaningful specificity levels of 90% and 95%. A specificity lower than these levels would result in a larger number of false positives, which would unnecessarily burden scarce health care resources. HQOL did not reach sufficient levels of combined sensitivity and specificity. For VL, a specificity of 91% (the nearest possible value to 90%), resulted in sufficient sensitivity of 72%. A specificity of 95% resulted in a significant drop of sensitivity to 64%, meaning a significant proportion of patients that may need additional care would be missed, which is not deemed preferable for a screening instrument. In the specific setting of our study, a tertiary referral clinic, a cutoff at a specificity of 91%, would imply a positive predictive value of 79% and a negative predictive value of 87%. Presumably, these numbers may be different in a general hospital, where the prevalence of visual impairment and experienced vision-related limitations might be lower than in our setting – in such a setting it might be better to opt for a higher specificity.
To our knowledge, sensitivity and specificity of other questionnaires in predicting a need for additional care have not been reported. Whilst some studies reported the sensitivity and specificity of questionnaires in predicting measurements of visual functions or discriminating between patients and controls,38,39 due to the use of different reference standards, direct comparison with our results is not possible. Guo et al developed an Electronic Medical Record (EMR)-based tool for ophthalmic patients in need of referral to low vision services that fires an alert based on visual acuity and diagnosis criteria.40 They reported sensitivity and specificity based on when the firing of the alert should have been suppressed or when firing criteria were not met,40 which limits direct comparison to our results. Furthermore, the use of visual acuity-based criteria as a reference standard may not be optimal, as our study’s findings confirm that visual acuity is less accurate in predicting a need for additional care.
Our study revealed the highest capacity of self-reported vision-related limitations in predicting a need for additional care, measured by the subscale VL. Visual acuity was an important predictor as well, yet it was inferior to VL. This is similar to prior research that showed that various factors next to visual acuity were significant predictors of rehabilitation needs, including physical and mental health, comorbidity and dependence of others.6,13 While some studies, including O’Connor et al,13 have identified visual acuity as the most important predictor (contrasting with our findings), others did not find a significant association between the prevalence of rehabilitation needs and visual acuity.14,42 Taken together, referral practices should not be guided by measures of visual function in isolation. The importance of considering other factors, such as emotional wellbeing and comorbidities, beyond visual acuity is clear. This could be achieved by asking the appropriate questions through quality of life questionnaires,43 for which the VQL-6 may offer a practical solution.
Our study presents both strengths and limitations. A strength of this study is the evaluation of sensitivity and specificity of our questionnaire. In the absence of an established gold standard, we employed a carefully constructed interview.44 Although we found acceptable interrater reliability, a note of caution is due here as the outcome of the interview is inevitably of subjective nature and may be biased by our own personal concept of needing additional care. Moreover, as the interview is partly based on the VQL-6, the resultant upward bias may lead to an overstatement of the discriminative capacity of the VQL-6. All considered, further exploration of the diagnostic capacity of the VQL-6 is warranted. Future research could evaluate the accuracy of referrals guided by the VQL-6, by investigating the outcomes and effectiveness of the referred patients at their follow-up in integrated care. Another strength of this study is the inclusion of a variety of chronic ophthalmic diseases that may benefit from additional care, including patients with comorbidity of (chronic) ophthalmic diseases. Most studies exclude comorbidities or validate questionnaires in specific patient groups, limiting external validity. Yet, a limitation is that all patients were included from a single university medical center study. Therefore, generalizability of our results on the discriminative capacity of the VQL-6 and the suggestions for cut-off criteria to other ophthalmic patient populations in different settings is not self-evident.
The VQL-6 offers a standardized and automated method to screen for patients in need of additional care. A cutoff of ≤ 17.75 on the VL subscale showed good sensitivity and specificity, and may be an adequate criterion for use in clinical practice. It is important to recognize that a VQL-6 alert does not imply a direct referral, as patients may be familiar with low vision services, have an active medical plan, or the visual impairment is expected to improve. It is up to the ophthalmologist to consider all factors and determine – together with the patient – if a referral to additional care is necessary. We found no significant differences in interview outcomes and VQL-6 subscale scores between different patient groups, aligning with previous studies that found that ophthalmic diagnosis did not have a significant effect on quality of life outcomes or rehabilitation needs.6,13,45,46 This suggests that the VQL-6 is equally applicable across ophthalmic conditions. Administration of the VQL-6 in EMR ahead of consultations may offer an effective method and minimizes time constraints.47 As the VQL-6 is short and a simple sum score is employed, paper administration provides an alternative for those without digital access.
In conclusion, the VQL-6 appears to be a valid instrument with good discriminative capacity. For clinical practice, a cutoff score of 17.75 and lower on the VL subscale may be a good criterion to screen for patients in need of additional care. Future studies should explore the use of the VQL-6 in a clinical setting and evaluate its feasibility in reducing late referrals to low vision rehabilitation.
Conflict of interest
None.
Data availability statement
The participants of this study did not give written consent for their data to be shared publicly, so due to the sensitive nature of the research supporting data is not available.
Funding
This work was supported by Royal Dutch Visio, Huizen, and ZonMW, program Expertisefunctie Zintuiglijk Gehandicapten [grant number 6370051070]. The funding organizations had no role in the design, conduct, analysis, or publication of this research.
Meeting presentation
VISION 2022, Dublin, Ireland. NOG 2023, Groningen, Netherlands.
Acknowledgments
We thank Petra Pijnakker for her valuable contribution as a clinical expert to the reassessments of the interviews.
Footnotes
E-mail co-authors: v.l.dol{at}umcg.nl; tonroelofs{at}visio.org; a.b.m.fuermaier{at}rug.nl; annevrijling{at}visio.org; j.h.c.heutink{at}rug.nl; n.m.jansonius{at}umcg.nl
References
Subject Area
Reviews and Context
0
Comment
0
TRIP Peer Reviews
0
Community Reviews
0
Automated Services
0
Blogs/Media
Author Videos