An initial validation study of DigiBel, a web-application enabling self-assessment of air and bone-conduction audiometry in the community

Introduction

Over 5% of the world’s population, approximately 430 million people worldwide, have disabling hearing loss; 34 million of these are children. (1, 2) Children in low and middle income countries, families experiencing socio-economic deprivation and disadvantaged populations are disproportionally affected by conductive hearing loss caused by glue ear (otitis media with effusion) and its complications, or from damage to the ear drum as seen in chronic tympanic perforations or chronic serous otitis media. (3, 4) Since delayed recognition and management of childhood hearing impairment has long-term consequences for socialisation and educational attainment, screening audiometry is recommended in primary school. (5) However, population screening programmes are hindered by cost, standardisation, requirement for staff-training, false-positive referrals, and poor data capture. (6, 7) Even where available, school screening may miss children with fluctuating hearing loss due to glue ear. Additionally, several year groups have missed screening during the Covid pandemic. These children may face months of impaired hearing before diagnosis and management due to backlogs in audiometry and specialist services.

Hearing thresholds are assessed using pure-tone audiometry (PTA) with air-conduction (AC) and bone-conduction (BC) headphones, but this usually requires specialist equipment and trained clinicians. Automated audiometry and, more recently, validated self-testing hearing software applications (apps), may improve the accessibility of screening and threshold audiometry testing, particularly in rural areas. DigiBel is a recently developed CE marked online app which enables self-testing of AC and BC hearing levels. Like some other audiometry apps, it is suitable for community use in adults and children without clinical support. Its potential, more unique, function is to identify children with hearing impairment who may derive functional benefit from using a BC headphone / microphone kit (Raspberry Pi, Cambridge, UK) while waiting for diagnosis, spontaneous resolution or definitive management of their glue ear (8).

The purpose of this preliminary study of DigiBel audiometry, undertaken in healthy adults situated in quiet community settings, is to determine the app’s sensitivity and specificity for detecting simulated conductive hearing impairment of more than 20 decibels (dB). The secondary aim is to assess the web-application’s usability to enable improvement of the app design.

Materials and methods

In this comparative study, healthy adult volunteers from the community without previous history of hearing impairment were invited to participate. After receiving an explanation of the study, participants provided verbal consent to proceed to audiometry testing. Each participant was assigned a unique study identification number; no personal identifiable information was recorded. The study adhered to the tenets of the Declaration of Helsinki and the protocol was approved by the Quality and Safety Committee of Cambridge University Hospitals NHS Foundation Trust as part of a service improvement project.

Testing was undertaken in community settings such as participants’ homes and classrooms. Prior to testing, each volunteer placed a foam earplug into their left ear to simulate a conductive hearing loss, making each ear an independent entity for the purpose of statistical analysis and providing a range of hearing levels.

The reference automated AC PTA and the index DigiBel audiometry test were undertaken one after the other in random sequence. DigiBel audiometry for four frequencies was repeated immediately after the initial test, to assess test-retest (TRT) variability. After completing both audiometry tests, each volunteer was asked to complete a feedback questionnaire covering test preference and usability.

The DigiBel index test

DigiBel has been laboratory and biologically calibrated (Institute of Sound and Vibration Research, University of Southampton, UK and Chears-audiology, Royston, UK) to run specifically on any model iPad tablet (Apple, Cupertino, California, USA) with Seinnheiser HD 400S AC headphones (Wedemark, Germany). BC audiometry (not tested in this study) uses Bluetooth-connected Raspberry Pi children’s BC headphones.

A pre-test embedded video provides instructions for use and a checklist ensures that the iPad volume is on maximum and the headphones are fitted correctly. Noise sampling, using the inbuilt Sennheiser headphone microphone, ensures that the ambient noise level is less than 40 decibels (dB) prior to testing. Configurable settings include a choice of warble / pure tone, test frequencies, ambient noise setting and child or adult version of the test.

The user taps a central animated button on the iPad display when they hear the tone (Fig 1A). A conditioning step requires the user to accurately tap the button on hearing a random onset supra-threshold tone at 1000Hz before testing can begin. During testing, the onset of the tone is randomised from 0-3 seconds after appearance of the response button to avoid a predictable response pattern. The tone stops in response to the tap and a psychophysical staircase algorithm starting from 60dB (10dB down, 5dB up) is followed requiring three reversals dependent on the user’s input. The final threshold is calculated by the mean of the three reversal thresholds. Once completed, a standard audiometry graph and the number of false positive responses is displayed (Fig 1B). The user can choose to repeat testing or undertake BC testing to determine the functional effect on hearing levels.

• Download figure
• Open in new tab

Fig 1.

(A) DigiBel test interface (B) DigiBel audiometry graph

For this study, DigiBel AC PTA was undertaken in the sequence 2000, 4000, 8000, 1000, 500 and 250 Hz and retested in the sequence 2000, 4000, 8000, 500 Hz using the adult test version.

Statistics

Data was analysed in R (version 4.1.2; R Foundation for Statistical Computing, Vienna, Austria). (11, 12) Accuracy and TRT reliability were assessed through Bland-Altman analysis and calculation of intraclass correlation coefficients (ICCs). Qualitative appraisal of ICC was based on conventional standards with moderate, good and excellent agreement indicated by ICC ≥0.50, ≥0.75 and ≥0.90 respectively. (13) The percentages of DigiBel threshold measurements lying within 10 dB of both the reference test and the repeated DigiBel test were calculated. (14) Statistical significance was calculated for the bias (mean difference between measurements) where confidence intervals did not cross zero, and for ICCs where p < 0.05. To assess diagnostic efficacy, sensitivity, and specificity for detection of hearing thresholds above 20Hz were calculated, using automated PTA as a reference. Student’s t-test was used to compare the number of false positive responses for each test. Throughout, magnitudes were reported as the mean (± standard deviation) unless otherwise stated.

Results

Participant characteristics

32 healthy subjects agreed to take part, but two were excluded due to malfunction of the reference automated PTA test. The 30 subjects who completed both the reference and index tests were aged 21-66 years of age (mean 27.9 ±10.3 years). TRT data was collected from 29 subjects (one volunteer left early due to time constraints). 24 subjects completed feedback forms (Fig 2).

• Download figure
• Open in new tab

Fig 2.

Participant flow diagram

Accuracy and reliability of DigiBel

Across the six tested frequencies, threshold hearing levels of DigiBel compared to standard PTA gave ICC > 0.75 (good or excellent agreement) and p < 0.001 in every trial (Table 1).

View this table:

• View inline
• View popup
• Download powerpoint

Table 1.

Comparison of threshold values in decibels (dB) using Bland Altman statistics and intra-class correlation coefficients (ICCs) between a) DigiBel and standard automated audiometry and b) DigiBel test and retest, where LLoA is the lower limit of agreement and ULoA is the upper limit of agreement.

Mean lower limit of agreement (LOA) was -17.04 dB (±2.12 dB); mean upper LOA was 20.39 dB (±3.41 dB). No significant bias was apparent except at 4000 and 8000Hz where a small but statistically significant bias was apparent (2.62 and 4.60 dB, respectively), with DigiBel providing systematically higher threshold results at those frequencies (Fig 3). An agreeable level of threshold difference in audiometry assessments has previously been defined as 10dB; 73.1% of DigiBel threshold measurements were within this standard. (15)

• Download figure
• Open in new tab

Fig 3.

Bland-Altman plots comparing mean and difference in threshold measurements in decibels (dB) for DigiBel and standard automated pure tone audiometry at six frequencies (Hertz). 95% confidence intervals are shaded for the bias (red) and 95% limits of agreement (blue).

In TRT comparisons, ICC > 0.90 (excellent) at all tested frequencies: 500, 2000, 4000 and 8000Hz (Table 1). No statistically significant mean bias was exhibited at any frequency. Mean lower LOA was -13.99 dB (±2.34 dB); mean upper LOA was 15.76 dB (±2.20 dB). Overall, 84.6% of TRT thresholds were within 10dB of each other.

A mean 3.57±4.68 false positive responses were recorded during reference testing and 11.43±7.12 during the first DigiBel test (p<0.01).

Discussion

At least five validated downloadable apps enable automated AC audiometry, several of which support self-testing without clinician involvement (16). Two, uHear and SHOEBOX for iOS, include a BC audiometry facility (17, 18). uHear has been calibrated for use with commercial in-ear Apple headphones and SHOEBOX uses purpose-built audiometry headphones. A potential advantage of DigiBel is that the system is calibrated to use affordable (retail price $41, €39), lightweight and wipe clean Sennheiser HD 400S AC headphones and Raspberry Pi BC headphones (retail price $28, €26).

DigiBel’s sensitivity and specificity for hearing impairment (>20dB threshold depression) screening was 100% (95%CI 87.2-100) and 72.7% (95%CI 54.45-86.7) respectively. This is comparable to previous studies of both uHear (98.2-100% sensitivity and 60.0-82.1 specificity) and SHOEBOX (91.2-93.3% and specificity of 57.8-94.5%). (18, 19, 17, 20, 21)

The hearing threshold measurement bias with DigiBel was insignificant compared to reference except at 4000 and 8000Hz, where the bias (2.62 and 4.60 dB higher than reference, respectively) reached statistical significance. Overall, 73% of threshold measurements with DigiBel were within 10dB of standard PTA; a previous study of SHOEBOX has found that 92.9% measurements were within this range. (14) The comparatively poor performance of DigiBel for this metric may be due to environmental noise leakage through the Sennheiser headphones, a disadvantage of their comfort.

Staircase methods of psychophysical threshold assessment are generally reported to be more reliable with less variability than ascending methods. (22) To our knowledge, DigiBel is the only application to apply this technique to audiometry, although it is commonly used for visual threshold testing. (13, 23) In this study, the number of false positive subject responses was significantly higher with DigiBel than standard testing in this study and may reflect its higher number of stimulus presentations compared to the ascending method used in standard automated PTA. The sensitivity of the iPad screen to a tap compared to the standard audiometer’s hand-held responder may an additional factor.

All subjects rated DigiBel good or excellent, but 41.7% complained that the test took too long or was boring. Test duration was not measured during this study, but threshold testing for six frequencies is expected to take approximately 13 minutes with DigiBel, several minutes longer than standard automated PTA, primarily due to its staircasing algorithm.

Study participants had also performed retests which may have contributed to perceived length of testing. Although the child-friendly version of the test is more entertaining, the test duration of threshold audiometry may limit its usability in young children. The DigiBel screening test of four frequencies takes approximately three minutes and may prove more feasible in this, its target population.

To simulate a range of hearing thresholds in the subject cohort, an earplug was used. This is a major limitation of the study because the effectiveness of the earplug may have altered during testing and earplugs may not accurately mimic genuine hearing impairment.

Although this study indicates that DigiBel is an acceptable and accurate method for detecting simulated hearing impairment in healthy adult volunteers, this is not generalisable to clinical settings.

This preliminary study confirms that the DigiBel app is an acceptable and easy-to-use self-testing online tool which accurately detects more than 20dB of simulated hearing impairment in adults. Future studies are planned to confirm DigiBel’s accuracy in clinical populations, particularly young children, and to determine if the app can identify those who may benefit from temporary use of a BC headphone / microphone kit at school, work and home while waiting for specialist care.

Conflicts of interest

Louise Allen is the inventor of the DigiBel app and founding director of Cambridge Medical Innovation Ltd.

Funding

This study received no funding.