Protocol: Trust Your Gut: An Analysis of Dermatologic Diagnostic Accuracy
=========================================================================

* Dana Jolley
* Abraham M. Korman

## Abstract

**Background** The current clinical misdiagnosis rate among all medical specialties is approximately 10-15%, but diagnostic error within the field of dermatology has not been studied thoroughly1,2. As a field that relies heavily on visual perception, many physicians consider clinical intuition to be advantageous in diagnosing skin diseases and consider it to be a rapid and unconscious phenomenon7. Therefore, too much contemplation may lead to more incorrect diagnoses4. However, while clinical intuition is a valuable clinical tool, it is widely considered to be developed throughout medical training and only successfully employed by experienced attending physicians, perhaps due to experiential knowledge and associated confidence1,2,5. One may expect that self-reported confidence in diagnosis would correlate with diagnostic accuracy, but this is not supported in the literature9. The focus of our study is to examine the development and reliability of clinical intuition as well as associated self-reported confidence levels in diagnoses at different levels of medical training among dermatologists.

**Methods** Approximately 20 dermatologists who are PGY-2 or higher will be recruited for study participation via email. Participants will be sent a Qualtrics survey at two separate time points with a month waiting period in between. The survey will contain demographics questions, photos of 10 different dermatologic conditions for dermatologists to diagnose, and a self-reported confidence level for each diagnosis. The first survey will allow 5 seconds to evaluate a clinical photo prior to diagnosis, and this timeframe will be extended to 15 seconds in the second survey. The second survey will contain the same diagnoses, but with different pictures to avoid recall of specific photos. Following completion of all surveys, descriptive statistics will be completed with goal of publication.

**Discussion** This study has the potential to provide invaluable information regarding the development of clinical intuition among dermatologic physicians while also examining their confidence levels and likelihood of changing correct diagnoses when given more time to ruminate. It is possible that physicians are more likely to second guess original diagnoses based off of certain demographic factors, as one systematic review found that women in medicine perceive their clinical performance as deficient more often than men10. Therefore, this study may give insight to the ways that complicated societal factors contribute to clinical decision making. Data from this study may be used to aid dermatologists in understanding their thought processes when diagnosing patients, and may be useful in developing education curriculum. The protocol will hopefully serve as a blueprint for creation of studies in a multitude of fields, ultimately leading to better understanding of clinical decision making and, thus, improved patient care.

## Introduction and Rationale

The current clinical misdiagnosis rate among all medical specialties is approximately 10-15%1,2. Diagnostic error leads to a delay in treatment for patients, unnecessary medical care, and increased health care spending. One article published on cellulitis even suggested that misdiagnosis led to approximately 50,000-130,000 unnecessary hospitalizations and 195-515 million dollars in health care spending that could have been avoided3. Within the field of dermatology, diagnostic error is a serious issue that has not been studied thoroughly1,2. In order to determine the root cause of dermatologic misdiagnosis, it is prudent to examine the process of diagnosing skin disease at all levels of training and to consider social factors that could potentially lead to medical error.

Previous studies have shown that deliberate reflection on initial diagnosis increased diagnostic accuracy within the fields of internal medicine and general practice, but fields like dermatology that rely heavily on visual perception, mental schemas, and pattern recognition for diagnosis, too much contemplation may lead to more incorrect diagnoses4. However, some studies assert that dermatologic diagnoses do require a great deal of deliberation and discourage diagnostic shortcuts in order to avoid diagnostic errors1,2,5. Others describe the visual recognition involved in dermatology as an instantaneous process6. In fact, many physicians themselves consider clinical intuition to be advantageous in diagnosing skin diseases and consider it to be a rapid and unconscious phenomenon7. Given these two differing processes for determining a dermatologic diagnosis, it is important to explore the development of clinical intuition.

While clinical intuition is a valuable clinical tool, it is widely considered to be developed throughout medical training and only successfully employed by experienced attending physicians1,2,5. Most medical students and residents cannot rely on intuition or gut feelings due to a lack of experience and retrospective analysis of cases8. This assertion is supported by cognitive science research into the diagnostic process which emphasizes that experiential knowledge is committed to long term memory and then accessed for classifying future diagnoses5.

Progression through medical training and development of additional diagnostic skills can lead to an expectation of increased confidence in one’s medical capabilities. This raises the additional question of whether self-reported confidence in diagnosis has any correlation with decreased medical error. Surprisingly, a previous study found that a physician’s level of confidence actually had little association with diagnostic accuracy and showed that although there was wide variation in clinical diagnosis, there was very little difference in self-reported confidence9. Although confidence does not seem to give way to more accurate diagnoses, it could potentially interfere with trusting one’s own intuition. This may cause physicians from underrepresented groups to second guess their diagnoses more frequently. One systematic review found that women in medicine perceive their clinical performance as deficient more often than men, including rating themselves lower in areas of knowledge, experience, and confidence even though evidence suggests no difference in actual performance between men and women at any level of training10. This variation in self-reported perception of skills may lead to a higher tendency among females to change diagnoses when given more time to evaluate the medical problem at hand.

The focus of our study is to examine the development and reliability of clinical intuition at different levels of medical training by surveying resident and attending dermatologists at two separate time points to calculate accuracy of diagnoses and how frequently diagnoses are changed with increasing amounts of time. We will also collect self-rated confidence levels in diagnoses and demographic factors. We hypothesize that resident dermatologists who are early in their training will have more accurate diagnoses when given increased time to evaluate a presentation, indicating poor clinical intuition. We also suspect that more seasoned residents and attending dermatologists will have better clinical intuition as shown by more correct diagnoses at a shorter time period of evaluation. However, we expect some of these more experienced physicians to have a high likelihood to change their initial diagnosis to an incorrect one with more time to evaluate a presentation. The likelihood of second-guessing an initial diagnosis will likely be dependent on confidence or certain social factors, such as race or gender.

## Materials and Methods

### Aims

#### Primary aim

1.  Determine the accuracy of resident and attending dermatologists’ clinical diagnoses versus examination time variations

2.  Assess the development of clinical intuition among resident and attending dermatologists

#### Secondary aims

1.  Describe the likelihood of incorrectly changing diagnoses with greater amounts of time to second guess

2.  Determine the association between confidence in diagnoses and accuracy of diagnoses

### IRB Approval

This study was determined exempt from IRB review on 11/07/2023 by Ohio State University’s Office of Responsible Research Practices in qualifying exempt category 2b. Study number 2023E0952.

### Procedures

#### A. Targeted Audience

Our research is targeted to dermatologists.

#### B. Data Distribution

The data collected from this study will be used to evaluate the development of clinical intuition and examine social factors that may play into trusting that intuition. The data will be published in a dermatology journal to distribute information to practicing dermatologists.

#### C. Research Design

1.  *Sites*
    
    1.  The Ohio State University

2.  *Inclusion criteria*
    
    1.  Dermatologists who are PGY-2 or higher in level of medical training.

3.  *Methods*
    
    1.  We are designing a survey-based, crossover study in which we query approximately 20 practicing dermatologists at two separate time points.
    
    2.  We will send a recruitment email to the dermatologists to gauge interest in participating in the study. We will include approximately 20 dermatologists who agree to participate. Recruitment will begin on 2/11/24 and end on 3/20/24.
    
    3.  The participating dermatologists will be sent a Qualtrics survey at two separate time points. They will be sent with a month waiting period in between.
    
    4.  The email containing the survey link will also include a copy of IRB-approved informed consent document, containing verbiage that clicking the survey link serves as consent to participate in the study.
    
    5.  The survey will be built as follows:
        
        1.  We will provide photos of 10 different dermatologic conditions for dermatologists to diagnose.
        
        2.  For the first survey, dermatologists will be given 5 seconds to evaluate the photo prior to making a diagnosis. The time will be extended to 15 seconds for the second survey.
        
        3.  Each survey will also gather the physician’s confidence level in each diagnosis on a scale of 1-5, with 5 being highest level of confidence.
        
        4.  Each survey will collect demographic information such as level of medical training, race, age, and sex.
        
        5.  Each survey will be scored as correct or incorrect for accuracy of diagnoses using a master key.
        
        6.  The survey at the second time point will include the same diagnoses, but with different pictures to avoid participants recalling specific photos.
        
        7.  The second survey will be administered 1 month following the administration of the first survey.
        
        8.  Each survey will take approximately 5-10 minutes to complete.
    
    6.  Only participants who complete the first survey will be sent the second survey. The Qualtrics survey will include their name and email, which is how we will know where to send follow up surveys.
    
    7.  Once the surveys are completed by all dermatologists, the data will be analyzed and interpreted with goal of publication.

4.  *Variables*
    
    1.  Independent: survey questions along with a mixture of inpatient and outpatient dermatologic diagnoses
    
    2.  Dependent: survey answers

#### D. Sample

We plan to query approximately 20 dermatologists from The Ohio State Department of Dermatology.

#### E. Measurement/Instrumentation

Descriptive statistics will be performed on results of survey questions and diagnoses.

#### F. Detailed Study Procedures

After IRB exemption, we will begin contacting dermatologists to participate in our study. All data will be viewed at the OSU facilities under a locked door and saved within the secured access dermatology research drives. Recruitment will begin on 2/5/24 and is expected to continue for approximately two months. Survey data will be analyzed once all participants complete the second survey and results will be reported via academic journal in order to disseminate information to practicing dermatologists.

#### G. Data Analysis

Primary Outcome Measures: Quantitative assessment of correct diagnoses by dermatologists at different time points.

Secondary Outcome Measures: Quantitative assessment of factors such as demographics and level of medical training which may affect one’s trust in a clinical diagnosis.

## Discussion

### Limitations

#### A. Internal Validity

Dermatologists may possess varying levels of experience with treating certain clinical diagnoses (e.g. inpatient vs outpatient). To address this internal issue, we will ensure equal numbers of diagnostic categories and recruit dermatologists from all backgrounds.

#### B. Loss to Follow-Up

Completing surveys at two separate time points may be a difficult task for many busy practicing physicians. To address this issue, we have designed the survey to be as short and user-friendly as possible. However, we still expect to lose some participants to follow-up and will try to recruit more participants than our sample size to offset this issue.

## Data Availability

No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.

## Authors’ Contributions

Dana Jolley, BS; Conceptualization, data curation, formal analysis, methodology, project administration, validation, manuscript preparation

Dr. Abraham Korman, MD; Conceptualization, methodology, writing (review & editing)

## Footnotes

*   **Metadata**
    
    The authors received no specific funding for this work.
    
    The authors have declared that no competing interests exist.
    
    No datasets were generated or analyzed during the current study. All relevant data from this study will be made available upon study completion.

*   Received March 27, 2024.
*   Revision received March 27, 2024.
*   Accepted March 28, 2024.


*   © 2024, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)

## References

1.  1.Lowenstein, E. J., Sidlow, R., & Ko, C. J. (2019). Visual perception, cognition, and error in dermatologic diagnosis: Diagnosis and error. Journal of the American Academy of Dermatology, 81(6), 1237–1245. doi:10.1016/j.jaad.2018.12.072
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jaad.2018.12.072&link_type=DOI) 

2.  2.Lowenstein, E. J. (2018). Dermatology and its unique diagnostic heuristics. Journal of the American Academy of Dermatology, 78(6), 1239–1240. doi:10.1016/j.jaad.2017.11.018
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jaad.2017.11.018&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F03%2F28%2F2024.03.27.24304982.atom) 

3.  3.Weng, Q. Y., Raff, A. B., Cohen, J. M., Gunasekera, N., Okhovat, J.-P., Vedak, P., Joyce, C., Kroshinsky, D., & Mostaghimi, A. (2017). Costs and Consequences Associated With Misdiagnosed Lower Extremity Cellulitis. JAMA Dermatology, 153(2), 141–146. doi:10.1001/jamadermatol.2016.3816
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamadermatol.2016.3816&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27806170&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F03%2F28%2F2024.03.27.24304982.atom) 

4.  4.Costa Filho, G. B., Moura, A. S., Brandão, P. R., Schmidt, H. G., & Mamede, S. (2019). Effects of deliberate reflection on diagnostic accuracy, confidence and diagnostic calibration in dermatology. Perspectives on Medical Education, 8(4), 230–236. doi:10.1007/s40037-019-0522-5
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s40037-019-0522-5&link_type=DOI) 

5.  5.Brush, J. E., Sherbino, J., & Norman, G. R. (2017). How Expert Clinicians Intuitively Recognize a Medical Diagnosis. The American Journal of Medicine, 130(6), 629–634. doi:10.1016/j.amjmed.2017.01.045
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.amjmed.2017.01.045&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F03%2F28%2F2024.03.27.24304982.atom) 

6.  6.Ko, C. J., Braverman, I., Sidlow, R., & Lowenstein, E. J. (2019). Visual perception, cognition, and error in dermatologic diagnosis: Key cognitive principles. Journal of the American Academy of Dermatology, 81(6), 1227–1234. doi:10.1016/j.jaad.2018.10.082
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jaad.2018.10.082&link_type=DOI) 

7.  7.Vanstone, M., Monteiro, S., Colvin, E., Norman, G., Sherbino, J., Sibbald, M., Dore, K., & Peters, A. (2019). Experienced physician descriptions of intuition in clinical reasoning: A typology. Diagnosis, 6(3), 259–268. doi:10.1515/dx-2018-0069
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1515/dx-2018-0069&link_type=DOI) 

8.  8.Biswas, A. (2015). Gut feeling: Does it have a place in the modern physician’s toolkit? Medical Teacher, 37(4), 309–311. doi:10.3109/0142159X.2014.960378
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3109/0142159X.2014.960378&link_type=DOI) 

9.  9.Meyer, A. N. D., Payne, V. L., Meeks, D. W., Rao, R., & Singh, H. (2013). Physicians’ Diagnostic Accuracy, Confidence, and Resource Requests: A Vignette Study. JAMA Internal Medicine, 173(21), 1952–1958. doi:10.1001/jamainternmed.2013.10081
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamainternmed.2013.10081&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23979070&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F03%2F28%2F2024.03.27.24304982.atom) 

10. 10.Vajapey, S. P., Weber, K. L., & Samora, J. B. (2020). Confidence gap between men and women in medicine: A systematic review. Current Orthopaedic Practice, 31(5), 494. doi:10.1097/BCO.0000000000000906
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/BCO.0000000000000906&link_type=DOI)