Abstract
Introduction The Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS) is a validated self-assessment tool for health and mental health professionals who provide care for sexual and gender minority patients. This study aimed to develop and validate a Japanese version of LGBT-DOCSS (LGBT-DOCSS-JP) and examine its psychometric properties.
Methods LGBT-DOCSS was translated into Japanese and cross-culturally validated using cognitive debriefing. We then evaluated the structural validity, convergent and discriminant validity, internal consistency, and test–retest reliability of LGBT-DOCSS-JP using an online survey.
Results Data were analyzed for 381 health and mental health professionals aged 20 years or older from three suburban medical institutions. The confirmatory factor analysis indicated that the original three-factor model did not fit well with LGBT-DOCSS-JP. Exploratory factor analysis revealed four new factors: Attitudinal Awareness, Basic Knowledge, Clinical Preparedness, and Clinical Training. Convergent and discriminant validity were supported using four established scales that measured attitudes toward lesbians and gay men, genderism and transphobia, authoritarianism and conventionalism, and social desirability. The internal consistency of LGBT-DOCSS-JP was supported by the Cronbach’s alpha values for the overall scale (0.84), as well for each of its subscales (Attitudinal Awareness and Basic Knowledge both 0.87, Clinical Preparedness 0.78, and Clinical Training 0.97). The test–retest reliability for the overall LGBT-DOCSS-JP was supported by an intraclass correlation coefficient score of 0.86.
Conclusions LGBT-DOCSS-JP has the potential to serve as a valuable tool in the development and assessment of effective curricula for LGBT care education, as well as a means to promote self-reflection among trainees and professionals.
Introduction
Sexual and gender minorities including lesbian, gay, bisexual, and transgender (LGBT) people have been reported to experience health and healthcare disparities [1]. Healthy People 2020, the USA government objectives for improving health over the next decade, committed to eliminating LGBT health disparities [2]. It also emphasized the importance of enhancing efforts to improve LGBT health including providing medical students with training to increase the provision of culturally competent care [2]. Appropriate education and competency assessment are key to improving care for LGBT people. A systematic review has shown that education improves the knowledge and attitudes of medical staff and students toward LGBT patients [3].
The Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS), developed by Bidell in 2017, is an important instrument for enhancing healthcare provided to sexual and gender minorities. This scale assesses the knowledge, attitudes, and skills of healthcare and mental healthcare professionals towards caring for LGBT patients [4]. The scale was created to develop competent clinical services for LGBT people, and improve on the shortcomings of the previous scales [4]. LGBT-DOCSS has been used in recent studies to show that sufficient education and experience improves medical students’ clinical skills [5] and to develop guidelines and recommendations for the care of sexual and gender minorities [6].
In Japan, however, the provision of education on the care of sexual and gender minorities is inadequate. The 2016 revision of the model core curriculum for medical education emphasized for the first time the importance of explaining sexual orientation and gender identity as a core competency [7]. However, a recent survey showed that Japanese medical schools allocated only a limited amount of time to educate students about LGBT care [8]. There are also no guidelines on how to teach this content in medical education. One major obstacle is the lack of a suitable competence evaluation scale. To improve the capacity of medical students and staff and overcome the health disparities among sexual and gender minorities in Japan, it is essential to develop a Japanese tool for evaluating the clinical skills required to provide LGBT care.
We therefore aimed to develop a Japanese version of LGBT-DOCSS (LGBT-DOCSS-JP) and examine the scale’s reliability and validity among healthcare and mental healthcare professionals in Japan.
Methods
This study took place in two phases. The first was translation of LGBT-DOCSS into Japanese, with cross-cultural adaptation in line with guidelines from the International Society for Pharmacoeconomics and Outcomes Research Task Force for Translation (ISPOR) [9]. The second was evaluation of reliability and validity of LGBT-DOCSS-JP using an online survey. We recruited healthcare and mental healthcare professionals from Kameda General Hospital (tertiary teaching hospital), Kameda Clinic (multispecialty outpatient clinic), and Kameda Family Clinic Tateyama (family medicine clinic) as research participants. All these locations are in suburban areas within a 10 km radius of Chiba, Japan. Ethical approval was obtained from the Ethics Committees of both The Jikei University School of Medicine (approval numbers: 33-364(10988) and 34-134(11285)) and Kameda Medical Center (approval numbers: 21-012 and 22-019).
First phase: development of LGBT-DOCSS-JP
We developed LGBT-DOCSS-JP in line with ISPOR guidelines [9]. With the original author’s permission, three authors (YK, EY, YS), all of whom were native Japanese speakers and two of whom were familiar with sexual and gender minorities issues, translated the original LGBT-DOCSS into Japanese. All the authors discussed cross-cultural adaptation and made a provisional version. Next, an English-Japanese bilingual physician, who had sufficient experience in back-translation and was not familiar with LGBT-DOCSS, back-translated the provisional version into English. The original author reviewed the back-translated version and checked the discrepancies. After completing a prototype version, we recruited participants via personal connections by purposive sampling to conduct a cognitive debriefing. In recruiting participants, we considered age, sexual orientation, gender identity, occupation, and years of professional experience to ensure maximum variation. Five to eight participants were required [9]. Written informed consent was obtained from all participants. The primary investigator (YK) asked the participants to fill out the prototype version questionnaire and conducted face-to-face interviews individually using an interview guide to check interpretation, understandability, cultural adaptation, and alternative wording to make the scale better. All interviews were recorded and transcribed. Interviews were conducted between January 28th and March 9th, 2022. All the authors discussed the results, and revised the prototype version before it was reviewed again by the original author, and a final version of LGBT-DOCSS-JP developed. Content validity and cross-cultural validity were verified through these processes.
Second phase: evaluation of reliability and validity of LGBT-DOCSS-JP
Study design and data collection
We conducted an online survey among health and mental health professionals aged 20 years or older including physicians (including dentists), nurses, pharmacists, and psychologists affiliated with the study facilities. Two types of mass emails containing a link to self-administered questionnaires were sent separately to all participants. The second set were used to examine the test–retest reliability. We asked participants to complete the two questionnaires voluntarily at intervals of two to four weeks. Before participation, participants were provided with study information and required to indicate their willingness to participate by opting in. The first survey was administered on July 11th, 2022. Participants were asked to provide demographic information, and complete LGBT-DOCSS-JP and four additional scales (see “Measures” section) to examine structural validity, hypothesis testing, and internal consistency. We also included a Directed Question Scale (“Check option 6 here”), which is commonly used to detect “satisficing” or answering behaviors in which participants do not devote appropriate attentional resources to the survey [10]. The second survey was administered on July 28th, 2022, and aimed to evaluate the test–retest reliability of LGBT-DOCSS-JP. Both surveys were anonymous, but participants were asked to set a password at the first survey and enter the same password at the second survey to enable us to connect their responses. Those who responded incorrectly to the Directed Question Scale in the first survey or entered invalid passwords in the second survey were excluded from the analysis. Respondents got no reward. Reminder emails were sent twice per survey to mitigate selection bias. The questionnaires were designed using the required-response function to ensure no item response omissions except items related to participants’ sexuality such as sex assigned at birth, gender identity, and sexual orientation. The data were collected between July 11th and August 31st, 2022.
Measures
Participants’ characteristics
The demographic questionnaire asked about age groups, sex assigned at birth, gender identity, sexual orientation, and professional healthcare specialization. It also asked whether respondents were aware of any coworkers who identified as homosexual or transgender/transsexual, in both their present and previous workplaces (referred to as “homosexual coworker” or “trans coworker”), and whether they had close friends, relatives, or family members who identified as homosexual or transgender/transsexual (referred to as “homosexual friends/family” or “trans friends/family”) using a four-point Likert-type scale (yes, probably, probably not, or no) [11]. There is a lack of consensus about the appropriate method for inquiring about sexual orientation and gender identity in Japan, so we used a validated approach from a previous population-based study [12].
LGBT-DOCSS-JP
The original LGBT-DOCSS consists of 18 items across three factors: “Basic Knowledge” (four items: 1, 2, 6, and 8), “Attitudinal Awareness” (seven items: 3, 5, 7, 9, 12, 17, and 18), and “Clinical Preparedness” (seven items: 4, 10, 11, 13, 14, 15, and 16). Each item was rated on a seven-point Likert-type scale using the anchors strongly disagree to strongly agree (1 = strongly disagree, 4 = somewhat agree/disagree, 7 = strongly agree). Eight of the 18 were inverted items: 3, 4, 5, 7, 9, 12, 17, and 18. A reverse-scoring Likert-type scale was used for the inverted items (i.e., seven points were given for choice 1). The total mean score was determined by adding all test items and dividing the result by 18. The mean subscale scores were calculated by summing the scores of the corresponding questions and dividing the sum by the number of questions in each subscale. The total mean score and the mean subscale scores are therefore standardized on a scale of 1 to 7 points, with a higher score meaning greater care competency.
The Japanese 6-item revised version of the Attitudes Toward Lesbians and Gay Men Scale (ATLG-J6R)
ATLG-J6R is a six-item instrument designed to evaluate the degree of condemnation or tolerance held by heterosexual laypeople toward lesbian and gay populations [13]. ATLG-J6R uses a seven-point Likert-type scale and has two subscales: “Attitudes toward Lesbians” and “Attitudes toward Gay Men”. It has adequate reliability and validity among cisgender-heterosexual college students in Japan [13]. We used all six items of ATLG-J6R. A higher score indicates a worse attitude.
The Japanese Version of the Genderism and Transphobia Scale-Revised Short Form (J-GTS-R-SF)
The Japanese version of the Genderism and Transphobia Scale-Revised (J-GTS-R) is a 22-item instrument to measure the prejudicial attitudes of laypeople toward transgender individuals [14]. Adequate reliability and structural validity have been demonstrated among college students in Japan [14]. J-GTS-R-SF is the abbreviated version of J-GTS-R and contains 13 items. It uses a seven-point Likert-type scale [14]. Both scales have the same two factors: “Genderism and Transphobia”, and “Gender Bashing”. We used all seven items of the “Genderism and Transphobia” subscale of J-GTS-R-SF. A higher score shows a higher level of genderism and transphobia.
Japanese versions of the Right-Wing Authoritarianism scale (J-RWA)
J-RWA is a 30-item instrument with two factors (“Authoritarianism” and “Conventionalism”) and a method factor [15]. It has adequate reliability and validity among survey panel members in Japan [15]. We selected five items focusing on sexual morality, homosexuality, sexual preference, gays and lesbians, and premarital sexual intercourse, in line with a previous study [4]. J-RWA uses a nine-point Likert-type scale, ranging from −4 = very strongly disagree to 4 = very strongly agree, but we used the seven-point Likert-type scale from the LGBT-DOCSS-JP (1 = strongly disagree to 7 = strongly agree) to minimize participants’ fatigue with multiple scale changes. A higher score indicates a higher level of authoritarianism and conventionalism.
The Japanese version of Balanced Inventory of Desirable Responding (J-BIDR)
J-BIDR is a 24-item instrument that quantifies social desirability, and contains two distinct factors: “Self-Deceptive Enhancement” and “Impression Management” [16]. Its reliability and validity have been rigorously examined among a sample of college students in Japan, using a seven-point Likert-type scale [16]. The “Self-Deceptive Enhancement” factor assesses the extent to which respondents truthfully depict themselves as socially desirable, and the “Impression Management” factor evaluates the degree to which respondents misrepresent themselves to manipulate their self-presentation. Certain items within the LGBT-DOCSS-JP contain socially undesirable or negative expressions towards LGBT individuals, which may result in a bias toward positive responses. To investigate the correlation between the LGBT-DOCSS-JP and social desirability, all 24 items of the J-BIDR were included in the study. A higher score shows a greater inclination toward socially desirable responding.
Sample size
The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) guidelines suggest that studies on the development of patient-reported outcome measures should include a minimum of 100 participants [17]. The optimal subjects-to-variables ratio for exploratory factor analysis ranges from 3:1 to 20:1 [18]. LGBT-DOCSS contains 18 items, and it was therefore decided to include a minimum of 360 participants to ensure robust results.
Statistical analysis
Descriptive statistics of participants’ characteristics were generated for the study population. We also examined the psychometric properties of the questionnaires against the COSMIN guideline [17].
Structural validity
We carried out a confirmatory factor analysis with maximum likelihood estimation to assess the structural validity of the original three-factor model (Attitudinal Awareness, Basic Knowledge, and Clinical Preparedness) [4]. We used several criteria to evaluate the fit of the model, including the comparative fit index (CFI), Tucker–Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). Generally, values close to 0.95 or higher for CFI and TLI, less than 0.06 for RMSEA, and less than 0.08 for SRMR are considered to indicate a good model fit [19]. The model fit was inadequate, and we therefore carried out an exploratory factor analysis using promax rotation with maximum likelihood estimation to ascertain the structure of LGBT-DOCSS-JP. The determination of the number of factors to retain was based on Cattell’s scree test [20]. Only items with a factor loading exceeding 0.30 were incorporated.
Hypothesis testing
We compared LGBT-DOCSS-JP with the other outcome measurement instruments to evaluate convergent and discriminant validity. Convergent validity was assessed by examining Spearman’s rank correlation coefficients between the scores on LGBT-DOCSS-JP, including its overall score and subscales, and the scores on ATLG-J6R, J-GTS-R-SF, and J-RWA. Previous studies suggested that the LGBT-DOCSS-JP Attitudinal Awareness subscale would have the strongest correlation with ATLG-J6R, J-GTS-R-SF, and J-RWA. Conversely, to support discriminant validity, we hypothesized that the scores on LGBT-DOCSS-JP would demonstrate minimal correlation with J-BIDR. Certain overarching principles may be used when interpreting correlation coefficients of varying magnitude. For example, a coefficient of 0.10 is deemed small, a coefficient of 0.30 is considered medium, and a coefficient of 0.50 is deemed large [21]. Convergent validity is often considered adequate when the correlation with a measure evaluating the same construct exceeds 0.50 [22].
We also compared the scores of LGBT-DOCSS-JP among subgroups. Several studies have reported a correlation between being relatively young and affirmative attitudes toward LGBT individuals [11, 23]. We hypothesized that scores on the LGBT-DOCSS-JP scale would be higher among younger than older adults. To test this hypothesis, we used Jonckheere–Terpstra tests to compare scores among the five age groups.
Previous studies have shown that participants identifying as LGBT show significantly higher scores on the LGBT-DOCSS scale [4]. We hypothesized that scores would be higher among individuals classified as sexual and gender minorities compared to cisgender-heterosexual participants. To test this hypothesis, we used a Wilcoxon rank sum test to compare scores between cisgender-heterosexual participants and individuals classified as “sexual and gender minorities (LGBTQA)”, which includes lesbian, gay, bisexual, transgender, queer/questioning (including those who have not decided their sexualities), and asexual. Data from individuals who were missing responses to the questions on sexuality or who did not understand the question were excluded from this analysis.
Some studies of intergroup contact theory have reported that contact with members of stigmatized groups can reduce prejudice and improve attitudes towards those groups. In particular, heterosexual individuals who report personal acquaintance with gay men or lesbians show significantly more favorable attitudes toward the gay community than those who do not have this contact [24, 25]. We therefore also hypothesized that participants who know more about sexual and gender minorities would have higher scores than those who are less aware. To test these hypotheses among cisgender-heterosexual participants, we used Jonckheere–Terpstra tests to compare scores among the four levels of awareness of individuals who identify as homosexual or transgender/transsexual in current and previous workplaces and among close friends, relatives, or family members.
Internal consistency
Cronbach’s α was calculated to assess internal consistency of total and subscale scores. A score of 0.70–0.95 is considered an acceptable range [26].
Test–retest reliability
The temporal stability of the measure was evaluated by assessing its two-to-four week test–retest reliability using the intraclass correlation coefficient (ICC (2,1)). Values less than 0.5, between 0.5 and 0.75, between 0.75 and 0.9, and greater than 0.90 are indicative of poor, moderate, good, and excellent reliability [27]. Responses were determined to be from the same respondent when the passwords matched.
Responsiveness and interpretability
Responsiveness and interpretability of scores were not assessed. All statistical analyses used R, version 4.2.1 (the R Foundation). For each analysis, we used a two-tailed significance level of P < 0.05. Missing values were limited to three items: sex assigned at birth, gender identity, and sexual orientations. We used listwise deletion for comparative analysis between LGBTQA and cisgender-heterosexual participants, and among cisgender-heterosexual individuals with varying levels of awareness about sexual and gender minorities. Missing values did not affect other analyses in which sexuality was not taken into account.
Results
First phase: development of LGBT-DOCSS-JP
In the first phase, we recruited eight participants, including three physicians (a family physician, a pediatrician, and a urologist), three nurses, a pharmacist, and a psychologist.
The participants’ ages ranged from 27 to 54 years (median = 39, interquartile range (IQR) = 30.5–47.5) and years of experience ranged from 3 to 32 (median = 9.5, IQR = 5–21.5). Four participants were cisgender males, and four were cisgender females including one lesbian. No transgender individuals were included. The median interview time (IQR) was 36 (27–38) minutes. During the translation process, some expressions were adjusted to facilitate cultural adaptation. For example, some terms such as sexual orientation, gender identity, lesbian, gay, bisexual, transgender, cisgender, and the umbrella term LGBT are not widely known in Japan, and annotations were therefore added to provide clarity (S1 Appendix). Some participants mentioned that these annotations were useful.
The participants pointed out wording or expressions that required improvement, and we discussed the modifications later. For instance, most of the participants were not familiar with the concept of “institutional barriers”. We therefore adopted the expression “systematic barriers, such as rules and customs” for semantic equivalence. The expression of choices in the original questionnaire (strongly disagree–strongly agree) was considered inadequate because of the variance in natural Japanese response patterns contingent on the item type. We therefore modified the expression of choices for some items inquiring about knowledge or experience (i.e., not at all–very well or none–very much) until we had a final version of LGBT-DOCSS-JP (S1 Appendix). Content validity and cross-cultural validity were established through these processes.
Second phase: evaluation of reliability and validity of LGBT-DOCSS-JP
A total of 414 (22.3%) out of 1855 participants responded to the first questionnaire (Fig 1). However, 33 of these responses were deemed incorrect from the Directed Question Scale [10], resulting in a total of 381 (20.5%) responses available for analysis. We obtained 89 (14.2%) suitable responses from physicians/dentists, 252 (22.6%) from nurses, 34 (31.8%) from pharmacists, and six (100%) from licensed or certified clinical psychologists. In total, 151 (36.5%) of the 414 participants responded to the second questionnaire. However, 51 of these responses were excluded because of incorrect responses to the Directed Question Scale or password mismatch, giving a total of 100 responses for analysis.
Flow diagram of the study
Table 1 shows the baseline characteristics of participants. Overall, 47.5% of the sample were in their 20s, 69.6% identified as cisgender females, and 78.2% identified as heterosexual. The majority of participants (66.1%) were nurses. The mean (standard deviation (SD)) LGBT-DOCSS-JP total score was 4.16 (0.74) and the median (IQR) score was 4.11 (3.67–4.72).
Structural validity
A confirmatory factor analysis using the model described in the original paper [4] showed the following indices of model fit: CFI = 0.718, TLI = 0.673, RMSEA = 0.158 [90% confidence interval (CI) 0.151–0.166], and SRMR = 0.095. The model fit was therefore deemed insufficient, and we carried out an exploratory factor analysis. The scree test suggested a four- or five-factor model. A five-factor model was rejected because it included items with low factor loading (< 0.30). However, the four-factor model was deemed interpretable, and all factor loadings exceeded 0.30. The factors were named “Attitudinal Awareness” (items 3, 5, 7, 9, 12, 17, and 18), “Basic Knowledge” (items 1, 2, 6, and 8), “Clinical Preparedness” (items 4, 13, 14, 15, and 16), and “Clinical Training” (items 10 and 11). Table 2 shows structure coefficients. The score distributions for the overall scale and each subscale are shown in S1 Fig.
Hypothesis testing
The results of hypothesis testing for convergent and discriminant validity are shown in Table 3. They showed adequate convergent and discriminant validity. The total scores for LGBT-DOCSS-JP showed moderate correlations with ATLG-J6R, J-GTS-R-SF, and J-RWA, indicating adequate convergent validity. The Attitudinal Awareness subscale showed the strongest correlation with the J-GTS-R-SF Genderism/Transphobia subscale (r = −0.70), followed by the ATLG-J6R (r = −0.63). The weak correlation between LGBT-DOCSS-JP and J-BIDR attests to the strong discriminant validity of LGBT-DOCSS-JP.
Table 4 shows the results of the hypothesis testing. The scores for the Attitudinal Awareness subscale were significantly higher among younger age groups. However, the scores for the Clinical Preparedness subscale were higher among older age groups. The overall scores, and the scores for the Basic Knowledge and Clinical Preparedness subscales, did not show significant differences. The cisgender-heterosexual participants had significantly lower scores for the total scale and each subscale compared to LGBTQA participants. S1 Table shows that cisgender-heterosexual participants who were aware of homosexual coworkers, transgender/transsexual coworkers, and transgender/transsexual friends, family, or relatives had significantly higher overall scores and scores for some subscales. Those who were aware of sexual and gender minorities also had significantly higher scores on the Clinical Preparedness subscale in each subgroup analysis.
Internal consistency
In this group, Cronbach’s alpha of LGBT-DOCSS-JP total score was 0.84 [95% CI 0.81–0.86]. The Cronbach’s alpha for the four factors was 0.87 for both Attitudinal Awareness and Basic Knowledge, 0.78 for Clinical Preparedness, and 0.97 for Clinical Training.
Test-retest reliability
ICC(2,1) of LGBT-DOCSS-JP total score was 0.86 [95% CI 0.80–0.91]. The values for the four factors were 0.77 for Attitudinal Awareness, 0.80 for Basic Knowledge, 0.71 for Clinical Preparedness, and 0.82 for Clinical Training.
Discussion
This paper describes the process of developing LGBT-DOCSS-JP through a cross-cultural validation process following a scale translation guideline and examining its psychometric properties. We confirmed the reliability and validity of LGBT-DOCSS-JP.
In terms of structural validity, LGBT-DOCSS-JP has a distinct four-factor structure, covering Attitudinal Awareness, Basic Knowledge, Clinical Preparedness, and Clinical Training. The Attitudinal Awareness and Basic Knowledge subscales include the same items as the original LGBT-DOCSS, but we separated the original scale’s Clinical Preparedness subscale into two domains: Clinical Preparedness and Clinical Training. One of the main reasons is the difference in educational systems across nations. Education on LGBT issues is lacking within the USA, but the deficiency is even more conspicuous in Japan. A recent study reported that the proportion of medical schools that did not teach LGBT content at all during clinical training was significantly higher in Japan than in the USA and Canada (47.2% vs 33.3%) [8]. Other recent studies have found that over 90% of hospitals in Japan do not provide nursing training with LGBT content, despite the demand for this education [28, 29]. This is much worse than the 28% of baccalaureate nursing programs in the USA that did not teach LGBT sexual health at all [30]. These training issues in Japan are mainly due to the unavailability of suitable instructors and lack of school policy about LGBT education [8, 30, 31]. However, despite the lack of official practical training, many healthcare professionals are aware of the needs of LGBT patients and provide care for them regularly, increasing their clinical preparedness through trial and error. The discrepancy between the lack of official education and actual clinical practice may account for the separation of the original Clinical Preparedness factor into two factors in our study. This separation was also supported by comparing the scores with the original study [4]. The mean (SD) scores in our study for the Clinical Preparedness and Clinical Training subscales were 2.37 (1.14) and 1.75 (1.21), notably lower than the Clinical Preparedness score of 3.51 (1.45) reported in the original study [4]. The Basic Knowledge subscale mean (SD) score was also lower in our study than in the original study (3.85 (1.51) vs. 4.95 (1.51)). Providing precise knowledge and on-the-job clinical training with LGBT patients may be effective in the current Japanese context. However, the Attitudinal Awareness subscale score was as high in this study as in the original study (6.30 (0.82) vs. 6.52 (0.72)) [4]. These findings align with the social milieu of Japan, where a succession of same-sex partnership schemes have been instituted at the municipal level since 2015, and the proportion of citizens showing favorable views of LGBT individuals is increasing [11]. A higher score for Attitudinal Awareness is therefore reasonable.
We also conducted some hypothesis testing. As hypothesized, younger people scored better in the Attitudinal Awareness subscale, which is in line with previous research [4, 11]. Older participants scored higher on the Clinical Preparedness subscale, probably because of increased clinical expertise acquired over their years of professional healthcare experience. This is inconsistent with a previous study of healthcare providers in the USA, which reported no significant difference in LGBT-DOCSS scores based on age [32]. As expected, LGBTQA participants scored higher on LGBT-DOCSS-JP than cisgender-heterosexual participants. Cisgender-heterosexual individuals who were aware of homosexual coworkers at their workplace or knew there were homosexual or transgender/transsexual individuals among their friends, relatives, or family scored higher than their peers. These findings are in line with previous studies [24, 25]. However, awareness of transgender/transsexual coworkers was not significantly associated with higher scores except for the Clinical Preparedness subscale. The literature offers inconsistent conclusions about the effectiveness of contact with transgender people in reducing bias. A critical shortcoming is the limited measurement of intergroup contact with transgender people [33]. Previous studies have used a dichotomous measure of contact, disregarding the importance of both quantity and quality of contact [33]. Our study asked solely about participants’ perceptions of the presence or absence of transgender/transsexual individuals. Consequently, our measure may have failed to accurately capture the frequency and quality of intergroup contact, potentially undermining the observed effects of this contact on bias reduction. The restricted number of participants may also explain the lack of statistical power.
Our analyses showed satisfactory validity. LGBT-DOCSS-JP, especially the Attitudinal Awareness subscale, showed a robust association with established scales assessing attitudes toward lesbian and gay individuals, genderism and transphobia, as well as authoritarianism and conventionalism. It was not correlated with a social desirability scale unrelated to the construct of LGBT-DOCSS-JP. These findings show sufficient convergent and discriminant validity of LGBT-DOCSS-JP.
The results also suggest that the scale is reliable. The Cronbach’s alpha and ICC(2,1) values of LGBT-DOCSS-JP total score were sufficiently good, and almost the same as those reported in the original study [4]. These findings indicate that LGBT-DOCSS-JP shows robust internal consistency and test–retest reliability.
This study has numerous strengths. First, LGBT-DOCSS-JP is the first Japanese scale to assess health and mental health professionals’ clinical skills with LGBT patients. However, this scale was not designed to serve as a high-stakes assessment such as a pass–fail comprehensive degree examination, certification test, or licensing examination [4]. Second, the translation is based on a culturally and academically validated process. Third, we had a sufficiently large sample to evaluate psychometric properties, and the results of hypothesis testing were also conceptually acceptable. Fourth, we used an online survey to guarantee anonymity because of the sensitive nature of the topic. Some participants may not have engaged fully with the survey, but we included a question to detect and mitigate this drawback [10]. Fifth, the study demonstrated the applicability of LGBT-DOCSS in non-English-speaking and non-Western countries and cultures, addressing a limitation of the previous study [4].
However, this study also had some limitations. First, it was conducted exclusively within three suburban medical institutions, imposing constraints on its external validity. Future investigations are needed in diverse clinical settings. Second, there was some self-selection bias. Our respondents included a significantly higher percentage of sexual and gender minorities (LGBTQA) compared to a previous general population study (22.8% vs. 8.2%) [23]. Additionally, approximately four times as many individuals responded “yes” or “probably” about their awareness of sexual and gender minorities than in a previous study [11]. Those who had more awareness of sexual and gender minorities may have been more inclined to respond. Furthermore, the overall response rate was limited despite our reminder emails. It is therefore conceivable that our scale scores were overestimated when compared with the general population. Third, interpretability and responsiveness were not evaluated, and these psychometric properties should be confirmed in future studies.
LGBT-DOCSS-JP has the potential to help with the development and assessment of effective curricula for education and training in providing care for LGBT people. This is urgently needed in Japan, where effective training programs and methods have yet to be established. LGBT-DOCSS-JP may also be a valuable tool to promote self-reflection among trainees and professionals about their LGBT attitudinal awareness, basic knowledge, clinical preparedness, and clinical training. We hope this scale will primarily be used for self-exploration and competency development.
Supporting information
S1 Appendix (LGBT-DOCSS-JP). The Japanese version of the Lesbian, Gay, Bisexual, and Transgender Development of Clinical Skills Scale (LGBT-DOCSS-JP) (PDF) (Any information presented in a language other than English is not published on medRxiv. S1 Appendix is in Japanese, and thus, it is not available here. If you need this file, please contact the corresponding author.)
S1 Fig. The score distributions for the overall scale and each subscale (PDF)
S1 Table. Hypothesis testing among cisgender-heterosexual participants (PDF)
Acknowledgments
The authors are grateful to Toshiaki Kameda and Yaeko Watanabe for supporting the data collection. We also thank Daichi Hayashi for the back-translation. We would like to take this opportunity to thank Mamiko Ukai and Ryota Takahashi for their collaboration and advice as Kameda Family Clinic Tateyama primary care research team. We also thank Melissa Leffler, MBA, from Edanz (https://jp.edanz.com/ac) for editing a draft of this manuscript.
References
Subject Area
Reviews and Context
0
Comment
0
TRIP Peer Reviews
0
Community Reviews
0
Automated Services
0
Blogs/Media
Author Videos