ABSTRACT
Objectives To evaluate the inter-rater agreement (IRA) among members of the Patient’s Health Record Review Board (PHRRB), in routine auditing of medical records, and the impact of periodic discussions of results with raters.
Design Prospective longitudinal study conducted between July of 2015 and April of 2016.
Setting Hospital Municipal Dr. Moysés Deutsch, a large public hospital in São Paulo.
Participants The PHRRB was composed of 12 physicians, 9 nurses and 3 physiotherapists, who audited medical records, monthly, with the number of raters changing throughout the study.
Interventions It was carried out PHRRB meetings in order to reach a consensus on criteria that the members have to rate in the auditing process. It was created a review chart that raters should verify the registry of patient’s secondary diagnosis, chief complaint, history of presenting complaint, past medical history, medication history, physical exam and diagnostic testing. It was obtained the IRA every three months.
Measures The Gwet’s AC1 coefficient and Proportion of Agreement (PA) were calculated to evaluate the IRA for each item over time.
Results The study included 1884 items from 239 records with an overall full agreement among raters of 71.2%. A significant IRA increase by 16.5% (OR=1.17; 95% CI=1.03—1.32; p=0.014) was found in the routine PHRRB auditing, with no significant differences between the PA and the Gwet’s AC1, that showed a similar evolution over time. The PA decreased by 27.1% when at least one of the raters was absent from the review meeting (OR=0.73; 95% CI=0.53—1.00; p=0.048).
Conclusions Medical record quality has been associated with the quality of care and could be optimized and improved by targeted interventions. The PA and the Gwet’s AC1 are suitable agreement coefficients that are feasible to be incorporated in the routine of PHRRB evaluation process.
STRENGTHS AND LIMITATIONS OF THIS STUDY
Implementation of a scientific method in a routine task of data generation of a PHRRB.
Prospective longitudinal design evaluating inter-rater agreement over time and associated factors.
Agreement comparisons among more than two simultaneous raters.
Relatively short follow-up.
Results are not generalizable to other health facilities.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was funded by the Brazilian Ministry of Health and Sao Paulo State Research Foundation (FAPESP) through Research Program for the Unified Health System-PPSUS grant 2012/51228-9.
Author Declarations
All relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained and details of the IRB/oversight body are included in the manuscript.
Yes
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
The dataset is available to researchers who want to explore the data. To request, please send an email to ana.mafra{at}einstein.br.
