Protocol for Development of a Reporting Guideline for Causal and Counterfactual Prediction Models

Introduction While there are protocols for reporting on observational studies (e.g., STROBE, RECORD), estimation of causal effects from both observational data and randomized experiments (e.g., AGREMA, CONSORT), and on prediction modelling (e.g., TRIPOD), none is purposely made for assessing the ability and reliability of models to predict counterfactuals for individuals upon one or more possible interventions, on the basis of given (or inferred) causal structures. This paper describes methods and processes that will be used to develop a reporting guideline for causal and counterfactual prediction models (tentative acronym: PRECOG).

Materials and Methods PRECOG will be developed following published guidance from the EQUATOR network, and will comprise five stages. Stage 1 will be bi-weekly meetings of a working group with external advisors (active until stage 5). Stage 2 will comprise a scoping/systematic review of literature on counterfactual prediction modelling for biomedical sciences (registered in PROSPERO). In stage 3, we will perform a computer-based, real-time Delphi survey to consolidate the PRECOG checklist, involving experts in causal inference, statistics, machine learning, prediction modelling and protocols/standards. Stage 4 will involve the write-up of the PRECOG guideline (including its checklist) based on the results from the prior stages. In stage 5, we will work on the publication of the guideline and of the scoping/systematic review as peer-reviewed, open-access papers, and on their dissemination through conferences, websites, and social media.

Conclusions PRECOG can help researchers and policymakers to carry out and critically appraise causal and counterfactual prediction model studies. PRECOG will also be useful for designing interventions, and we anticipate further expansion of the guideline for specific areas, e.g., pharmaceutical interventions.