Assessment of the Reliability, Responsiveness, and Meaningfulness of the Scale for the Assessment and Rating of Ataxia (SARA) for Lysosomal Storage Disorders

Scale for the Assessment and Rating of Ataxia (SARA)

The Scale for the Assessment and Rating of Ataxia (SARA) was initially developed to be a reliable and valid scale measuring the severity of cerebellar ataxia [1–3]. The SARA scale is composed of 8 functional domain (“item”) assessments (Gait (0-8 points), Stance (0-6 points), Sitting (0-4 points), Speech disturbance (0-6 points), Finger chase (0-4 points), Nose-finger test (0-4 points), Fast alternating hand movement (0-4 points), Heel-shin slide (0-4 points)) with total scores ranging from 0 (normal) to 40 (most severe).

The SARA underwent a rigorous validation procedure involving three large multi-center trials in spinocerebellar ataxias (SCAs) and non-spinocerebellar ataxia patients, as well as controls, which found excellent interrater reliability, test-retest reliability, and high internal consistency [2] and has undergone thorough item-response testing for multiple ataxias [4]. The SARA has also been shown to have excellent concurrent validity with other COAs, including the International Cooperative Ataxia Rating Scale (ICARS) [5], Barthel Index or with Unified Huntington’s Disease Rating Scale. Multiple studies have demonstrated that the scale reflects patient-reported symptoms and the impact of illness in cerebellar motor ataxia disorders and accurately represents patient experience [6–8]. The correlations between total SARA score and measures of daily activities and functional assessment are well-established in patients with inherited cerebellar ataxias, allowing further practical translation into the patient’s everyday life (Table 1 provides an overview of each of the 8 SARA test items and the patient-reported activities impacted to which each test item relates to) [4, 9]. Multiple clinical studies validating the psychometric properties of the SARA scale in patients with inherited cerebellar ataxias showed an individual decrease (improvement) in the total SARA of at least 1 to 1.5 points as a clinically relevant improvement, and a decrease of 1.1 points at the group level to be clinically relevant [2].

SARA for Non-Ataxia Disorders

The SARA scale was thus initially developed to measure symptoms of cerebellar ataxia in dominant spinocerebellar ataxia (SCA). Later, it was validated for use in other various types of ataxias [4, 4, 10]. More recently, the SARA has been increasingly utilized as a clinical outcome assessment for a wide range of disorders, ranging from rare entities such as lysosomal disorders to more common pediatric cancers [11–14]. The generalizability of the SARA may be related to multi-item assessments that can be categorized into 4 disease-agnostic functionally different categories:

1. Ambulation & function of lower extremities: Test items (1) Gait, (8) heel-shin slide

2. Postural balance: Test items (2) Stance, (3) Sitting

3. Speech: Test items (4) Speech disturbance

4. Function of upper extremities (Fine motor): Test items (5) Finger chase, (6) Nose-finger, (7) Fast alternating hand movements

When a patient performs voluntary movements as part of the SARA assessments, such as speaking or walking, this requires a sequence of coordinated actions (e.g., adequate motivation, attention, cognition, hearing, planning of movements, muscle power, strength, control and precision of movements) that involve many regions of the brain from the frontal cortex, somatosensory cortex, basal ganglia, cerebellum, brainstem to the corticospinal tract, and the spinal cord. In LSDs, cellular damage and cell death occur throughout the entirety of the central nervous system, manifesting as a wide range of heterogeneous neurological signs and symptoms (e.g., dysarthrophonia, ocular motor, dysmetria, ataxia, dysdiadochokinesia, dystonia, tremor, hypotonia, dyskinesias, spasticity – see Table 2), each of which could impact the ability of the patient to undertake the necessary sequence and precision of actions required to perform the SARA tasks, ultimately resulting in dysfunction in one or more of the above functional categories.

Therefore, we hypothesized that a change in the functional performance as assessed by the SARA scale could be indicative of broad alterations in many functional neurological networks, allowing its use as a measure of overall neurological disease severity in LSDs, as opposed to an isolated measure of cerebellar ataxia.

Study Objective

Given the increased use of the SARA scale as an endpoint for LSDs, we aimed to evaluate the reliability, reproducibility, and responsiveness of the scale for LSDs that feature central nervous system involvement and investigate the range of neurological signs and symptoms which could be captured and measured. The study also evaluated a minimum threshold of change which would demonstrate clinical and functional significance.

Participants

Data was analyzed from three clinical trials conducted with the agent N-acetyl-L-leucine (IB1001) for LSDs, including 2 Phase IIb, open-label, rater-blinded studies with Niemann-Pick disease type C (NPC) [“IB1001-201”, NCT03759639, n= 32 patients] and GM2 Gangliosidoses (Tay Sachs and Sandhoff diseases) [“IB1001-202”, NCT03759665, n=30 patients] and a Phase III, double-blind, placebo-controlled trial for NPC [“IB1001-301”, NCT05163288, n=60]. In the IB1001-201 and IB1001-202 studies, the SARA was a secondary endpoint; in the IB1001-301 study, the SARA was the primary endpoint.

Patients were recruited in the three clinical trials between 07-Jun-2019 and 22-Dec-2022 from 17 centers.

This study was conducted in accordance with the International Conference for Harmonisation (of Technical Requirements for Pharmaceuticals for Human Use) - Good Clinical Practice Guideline, the General Data Protection Regulator, and the Declaration of Helsinki. Approval was obtained by the applicable responsible central research ethics committees / institutional review boards for each center. Written informed consent was obtained from all study participants (or their parent/ legal representative) at enrolment. The methodology and results of each trial has been previously published [11–13, 15, 16].

Procedures

The study design/schema for the Phase IIb (IB1001-201, IB1001-202) and Phase III (IB1001-301) trials are presented in Figure 1A & B [11–13, 15, 16].

• Download figure
• Open in new tab

Fig. 1 Study Schemes from IB1001 Clinical Trials

In each study, the SARA was assessed by a qualified investigator at every study visit. The investigators underwent standardized training on the assessment and the same investigator was required to perform the SARA assessment at each visit for each patient to exclude confounding by inter-rater variability. For the IB1001-201 and IB1001-202 studies, this included two baseline visits at approximately Day 1 (Visit 1) and after 2 weeks of screening (Visit 2), two treatment visits conducted after approximately 4 weeks (Visit 3) and 6 weeks (Visit 4) of treatment with IB1001, and 2 washout visits conducted after approximately 4 weeks (Visit 5) and 6 weeks (Visit 6) of post-treatment from IB1001. In the IB1001-301 study, this included two baseline visits at approximately Day 1 (Visit 1) and after 2 weeks of screening (Visit 2), two treatment visits conducted after approximately 6 weeks (Visit 3) and 12 weeks (Visit 4) of treatment with IB1001 or Placebo, and two treatment visits conducted after approximately 6 weeks (Visit 5) and 12 weeks (Visit 6) of the opposite treatment (IB1001 or Placebo).

In addition, the Clinical Global Impression of Improvement was assessed by the Investigator, Caregiver, and Patient (if able) at the end of every treatment period, e.g. at Visit 4 (versus Visit 2) and Visit 6 (versus Visit 4) [17]. Finally, in the Phase III trial, exit interviews (in the form of semi-structured interviews) were conducted with the patient (if able) and/or caregiver (if applicable) at Visit 6 (end of the Parent Study) or the ET visit to better inform and evaluate the meaningfulness of the within-patient changes on the outcome assessments and document the relevance and meaningfulness of functional improvements in patients’ everyday lives (see the questionnaire in Supplemental Material 1) [18]. These exit interviews were conducted prior to any unblinding.

Statistical Analyses

Patients

The analyzed subset of 122 patients (mean age 27.1, range 5 – 67, years; 66 male, 56 female, 81 NPC, 30 GM2) had a mean SARA score of 15.17 (7.28) at Visit 1. As indicated by the distribution of the SARA baseline scores (SD 7.28, range 4.5 to 35), the cohort was representative of a broad range of disease severity except asymptomatic patients or the most severely impaired patients. In total, 122 Δ SARA scores from Visit 1 and Visit 2 (from the IB1001-201, IB1001-202, and IB1001-301 study) and 57 Δ SARA scores from Visit 1 and Visit 6 (from the IB1001-201 and IB1001-202 study) were evaluated for reliability and reproducibility. 58 Investigator CGI-I scores, 50 Caregiver CGI-I scores, and 49 Patient CGI-I scores comparing Visit 4 to Visit 6 (from the IB1001-301 study) and were evaluated against the corresponding 58 Δ SARA scores for responsiveness. Across the three trials, there were 15 patients aged <10 years (range 5 to 9 years); in the IB1001-201 and IB1001-202 trials there were 7 patients aged <10 years (range 6 to 9 years).

Reliability & Reproducibility

Test re-test data for all three trials for the baseline period (Visit 1 and Visit 2) are shown in Table 3. The mean (SD) change between baseline Visit 1 and Visit 2 for the IB1001-201, IB1001-202 and IB1001-301 trials were -0.30 (1.75), +0.02 (1.23), and +0.03 (1.96) respectively (Table 2). There was no statistically significant change in the mean value from Visit 1 to Visit 2 for any study or statistically meaningful difference found between the re-test IB1001-301 score and the IB1001-201 or IB1001-202 scores, reflecting the SARA was highly reliable/reproducible. For patients <10 years, the mean (SD) change from baseline Visit 1 and Visit 2 was 0.27 (2.05) which was not statistically significantly different from the cohort of patients aged 10 years and older (p=0.50), reflecting the SARA was also reliable/reproducible in this population.

The ICCs for the SARA scale and each of its 8 items are given in Table 4. The SARA scale correlations were 0.971, 0.986 and 0.966 for the three trials. These item-level correlations all exceeded the 0.70 threshold and showed strong agreement, demonstrating a high degree of internal consistency. The Sitting test ICC was 1.0 for the IB1001-202 which was an indication of the flooring effect for that item (20 of 29 scores were 0).

Test re-test data for the IB1001-201 and IB1001-202 trials for the baseline versus post-treatment washout period data (Visit 1 and Visit 6) are shown in Table 5. There were no differences observed between the baseline visit and the post-washout visit (Visit 1 to Visit 6); the mean change (SD) in SARA score was -0.03 (2.61) in the IB1001-201 study and -0.04 (2.07) in the IB1001-202 study. This further reinforced the reliability of the administration of the SARA scale, and also demonstrated the absence of a learning effect on the 8 SARA items. For patients <10 years, the mean (SD) change from baseline Visit 1 and post-treatment washout Visit 6 was 0.00 (1.61) which was not statistically significantly different from the cohort of patients aged 10 years and older (p=0.95), reflecting the SARA was also reliable/reproducible in this population.

The ICCs for this comparison are shown in Table 6. The total SARA scale ICCs were high and above the 0.80 threshold. These item-level correlations all exceeded the 0.70 thresholds and showed strong agreement. There was also further evidence of the flooring effect in the Sitting test where the IB1001-202 trial ICC was 1.0.

Responsiveness

The CGI-I scores categorized as “unchanged”, “improved” and “worsened” and corresponding Δ SARA values for the IB1001-301 trial (Visit 4 to Visit 6) are summarized in Table 7. Δ SARA values ranged between -5.5 and +6.5 (expected variance given approximately 50% of patients in this treatment period were commencing IB1001, and 50% were stopping IB1001 treatment). The Mean (SD) was -0.06 (2.72).

The CGI-I values categorized as “improved” or “not improved” used for the AUC calculations are summarized in Table 8. As described above, patients randomized in the sequence Placebo-IB1001 were expected to show improvement during this period those randomized in the sequence IB1001-Placebo were expected to show worsening during the second period if the patient was a responder to the study drug. The AUC for Investigator CGI-I was 0.82, for Caregiver CGI-I it was 0.71, and for Patient CGI-I AUC was 0.77. All CGI-Is were above the threshold for discriminative ability, supporting changes in SARA aligned with CGI-I assessments of changes in patients’ overall function and well-being.

Correlation Between Total Score and Activities of Daily Life

42 exit interviews were qualitatively assessed for patients who experienced a 1-point or greater improvement on the SARA scale after treatment with IB1001 or, for those randomized to receive IB1001 followed by Placebo, a 1-point worsening or greater on the SARA. 70% of patients were identified to be responders to the study drug, meaning that the patient/caregiver described clinically meaningful, relevant changes in exit interviews, reinforcing previous findings that there is a close correlation between total SARA score and measures of daily activities and functional assessment and that a minimum 1-point change is clinically meaningful [2]. The exit interviews further elucidated that clinically meaningful changes included: increase in strength and energy; improved cataplexy, dysphagia, ataxia, dystonia; reduced (less) pain in muscles/general; improved speech, more easily understood/fluent speech, easier to integrate into a conversation, easier to communicate with; improved ambulation, mobility, balance, coordination, and autonomous gait; reduced falls; improved fine motor skills/general motor skills, less tremor; improved cognition, concentration, brain fog, focus, memory, cooperation, behavior, mood; reduced anxiety; less swallowing problems, less coughing while swallowing; improved incontinence (urine and anal); reduced seizures; improved sleep; improved ability to perform everyday tasks (feeding, dressing, playing, work, following orders, participating in leisure activities), and were not limited to the isolated measure of cerebellar ataxia. Examples from the exit interviews are provided in Table 9.