Group work - interpretation & responsiveness

1. Interpretation - MIC-predictive

You have collected data from a clinical trial evaluating the effect of lumbar spinal surgery compared to exercise therapy. As a secondary aim you have decided to look at how to interpret the Oswestry Disability Index (ODI) and want to calculate the MIC-predictive.

Recall from the introductory course that the ODI has 10 items, each item has 6 response options, and the scale range is 0-100 (high score equals high disability). It is based on a reflective model. You can view the ODI in full by clicking here: ODI

For this you need to download the dataset: data-interpretation.zip (unpack the zip-file). The dataset can be read by Stata 12-16.

1.1 Find the proportion of improved patients in the population?

Hint

Use the summarize command to answer the question.

Question

1.1 What is the proportion of improved patients in the population and why is it important?


1.2 Perform the logistic regression used to calculate the MIC-predictive

Hint

Use the logit command with the anchor (anc) as the dependent variable and the ODI change score (odich) as the independent variable.

Question

1.2 What are the intercept C, the regression coefficient Bx and their SE’s (standard errors)?


1.3 Calculate the correlation between the intercept C and the regression coefficient Bx

Hint

Right after the logistic regression you make a postestimation using the estat vce, corr command.

Question

1.3 What is the correlation between C and Bx?


1.4 Calculate the MIC-predictive and CI’s using the Excel spreadsheet available at the homepage

Hint

Question

1.4 What is the MIC-predictive and it’s CI’s?


1.5 Using the formula in the slideshow, calculate the adjusted MIC-predictive value

Hint

The formula is as follows:

MIC-adjusted = MIC-predictive − (0.09 + 0.103 x cor) x SD-change x log−odds(imp)

Where

These coefficients are found by using the following commands:

esize twosample odich, by(anc) pbcorr     // Point biserial correlation
sum odich     // SD of the change score
sum anc     // Proportion of improved patients

Questions

1.5.1 What is the adjusted MIC-predictive?


1.5.2 Describe what the MIC-predictive value and the CI’s mean. How can you use it?


2. van der Windt: Responsiveness & interpretation

This part requires that you have read the article:

van der Windt DA, van der Heijden GJ, de Winter AF, Koes BW, Deville W, Bouter LM. The responsiveness of the Shoulder Disability Questionnaire. Ann Rheum Dis. 1998;57(2):82-7. You can find it here.

Article summary

Some years ago, the responsiveness of the Shoulder Disability Questionnaire (SDQ) was evaluated in a general practice setting in patients with shoulder pain. The SDQ was compared with the Pain Severity Score (PSS) and Functional Status Questionnaire (FSQ).

The SDQ (shoulder Disability Questionnaire) consists of 16 items and is scored on a scale from 0–100, with higher scores indicating more severe disability (see Appendix 1 at the end of the page).

The PSS (Pain Severity Scale) is a single question about the severity of pain, scored on a scale of 0–10. The PSS is also converted to a scale of 0–100, with higher scores indicating more severe pain.

The FSQ (Functional Status Questionnaire) consists of a three-point scale (1: little discomfort during daily activities; 2: much discomfort during daily activities; 3: unable to perform daily activities).

Clinical improvements and deterioration were documented through self-reported changes since the beginning of the episode. No change and little improvement were considered clinical stability. Measurements were taken upon entrance into the study and at one and six months’ follow-up.

In the study, responsiveness was assessed in terms of Guyatt’s responsiveness ratio and a ROC curve. The most important findings are presented in Table 3 and Figure 1 (see below).

Table 3. Mean change scores (SD) and responsiveness ratios for the SDQ and PSS after 1 and 6 months.

* Substitution of missing values was conducted for patients reporting complete recovery: for 74 patients at one month (PSS only) and for 157 patients at six months (PSS and SDQ),
† Responsivess ratio: the ratio of the mean change score in clinically improved patients to the variability (SD) of change in scores in clinically stable patients.

Figure 1. ROC curves for change scores of the SDQ, PSS and FSQ at 1 month.

Note: True positive rate (sensitivity) and false positive rate (100-specificity) are for discriminating between patients reporting clinical improvement or clinical stability. Potential cut off points for the SDQ-change-score = 18.75: sensitivty 74%, specificity 77% (optimal trade off); SDQ-change-score = 40: (mean change in clinically improved patients) sensitivity 46%, specificity 98%.

Questions

2.1 How were the responsiveness ratios for the SDQ and PSS calculated? Can you check these calculations using the presented data?


2.2 Calculate the smallest detectable change (SDC) and the limits of agreement (LOA) for the SDQ after the one-month follow-up. Clarify the possible difference between these two measures and explain which one you prefer in this case.


2.3 What do think about the chosen external criterion? How would the SDC and responsiveness ratio change if the category ‘little improvement’ was considered ‘clinical improvement’?


2.4 On the basis of the data in Table 3, draw an anchor-based MIC distribution after one month for the SDQ: distribution of scores for the group without clinically relevant improvement or deterioration and distribution of scores for the group with clinically relevant improvement.


2.5 Estimate the MIC value with optimal sensitivity and specificity for the SDQ using the ROC curve.


2.6 How would the MIC value change if the category ‘little improvement’ was considered ‘clinical improvement’?


Appendix 1. The Shoulder Disability Questionnaire