Strategies Of A 2-Level Model

832 Words4 Pages
We used multilevel modeling (MLM; Hox, 2002; Raudenbush & Byrk, 2002) to evaluate the efficacy of feedback. MLM was used due to the nested data structure (i.e., clients nested within therapists). Given this, the assumption of independent observations is not met which can result in increase of Type I error rates; therefore, traditional analytical strategies such as analysis of variance are not appropriate. Specifically, MLM permits us to control for the variance effects from therapists to evaluate the effects of client feedback at the client level. We also will be able to see if the rates of change for each treatment condition were different.
Using an ANCOVA-like strategy, we constructed a 2-level model (client within therapist)
…show more content…
We also replicated this model to evaluate the effects of feedback on clients who were identified as not-on-track.
Effects of Client Feedback on Outcome – All Clients Descriptive statistics are shown in Table 1. The results from the covariate-only model (Equation 1) are shown in Table 2. This baseline model estimated covariance parameters to compute an intraclass coefficient (ICC) to determine the amount of variability at the therapist level. The ICC showed that less than 1% of the variance (.02) was accounted for by therapist outcome differences, which is smaller than other PCOMS studies that evaluated therapist effects (Anker et al., 2009; Reese et al., 2011). The covariate-only model indicates a significant positive slope (10 = .26, p < .001) between initial ORS scores and post-ORS scores across clients. This indicated that ORS scores significantly improved for all clients from pre- to post-treatment, with the average post-ORS for clients equal to 29.31 (00). The standardized mean effect size from pre- to post-ORS was 1.38.
The ANCOVA model included the treatment condition (FEEDBACK) as a predictor to the first model as shown in Equation 2. We refer to this as an ANCOVA model given that we are evaluating the differences in slopes for each treatment condition while controlling for initial ORS scores. Clients in the feedback condition demonstrated significantly more improvement
Open Document