Unexpected Direction of Differential Item Functioning PDF Download

Author: Sangwook Park
Publisher:
ISBN:
Category : Educational tests and measurements
Languages : en
Pages :

Get Book Here

Book Description
ABSTRACT: Many studies have been conducted to evaluate the performance of DIF detection methods, when two groups have different ability distributions. Such studies typically have demonstrated factors that are associated with inflation of Type I error rates in DIF detection, such as mean ability differences. However, no study has examined how the direction of DIF is affected by the factors that inflate the Type I error rate. Therefore, this study investigated the possibility that the direction of DIF is systematically detected in an unexpected way, which may result in unexpected detection of DIF advantaging lower ability groups on difficult items. An extensive simulation was conducted to evaluate whether DIF in unexpected directions was observed systematically under the logistic regression approach to DIF detection. Four factors were considered in this study: 1) means of ability distributions, 2) standard deviations of ability distributions, 3) sample sizes, and 4) the magnitude of the pseudo-guessing parameters. Three levels were considered for the ability means, and two levels were considered for the ability standard deviations. Three levels were examined for sample sizes and guessing parameters. As a result, 54 (3 x 2 x 3 x 3) simulation conditions were considered. In addition, items were grouped into five groups depending on their difficulties; very easy, easy, moderate, difficult, and very difficult. The effects of the four simulation factors were evaluated for each one of the five item-difficulty groups. For each condition, 500 replications were conducted. DIF error rates, bias, SE, RMSE, the direction of DIF, and distributions of the biases were examined to evaluate the effects of the four simulation factors. The results revealed that the mean of the ability distribution and the magnitude of the pseudo-guessing parameters indeed contributed dramatically to inflation of DIF error rates, especially for very easy and very difficult items. Moreover, the directions of DIF were all negative for very easy items, but all positive for very difficult items. Finally, limitations and practical implications were discussed.