Testlet Response Theory and Its Applications

Author: Howard Wainer; Eric T. Bradlow; Xiaohui Wang  

Publisher: Cambridge University Press‎

Publication year: 2007

E-ISBN: 9780511276163

P-ISBN(Paperback): 9780521862721

Subject: G424.7 Management and Examination Achievement

Keyword: 概率论与数理统计

Language: ENG

Access to resources Favorite

Disclaimer: Any content in publications that violate the sovereignty, the constitution or regulations of the PRC is not accepted or approved by CNPIEC.

Testlet Response Theory and Its Applications

Description

The measurement models employed to score tests have been evolving over the past century from those that focus on the entire test (true score theory) to models that focus on individual test items (item response theory) to models that use small groups of items (testlets) as the fungible unit from which tests are constructed and scored (testlet response theory, or TRT). In this book, the inventors of TRT trace the history of this evolution and explain the character of modern TRT. Written for researchers and professionals in statistics, psychometrics, and educational psychology, the first part offers an accessible introduction to TRT and its applications. The second part presents a comprehensive, self-contained discussion of the model couched within a fully Bayesian framework. Its parameters are estimated using Markov chain Monte Carlo procedures, and the resulting posterior distributions of the parameter estimates yield insights into score stability that were previously unsuspected.

Chapter

2.1. The Spearman–Brown formula for a test of double length

2.2. Which split half? Cronbach’s coefficient Alpha

2.3. Estimating true score

2.4. A model for error

Questions

References

3 Item response theory

3.1. An obiter dictum on P

3.2. Returning to the 1-PL

3.3. Estimating proficiency

3.4. Example of how ICCs multiply to yield the likelihood of proficiency

3.5. On the accuracy of the proficiency estimate

3.6. Estimating item parameters

3.6.1. Notation and general principles

3.6.2. Maximum marginal likelihood estimation

Questions

References

4 What’s a testlet and why do we need them?

4.1. Introduction

4.2. Problems

4.2.1. Context effects

What is a context effect?

Item location

Cross-information

Unbalanced content

4.2.2. Item difficulty ordering

4.2.3. Summary of context effect problems

4.2.4. An issue of robustness

4.3. Traditional solutions

4.3.1. Context effects

4.3.2. Robustness

4.3.3. Item difficulty ordering

4.4. Testlets – an alternative solution

4.5. An initial summing up

Questions

References

5 The origins of testlet response theory – three alternatives

5.1. Bock’s model

5.2. Summary and conclusions

Questions

References

6 Fitting testlets with polytomous IRT models

6.1. Introduction

6.2. Reliability

6.2.1. Methodology

6.2.2. An IRT model for testlets

6.3. Testlet-based DIF

6.3.1. Matching the model to the test

6.3.2. DIF cancellation

6.3.3. Increased sensitivity of detection

6.4. Methodology

6.4.1. Testlet DIF detection

Bock’s 1972 model

6.5. Results

6.5.1. Overall performance

6.5.2. Reliability

6.5.3. Differential performance

6.5.4. A testlet examination of White–African-American performance on Reading Comprehension: a brief case study

6.5.5. Standardized total impact

6.6. Conclusions

Questions

References

PART II Bayesian testlet response theory

Recapitulation and introduction

References

7 A brief history and the basic ideas of modern testlet response theory

7.1. Introduction. Testlets as a multidimensional structure

7.2. Advances in estimation and computing: essential developments to allow us to include items as part of the testlets

7.3. Why Bayesian?

7.4. Modeling testlets: introducing the testlet parameter…

7.5. Summing up

Questions

References

8 The 2-PL Bayesian testlet model

8.1. Introduction

8.2. The model specification

8.3. Computation under the Bayesian 2-PL TRT model

8.3.1. The JML and MML estimation approaches

8.3.2. Maximum a posteriori estimation

8.3.3. Bayesian estimation using Markov chain Monte Carlo methods

8.4. A demonstration of the efficacy of the Bayesian 2-PL testlet model

8.5. A roadmap for the remaining computational chapters

Questions

References

9 The 3-PL Bayesian testlet model

9.1. The 3-PL Bayesian testlet model

9.2. Varying local dependence within testlets

9.3. Simulation results for the Bayesian 3-PL testlet model

9.4. An application of the Bayesian 3-PL testlet model to GRE data

9.5. Summary

Questions

References

10 A Bayesian testlet model for a mixture of binary and polytomous data

10.1. Bayesian testlet model for binary and polytomous data

10.2. Simulation results for the Bayesian mixed testlet model

10.3. The application of the Bayesian mixed testlet model to the North Carolina Test of Computer Skills

10.4. A summary

Questions

References

11 A Bayesian testlet model with covariates

11.1. An extended Bayesian TRT model

11.2. Example 1: USMLE data

11.2.1. Modeling the data

11.2.2. The results of the covariate augmented analysis

11.3. Example 2: Using testlet response theory to understand a survey of patients with breast cancer

11.3.1. Introduction

11.3.2. The results

11.3.3. Using covariates to understand why

Using regression analysis

11.3.4. Using the posterior distributions of the covariate coefficients

11.3.5. What was the effect of local dependence?

11.3.6. Discussion

11.4. Conclusion

Questions

References

12 Testlet nonresponse theory: dealing with missing data

12.1. Missing at random and missing completely at random

12.2. Ignorable nonresponse

12.2.1. An illustrative example

12.3. Nonignorable nonresponse – omitted and not reached items

12.4. Conclusion

References

PART III Two applications and a tutorial

13 Using posterior distributions to evaluate passing scores: the PPoP curve

13.1. Introduction

13.2. PPoP curves

13.3. Example: the integrated clinical encounter score of Clinical Skills Assessment

13.4. Discussion and conclusion

Questions

References

14 A Bayesian method for studying DIF: cautionary tale filled with surprises and delights

14.1. Introduction

14.2. A Bayesian procedure for measuring DIF

14.2.1. The model

14.2.2. Measuring DIF

14.3. The data

14.4. A practical procedure and some results

14.5. Simulation studies

14.5.1. Small-scale study

14.5.2. Large simulation study

14.5.3. Interpretation of the simulations

14.6. Conclusions

Questions

References

15 A Bayesian primer

15.1. General Bayesian inference problem

15.2. Bayesian inference for IRT models

15.3. Computation for the Bayesian IRT model

15.4. Some advice for those using Bayesian IRT models

Questions

References

Glossary of terms

Epilogue

Bibliography

Author Index

Subject Index

The users who browse this book also browse