--- created: 2022-12-08T17:19:33-05:00 updated: 2022-12-08T17:19:33-05:00 --- topic: [[Assessing]] people: #people/waltereppich created: 2023-03-23 *Adding science to the political process of rating people's performance.* Rater Training is the process of selecting a rating tool, then helping to increase the accuracy of the people who are doing the rating using that tool. Rater training matters because if we are going to rate, we should have a sense of how accurate those ratings are, both to provide a validity to the ratings, and to provide a foundation for feedback. “As Latham and Wexley (1994) point out, it takes time and resources (two of the rarest commodities in a human resource department) to train raters.” (“Performance appraisal: state of the art in practice”, 1998, p. 7) The first process is ![[Performance Dimension Training]] After PDT which familiarizes the raters to the rating system and allows for rehearsal ratings with feedback, the goal is increasing rater accuracy. ![[Hauenstein Table 11.1.jpg]] ##### What would the opposite argument be? tags: #note/idea ##### Sources: Barusch, A., Gringeri, C., & George, M. (2011). Rigor in Qualitative Social Work Research: A Review of Strategies Used in Published Articles. _Social Work Research_, _35_(1), 11–19. [https://doi.org/10.1093/swr/35.1.11](https://doi.org/10.1093/swr/35.1.11) Govaerts, M. J. B., Schuwirth, L. W. T., Van der Vleuten, C. P. M., & Muijtjens, A. M. M. (2011). Workplace-based assessment: Effects of rater expertise. _Advances in Health Sciences Education_, _16_(2), 151–165. [https://doi.org/10.1007/s10459-010-9250-7](https://doi.org/10.1007/s10459-010-9250-7) Holst, D., Kowalewski, T. M., White, L. W., Brand, T. C., Harper, J. D., Sorenson, M. D., Kirsch, S., & Lendvay, T. S. (2015). Crowd-Sourced Assessment of Technical Skills: An Adjunct to Urology Resident Surgical Simulation Training. _Journal of Endourology_, _29_(5), 604–609. [https://doi.org/10.1089/end.2014.0616](https://doi.org/10.1089/end.2014.0616) Li, H., Xiong, Y., Hunter, C. V., Guo, X., & Tywoniw, R. (2020). Does peer assessment promote student learning? A meta-analysis. _Assessment & Evaluation in Higher Education_, _45_(2), 193–211. [https://doi.org/10.1080/02602938.2019.1620679](https://doi.org/10.1080/02602938.2019.1620679) Lundh, A., Kowalski, J., Sundberg, C. J., & Landén, M. (2012). A Comparison of Seminar and Computer Based Training on the Accuracy and Reliability of Raters Using the Children’s Global Assessment Scale (CGAS). _Administration and Policy in Mental Health and Mental Health Services Research_, _39_(6), 458–465. [https://doi.org/10.1007/s10488-011-0369-5](https://doi.org/10.1007/s10488-011-0369-5) Raufaste, E., Eyrolle, H., & Marine, C. (1998). Pertinence Generation in Radiological Diagnosis: Spreading Activation and the Nature of Expertise. _Cognitive Science_, _22_(4), 517–546. Roch, S. G., Woehr, D. J., Mishra, V., & Kieszczynska, U. (2012). Rater training revisited: An updated meta-analytic review of frame-of-reference training: Rater training revisited. _Journal of Occupational and Organizational Psychology_, _85_(2), 370–395. [https://doi.org/10.1111/j.2044-8325.2011.02045.x](https://doi.org/10.1111/j.2044-8325.2011.02045.x) Rosen, J., Mulsant, B. H., Marino, P., Groening, C., Young, R. C., & Fox, D. (2008). Web-based training and interrater reliability testing for scoring the Hamilton Depression Rating Scale. _Psychiatry Research_, _161_(1), 126–130. [https://doi.org/10.1016/j.psychres.2008.03.001](https://doi.org/10.1016/j.psychres.2008.03.001) Smither, J. W. (Ed.). (1998). _Performance appraisal: State of the art in practice_ (1st ed). Jossey-Bass Publishers. Taipalus, A. C., Hixson, M. D., Decker, D. M., & Fredrick, S. (2021). The effects of an online DBR training module on rater accuracy. _School Psychology_, _36_(4), 224–234. [https://doi.org/10.1037/spq0000439](https://doi.org/10.1037/spq0000439) van der Vleuten, C. P. M., Schuwirth, L. W. T., Driessen, E. W., Dijkstra, J., Tigelaar, D., Baartman, L. K. J., & van Tartwijk, J. (2012). A model for programmatic assessment fit for purpose. _Medical Teacher_, _34_(3), 205–214. [https://doi.org/10.3109/0142159X.2012.652239](https://doi.org/10.3109/0142159X.2012.652239) Yule, S., Gupta, A., Gazarian, D., Geraghty, A., Smink, D. S., Beard, J., Sundt, T., Youngson, G., McIlhenny, C., & Paterson-Brown, S. (2018). Construct and criterion validity testing of the Non-Technical Skills for Surgeons (NOTSS) behaviour assessment tool using videos of simulated operations. _British Journal of Surgery_, _105_(6), 719–727. [https://doi.org/10.1002/bjs.10779](https://doi.org/10.1002/bjs.10779)