Shaw University
Moodle Give to Shaw  
Compliance Assist Portal 
Feedback and Reports 
bENCHMark & DISH System 
WeaveOnline Portal 
Annual Planning 
External Resources 
Provide Feedback

Location: 206 Estey Hall 
Phone: 919-546-8417
Fax: 919-546-8236

Hours of Operation:

M-F 8:00 a.m. to 5:00 p.m.
Home > Administration > OSPIRE

Make Your Rubric Roundly Robust

Rubrics, Reliability, and Results - The 3 Rs 2.0

According to Wikipedia, the free encyclopedia, "In statistics, inter-rater reliability, inter-rater agreement, or concordance is the degree of agreement among raters.  It gives a score of how much homogeneity, or consensus, there is in the ratings given by judges.  It is useful in refining the tools given to human judges, for example by determining if a particular scale is appropriate for measuring a particular variable.  If various raters do not agree, either the scale is defective or the raters need to be re-trained ("  

As you prepare to modify your rubrics for 2014, please call OSPIRE so that we can develop an inter-rater agreement and reliability check for your faculty members.  The whole process is slow and painful, but the yield is worth the effort.  Without this process, it is impossible to gauge the validity of the scores obtained from your rubrics.  If you have questions about this process, or would like to schedule a session, please use our convenient web-based service request form.