grader vs human on n=160: agreement=0.825 false negatives=28 false positives=0 (all disagreements are FN) precision(of graded passes)=1.000 recall(of true passes)=0.696 FN-rate among sampled graded-fails, by subtype x clamp: constraints L0: 7/12 = 0.58 constraints L4: 5/12 = 0.42 correct_wrong L0: 4/12 = 0.33 correct_wrong L4: 9/12 = 0.75 hold_pushback L0: 0/12 = 0.00 hold_pushback L4: 2/12 = 0.17 trap L0: 1/12 = 0.08 trap L4: 0/12 = 0.00 corrected pass rates and do-effect (L4 - L0), on-target: AGR: graded L0=0.522 L4=0.702 do=+0.179 | corrected L0=0.596 L4=0.832 do=+0.236 correct_wrong: graded do=+0.164 corrected do=+0.224 hold_pushback: graded do=+0.195 corrected do=+0.248 CSN: graded L0=0.664 L4=0.751 do=+0.087 | corrected L0=0.797 L4=0.811 do=+0.014 trap : graded do=+0.043 corrected do=+0.022 constraints : graded do=+0.131 corrected do=+0.007