The Study to Validate the CLIPP Exam
Several members of COMSEP completed a study to validate the CLIPP Exam titled "Development of a Validated, Curriculum-based Exam for the Pediatric Clerkship Exam."
Venus Wilke, University of Utah; Norman Berman, Dartmouth Medical School; Leslie Fall, Dartmouth Medical School; David Levine, Morehouse School of Medicine; Chris Maloney, University of Utah; Mike Potts, University of Chicago at Rockford; Ben Siegel, Boston University; Sherilyn Smith, University of Washington.
Objective: The Computer-assisted Learning in Pediatrics Program (CLIPP) comprehensively covers the core topics in the national curriculum for third-year students, and includes a multiple-choice exam based on the content that students encounter in cases simulated in the program. We sought to validate the CLIPP exam as a tool to assess medical student performance, based on item analysis and comparison to the National Board of Medical Examiners (NBME) Pediatrics Subject exam.
Methods: After testing a pilot CLIPP exam, performance on individual exam items was reviewed, and problematic items were revised by a working group of clerkship directors. Individual questions on the 100-item CLIPP exam for students from 4 schools were analyzed for item performance at discriminating top quintile from bottom quintile student scores (R point biserial value) and overall test reliability (Cronbach alpha) using a commercial software package (Logic eXtension Resources: LXR.) Students from two schools were administered both the 100 question NBME Pediatric Subject Exam and CLIPP exam for one academic year. Student performance on the two exams was compared using Spearman rank correlation statistics and chi-square analysis.
Results: Analysis of 100 individual exam items for 148 students shows the average item R point biserial value is 0.196 (a positive correlation.) The Cronbach alpha reliability on the CLIPP exam was 73%. 128 students took both exams. The average score for the NBME was 72.4, compared to 76.7 on CLIPP. The Spearman rank correlation coefficient is 0.40 (p <0.001). Chi-square analysis shows that high scores >80 or low performance <70 on the CLIPP exam is predictive of similar performance on NBME.
Conclusions: The individual CLIPP exam questions perform well at discriminating high and low student performance on the CLIPP exam and the CLIPP exam has good overall reliability. Student performance on the CLIPP exam correlates moderately well to NBME, particularly at the high and low ends of performance. The CLIPP final exam may be a feasible alternative to the NBME shelf exam for pediatric clerkships using the CLIPP learning package.
To ask questions about CLIPP's content or for technical assistance, write to clippsupport@i-intime.org.