Multiple Measures Assessment Project

The Multiple Measures Assessment Project (MMAP) is a collaborative effort led by the RP Group and Educational Results Partnerships’ Cal-PASS Plus system, with support from the CCCCO.

The more than ten million students attending community colleges represent nearly half of all students in higher education in the United States each year. Along with millions of additional students at open access four-year institutions, these students arrive at college at a variety of levels of preparation, critically requiring accurate assessment of student readiness for college-level work. The vast majority of open access institutions rely primarily, if not exclusively, on standardized placement exams to make initial course placement decisions The results of these exams typically place two thirds of community college students (and more than 85% in California) into one or more developmental or remedial courses. Placement into developmental education can help provide students additional preparation for college-level work but research increasingly demonstrates that very few students placed into developmental education complete the sequence in that discipline, much less any meaningful educational outcome despite very substantial direct and opportunity costs.

However, placement tests are only weakly correlated with course and college outcomes and as many as one-third of students placed into remedial courses could have earned a B or higher in an associated college-level course. Further, despite the costs of time and resources, assignment to developmental education, if anything, reduces students’ likelihood of completion among otherwise identical students, particularly among those severely misplaced. As a result, colleges and systems have increasingly explored alternative methods introducing additional or “multiple” measures into admission, assessment, and placement into college-level work, relying on evidence, for example, that among both two-year and four-year institutions, high school GPA is a stronger predictor of course performance as well as college GPA, persistence, and graduation. Very promising evidence suggests that students’ placed into college-level coursework using multiple measures (e.g., students with a B- or better overall high school GPA) appear to perform just as well or better than students placed into the courses by standardized placement tests and to complete the college-level courses much sooner, saving, on average, 1-2 semesters of unnecessary developmental education per student and completing the college-level course in the sequence at rates between two to five times higher than students placed via usual procedures. Moreover, the effects appear to be shared across demographic groups, even showing evidence of potential to reduce equity gaps in the achievement of early educational milestones for underrepresented students of color.

Although the research is exceptionally promising with powerful implications for transformation of student outcomes and meaningful potential for significant reduction in college costs to students and to the state and federal governments, to date the research has been either largely predictive or, among what little evaluation of the effectiveness of these alternative placement practices has occurred, has continued to rely exclusively on either convenience samples or various correlational research techniques, forestalling more widespread adoption awaiting more definitive evidence. Given the enormous scale of both the direct and opportunity costs of developmental education ($4.6 billion in Pell grants alone for students placed into developmental education) and the number of states in the process of implementing this or highly similar developmental reforms at or near scale, this project seeks to address this pressing research question more directly with a multi-site, randomized controlled trial of the use of evidence-based multiple measures to increase the accuracy of placement in open access higher education in cooperation with multiple colleges in three states that are at early stages of adoption.

Decision Rules and Analysis Code

Phase II Rule Sets

Updated - Phase 2 Decision Rules for English, Math, Reading and ESL (May, 2016) - This document is the summary of decision rules for placing students in English, Math (now including Trigonometry), Reading, and ESL (Top Level L1) from the second phase of the project.

Math Decision Trees Output - Phase 2 (April 2016)- This document provides decision tree output used for the creation of high school transcript based placement rules for Phase 2 of the multiple measures project.

English Decision Trees Output - Phase 2 (January, 2016) - This document provides decision tree output used for the creation of English high school transcript based placement rules for Phase 2 of the multiple measures project.

Reading Decision Trees Output - Phase 2 (May, 2016) - This document provides decision tree output used for the creation of Reading high school transcript based placement rules for Phase 2 of the multiple measures project.

ESL Decision Trees Output - Phase 2 (March, 2016) - This document provides decision tree output used for the creation of ESL high school transcript based placement rules for Phase 2 of the multiple measures project.

Phase 2 Decision Rules - Presentation with examples of the phase 2 rule sets - Presented at the October 2015 In-Person Convenings

Creating College Level Decision Rules

Retrospective Analysis – This R Code can be used to create the decision rules for your college using the Retrospective file from Cal-PASS Plus


Visit this page below to find numerous practical tools for implementing multiple measures as well as publications, presentations, webinars, etc. produced by the project.



EDUCATIONAL RESULTS PARTNERSHIP
2300 N STREET, SUITE 3, SACRAMENTO, CA 95816
P: 916-498-8980