Portfolio review FAQs
- Our program makes initial cuts based on standardized test scores, after which we review all the materials of applicants who made the cut. That means we use portfolio review, right?
Making the initial cut on test scores is not consistent with portfolio review practice. As noted above, initial cuts on scores can eliminate qualified students from your applicant pool and affect your programs in the long run. The Graduate School's guidance above can help programs develop a process that can be structured and efficient.
- How can we implement portfolio review if federal funding agencies evaluate GRE scores for fellowship applications?
Note that GRE scores are no longer part of the Graduate Research Fellowship Program at NSF (https://www.nsfgrfp.org/applicants/faqs). In addition, NIH has eliminated standardized test scores from individual fellowship applications (F30 and F31): http://grants.nih.gov/grants/guide/notice-files/NOT-OD-15-120.html.
- Other high-profile programs require the GRE. If we do not, won't we lose applicants because our program won't look as competitive?
- Doesn't the WSU Graduate School require the GRE on internal scholarship and fellowship applications? No, the WSU Graduate School does not collect standardized test score information on scholarship and fellowship applications. In fact, the Graduate School has incorporated holistic review into many of its funding mechanisms. For instance, the Graduate Professional Scholarship essay allows students to discuss evidence of persistence and the ability to overcome obstacles. The Dean's Diversity Fellowship requests that programs evaluate candidates based on Inclusive Excellence principles, including the extent to which the student will contribute to diverse perspectives in the program and/or the discipline and how diversity and inclusion efforts benefit the program and/or research to be undertaken. Finally, a new endowed scholarship was created by Dean Ambika Mathur to further support portfolio review and retention (details and application information forthcoming).
- Our program makes a first cut using standardized test scores because our program demands high test scores to succeed in coursework. The decision to create cuts is often based on tradition, anecdotal or personal experience, and the availability of certain cases in our memories that fit predicted patterns (the so-called "availability heuristic" in social science research; Tversky & Kahneman, 1976). This is not necessarily bad if we also attend to cases that do not fit our expectations (e.g., high scoring students who have difficulty completing their degree; low scoring students who perform well). In other words, before the argument is made that a cut before portfolio review is necessary, admissions data should be fully evaluated rather than relying on select cases. A growing body of evidence shows that standardized test scores are inconsistent predictors of graduate school success; several of these same studies have shown that undergraduate GPA is a more robust predictor than standardized exam scores (e.g., Kuncel & Hezlett, 2010; Morrison & Morrison, 1995; Pacheco et al., 2015; Sternberg & Williams, 1997; Weiner, 2014). We recommend that programs conduct and publish research on the impact of their selection process using data from their own departments to contribute to this growing and important literature. The data can also be used to develop admissions procedures as described below.
- Isn't "portfolio review" vague? Won't it lead to decisions that are not consistent across applications, causing complaints and lawsuits? Portfolio review requires the evaluation of multiple pieces of data to come to an admissions decision. Best practice is for programs to identify what the desired qualities and experiences are of ideal applicants before the process begins and to operationalize those qualities and experiences. We provide two rubrics in the accompanying materials to assist departments in making consistent decisions across applications and to document their decision-making process. It is essential that programs undertake an intentional and documentable approach to admission because research has shown that, even with the best of intentions, faculty are subject to biases that adversely affect their ability to make admissions decisions based on objective data (Posselt). We also recommend that programs conduct research on their admissions rubrics to determine the impact of such methods on their admissions process and make necessary improvements over time.
- We already do portfolio review of materials, but why do we need to have rubrics?
We commend programs that have already put in place portfolio review procedures, and would love to learn of your processes so we can share best practices across campus. Keep in mind that portfolio review does not simply entail looking at the whole application but also involves operationalization of constructs and some standardization (e.g., consistent use of rubrics). As noted above, it is essential that programs undertake an intentional and documentable approach to admission because research has shown that, even with the best of intentions, faculty are subject to biases that adversely affect their ability to make admissions decisions based on objective data (Posselt, 2014).
- What do programs and individual faculty members get out of portfolio review? Portfolio review processes are expected to lead to more consistent and documentable review processes to ensure against biases (even well-meaning ones) and to enhance programmatic and student success consistent with the program goals and strategic goals of WSU. In addition, The Graduate School is offering recruiting funds for graduate programs that undertake a holistic review process and provide evidence of their portfolio review activities. Contact Associate Dean of Student Success Sharon Lean, firstname.lastname@example.org.
- Won't portfolio review result in less qualified applicants gaining admission? Our first question would be to ask how "less qualified" is operationalized. Based on research evidence reviewed earlier, we reject the notion that the standardized test scores are the sole indicator of graduate student success. A second point is that the question presumes that efforts to achieve diversity are at odds with efforts to achieve excellence. As noted by Williams, Berger, and McClendon (2005), the perceived relationship between diversity and excellence varies from institution to institution. According to the Inclusive Excellence Change Model, diversity is a critical component for achieving excellence. Students and faculty attain excellence by working in diverse groups, which can stimulate creative and novel approaches to problem-solving and offer skills development to work in an increasingly culturally diverse society. WSU's strategic plan is aligned with this conceptualization of Inclusive Excellence. The plan includes objectives to "Celebrate and increase the understanding and appreciation of diversity and inclusion," "Design and implement recruitment strategies that result in increased numbers of qualified and diverse underrepresented students, faculty, and staff," and "Develop and enhance programs focused on understanding multiculturalism and building diversity and inclusion competencies and expertise." Other institutions' efforts to embrace portfolio review do not appear to have lessened excellence in student outcomes, but we understand that this is an empirical question. WSU intends to conduct research to explore this question locally and will disseminate the results to the campus community.
- How can faculty and staff learn more? For consultation on portfolio review, contact Associate Dean of Student Success Sharon Lean, email@example.com.