Read our COVID-19 research and news.

multiple scientists in a laboratory
Hero Images Inc. / Alamy Stock Photo

Student performance measures that don’t perform

GRE scores and undergraduate GPA don’t predict students’ future graduate school productivity, but reference letters from previous research advisers may provide clues about whether they are going to publish well, according to two papers published today in PLOS ONE. These results add to the ongoing discussion about how graduate admissions decisions should be made, particularly in light of previous findings that the GRE is biased against students from underrepresented groups. The new studies emphasize that admissions committees should review applicants holistically and rely less on GRE scores in making decisions—a point that many acknowledge, but which requires significant time and energy to do well.

By tracking the research output of students who recently completed biological and biomedical doctorates at the University of North Carolina (UNC) in Chapel Hill and Vanderbilt University in Nashville, the researchers found that neither GRE scores nor GPA were associated with the number of first-author papers a student published. Moreover, these metrics did not predict other quantitative measures of productivity—the number of conference presentations and independent grants and fellowships received—or the student’s progress through the program, further analysis of the Vanderbilt cohort revealed. The UNC study also found that there was not a correlation between the extent of a student’s previous research experience and their publication record, although all the students had done some research prior to starting graduate school. But, among the UNC group, students with the strongest reference letters produced more first-author papers, while those with weaker letters tended to publish as middle authors or not at all.

Publication record, and particularly number of first-author publications, has its own flaws as a productivity metric: It’s dependent on the adviser and the nature of the project, and the number does not reflect the quality of the papers or of the student’s contribution. However, it’s a helpful measure because it’s objective and quantifiable in the short term, says corresponding author of the Vanderbilt study Roger Chalkley, senior associate dean of biomedical research education and training. And having at least one first-author publication is a common graduation requirement, notes UNC study lead author Joshua Hall, director of science outreach and the UNC Postbaccalaureate Research Education Program, so it’s useful in categorizing whether a student’s productivity meets expectation.

The finding that strong reference letters are correlated with greater productivity aligns with the emphasis that many professors serving on admissions committees place on letters from students’ research mentors (although reference letters have their own shortcomings and are subject to bias). Orion Weiner, a professor of biochemistry and biophysics at the University of California, San Francisco, who has also studied grad school admission metrics, notes that letters presenting specific details of the applicant’s contributions to the lab are most effective.

In line with the results of the new studies, Weiner found that scores on the general GRE exam and undergraduate GPA did not distinguish between high- and low-performing graduate students in his program. But approaches to using GRE scores and GPA for grad school admissions continue to be mixed. Weiner believes that GREs are not useful, and Leslie Vosshall, a professor of neurogenetics and behavior at The Rockefeller University in New York City who recently started a Twitter discussion about GRE use, agrees: She never considers GRE scores because of its reported biases. She also finds GPA largely irrelevant because of the lack of uniformity in grading standards between undergraduate institutions. Weiner, on the other hand, uses GPA to triage applicants, though he’ll still consider candidates with very low GPAs if something else in their application is outstanding. And Leslie Kay, a professor of psychology at the University of Chicago in Illinois who chimed into the Twitter discussion, also considers GPA when she evaluates applicants because she believes that being a good scholar is as important a qualification as being a good researcher. Like Weiner, she does not necessarily consider a low GPA to be completely disqualifying—but what she looks at to counter it is the GRE score. If both are low, she takes that as a sign that the applicant is not the best candidate for her program.

A caveat for both the new studies and Orion’s work is that, because they examined only accepted students, minimum GPA and GRE scores were effectively built into the data sets: Most of the students had undergraduate GPAs higher than 3.0 and GRE scores above the national average. So it may be that there is a correlation between scores and future productivity for students with lower GPAs, Weiner says, or with GRE scores below a certain level, Kay adds.

Even so, the studies show that students with high GRE scores and high GPA do not necessarily do better as graduate researchers than those with lower metrics, Hall says. Vosshall is an example. When she was applying for graduate programs, she was an active undergraduate researcher but had mediocre GRE scores and a 3.5 GPA, with a few Cs in her science classes. Now she’s an established professor. “I turned out OK,” she says.

*Correction, 12 January, 3:30 p.m.:  This article has been corrected to reflect that most students studied had GPAs above 3.0, not all.