In 2012, three friends were laughing about the academic rat race as they toiled away in a computational biology lab at the Weizmann Institute of Science in Rehovot, Israel. David van Dijk and Ohad Manor were both Ph.D. students, and Lucas Carey was doing his first postdoc. While scanning through PubMed, the vast online database of biological abstracts, van Dijk had an idea. Why not scrape PubMed's citation data, build a statistical model of people's academic careers based on their publication records—total number of journal papers, number that are first author, and the impact factor (IF) of those journals—and find out what factors are most important for career success? The key assumption is that the last name on a many authored paper is the principal investigator (PI), which, they figured, is a decent proxy for success in an academic science career.
In the 2 June issue of Current Biology, the trio debuted this scientific career crystal ball. (You can find the original article here. Also see our PI Predictor app, which lets you determine the probability that you'll become a PI, here.) Manor and van Dijk are now postdocs at the University of Washington, Seattle and at the Weizmann Institute, respectively, while Carey has landed a job as a PI at Pompeu Fabra University in Barcelona, Spain. They spoke with Science Careers about their deep dive into bibliometrics, meritocracy in research, and the "moneyball" approach to scientific hiring. The interview is edited for clarity and brevity.
The fact that having many publications helps is not surprising, but the fact that they only help when they are either in high IF journals or if you are first author is more surprising.
Q: What interested you in scientific careers as a research topic?
D.V.: Every grad student or postdoc dreams of getting a Cell, Nature, or Science (CNS) paper, not just for the fame, but more concretely to secure a job. We wondered whether it would be possible to actually quantify the effect of a CNS paper on getting a job.
O.M.: When David suggested to actually try to answer this question in a quantitative and data-driven way, I was very interested in applying machine-learning techniques to this problem.
Q: Isn't it a career risk to take a break like this from your own research?
D.V.: What I like most about science is that I can be creative, so once in a while I like to take a break from biology and try to use my skills to tackle other more general questions. This project has been a lot of fun and seems to be getting more attention than I had anticipated. I'm looking forward to continuing my career as a computational biologist, but perhaps in the near future I'll make another small "detour" if other interesting questions come along.
Q. The model shows that publishing lots of papers, particularly first-author papers and in journals with a high IF, makes the biggest difference. Isn't that what you expected?
D.V.: It was surprising to see how predictive publication record is in general for becoming a PI. The fact that having many publications helps is not surprising, but the fact that they only help when they are either in high IF journals or if you are first author is more surprising.
Q. Isn't it depressing that "publish or perish" seems to be the driver of scientific success after all?
D.V.: On the one hand, these results are encouraging, because they suggest that people are promoted based on merit (of past publications). On the other hand, many of the most groundbreaking papers were not published in high IF journals, and did not initially receive a high number of citations. These groundbreaking authors would perhaps not have been given a position. The easiest and most sensible way to judge people you don’t know is probably by their past work, especially for funding agencies and the initial screen of the hiring committee when they are looking through hundreds of applicants. However, this filtering method will certainly miss some phenomenal and ahead-of-their-time scientists.
L.C.: We can’t say if this predictability is a good thing for science.
O.M.: In my eyes, the predictability in the academic-career game is not necessarily a bad thing. Generally speaking, it looks as if it makes sense. Publishing more high profile, first-author papers [translates to] more chances of becoming a PI.
D.V.: However, being a successful scientist and group leader is more than just producing papers. The best way to interpret our model's prediction is that it compares you to other scientists with a similar publication record. So, while I think it is good to know where one stands compared to other scientists, as academia is very competitive, becoming a successful independent scientist and group leader is much more than just the sum of publications. For example, our model doesn’t take “soft skills” into account. I can imagine that, especially during a job interview or job talk, one could make up for a weaker publication record by showing leadership, vision, and social skills. Committees might just like you because you seem to be a nice person to work with. For sure I can see the opposite happening, where a candidate looks amazing on paper but less so in real life.
Q. You found that women had a lower chance of becoming a PI compared to men with the same publication record. Isn't that evidence of gender bias in hiring?
D.V.: We can't say for sure that the system is biased for gender, i.e. if the difference is due to external factors such as universities preferring to hire men, or due to internal factors such as women having less desire to become a PI. On the other hand, we do find that when we correct for CVs, men are still more likely to become PIs, suggesting that there is a bias or that gender correlates with something that we are not measuring. I do hope that our results inspire others to think about the current system and perhaps they will find ways to improve it.
Q. What other unknowns are there beyond your model's scope?
O.M.: One of the top things we don't know is who wanted to be a PI to begin with. Ideally, if we could survey Ph.D. students and postdocs once a year and record how much they want to be PIs, and then follow up on how this changes with publication record, and find out who eventually becomes a PI, I think we could both model the outcome better and also better understand the decision/career process.
D.V.: Since we find that journal IF is an important predictor for becoming a PI, it would be interesting to ask if getting a CNS paper before you become a PI correlates with scholarly output after you become a PI. Of course, we should correct for institution ranking, since having better publications will likely give you a job at a higher ranking university. However, it would be interesting to measure if pre-PI success is predictive of success as a PI.
Q. What about regional and national differences?
D.V.: It will be interesting to run our model for different countries. We could measure if there are differences in "fairness" between countries—in which countries are you more or less judged by your publication record? I can imagine that in some countries the academic system is more political or nepotism plays a stronger role. Also, we can then measure in which countries it is easier to become a PI, i.e. which countries require a lesser publication record. We could also do this per university. I would expect that highly ranked institutions require a better CV to become a PI.
Q. How much of a boost in your chances of becoming a PI does this Current Biology paper get you, according to the model?
O.M.: In my case I get a boost from 52% to 62%.
D.V.: For me, 71% to 81%.
Q. Should scientists worry if they get a low score?
D.V.: People should not make career changes based just on their score. I believe that in the end, the most important thing in science is to enjoy what you’re doing and keep being inspired. There is an element of luck in getting a CNS paper. So it can be frustrating if you think you are a good scientist and want to succeed, but that high IF paper doesn’t just happen. It’s encouraging that we find that doing good-quality science on a consistent basis—multiple first-author papers in journals with reasonable IF—does seem to be rewarded in the end. The strongest conclusion from our study is that becoming a PI is highly predictable, even after only a few years as a publishing scientist. It suggests the presence of positive feedback loops: Ph.D. students who publish early and publish often go on to have better careers.
Unfortunately, our results also have an, in our view, negative implication: Our data suggests that, unless you’re publishing in a very high-impact journal, a middle-author position really isn’t very helpful for your career, at least in terms of securing a job. So losing out the first-author position—or giving it to someone else—can be a very large sacrifice. Of course, helping others out (and receiving middle authorship in return) is important and can be useful in many other ways.