Crowdsourcing goes academic with platforms for reviewing advisers

Danika Khong was disappointed with her postdoc position. She had thought that it would make her competitive in the faculty job hunt. The principal investigator (PI) was a big name in the field and a prolific publisher. Khong’s project seemed a sure ticket to a first-author paper in a high-impact journal. But, a year and a half into it, things weren’t working out as she had hoped. Although she was publishing well, she wished her adviser mentored her more. She felt that she didn’t have a say in the project’s direction, and communication from her adviser was a one-way street. Many postdocs just want to publish and don’t mind this type of relationship, Khong notes. “It just didn’t fit me,” she says.

Khong knew her story was not unique. She met other postdocs who were not getting the training they had expected. And they were not only disappointed with their labs, she noticed. Some went through bouts of depression because of their frustration with their training. Others suffered anxiety because of their strained relationship with their PIs.

Khong saw a simple solution: Provide an avenue for early-career researchers to anonymously share information about their experiences in labs to help prospective students and postdocs make informed decisions. The resulting platform, Scismic, which Khong founded with labmates Claudia Dall’Osso and Elizabeth Wu, launched in November 2017.

Scismic joins a growing group of online platforms by trainees for trainees to find labs that match their needs. The peer-to-peer insights these websites aim to disseminate have traditionally been passed along through casual chats with lab members and people within a trainee’s network, and that means of communication is sure to continue. But the effort to broaden the reach with these review platforms could be particularly valuable for those who don’t have the opportunity for face time with prospective labs. A review website could also improve the quality of information circulating among students, adds Valerie Horsley, an associate professor of molecular, cellular, and developmental biology at Yale University who emphasizes mentoring and maintaining a collegial environment, as her lab website illustrates. Students could be basing their decision on rumors and isolated experiences, she worries. Platforms like Scismic “would just give more data points for people to evaluate on,” she says.

With modestly sized databases so far, the utility of review platforms is limited right now: As of 20 February, Scismic has collected reviews for 49 PIs at 38 universities in the United States and abroad, and GradPI, a similar effort launched in January 2017, has reviews for 93 PIs at 17 U.S. universities. Nevertheless, platforms like these could potentially be a powerful force to promote positive adviser-advisee relationships and empower early-career scientists to find environments that work for them.

The purpose isn’t to label PIs as “good” or “bad,” the founders of these platforms say. It’s about helping people find the right match. To that end, reviewers on Scismic fill out a 30-question survey covering topics including the PI’s mentoring style, the lab’s work-life balance, and the institution’s social environment to give users a complete picture of what life there could be like. Khong hopes the platform will also flag features of a training environment that rising scientists might not have thought of otherwise. During her postdoc search, for example, she hadn’t considered mentoring style because she had assumed that the nurturing approach she appreciated about her Ph.D. adviser was typical, she says.

On GradPI, former and current lab members rate PIs on five metrics: reputation, mentorship quality, degree of independence offered, funding, and interpersonal skills. Trainees should “choose the PIs based on whatever metrics are important to them,” emphasizes Gadareth Higgs, the Yale University Ph.D. student behind GradPI. For example, students looking for a strong mentor might not be turned off if a supportive PI is not well established in the field, and students who want to boost their publication record may not view infrequent meetings with a PI as a drawback. As Horsley tells her mentees looking for graduate and postdoc labs, no lab is perfect. “You have to evaluate for yourself if you think you can tolerate the things that are bad,” she says. “Does it have everything you need right now?”

Community sign-on

The value of a crowdsourced PI review website is an easy sell to rising scientists. Graduate and postdoc supervisors are important and long-lasting connections in a scientist’s career. The more information available to help choose a mentor, the better.

But getting trainees to open up about their experiences, especially bad ones, is another matter, the platforms’ creators found. Although the platforms stress that reviews are anonymous and the reviewers’ identities are kept confidential, students and postdocs worry that negative reviews could be linked to them and fear the repercussions that may result, Khong says. The platforms’ founders have taken different approaches to make trainees feel comfortable being honest about their time in their labs. The Scismic team hosts events and goes to graduate student and postdoc associations to meet potential users and build trust face-to-face. GradPI is designed to be accessible only to non-PIs, including grad students, postdocs, and research assistants. Higgs has measures in place to block PIs from viewing the website.

A review website seems to be a harder sell to the PIs. A negative comment about the lab’s culture can come across as a critique of the PI’s personality. And a crowdsourced system opens the door for fake reviews.

But some PIs are onboard. Jagan Srinivasan, an assistant professor of biology at Worcester Polytechnic Institute in Massachusetts, endorsed Scismic when the platform’s founders were looking for support from members of the academic community. He agrees with the founders’ premise that an open forum of experiences would help students and postdocs make informed decisions. Thinking back to his postdoc days and seeing his friends struggle, “I wish this was there for some of my friends who had some really bad experiences,” he says.

Now, as a faculty member, he also thinks that reading reviews about his lab and advising style could offer helpful feedback about areas he can improve on as a mentor, he says. Negative reviews sting, he acknowledges, as many professors have experienced via student evaluations for the courses they teach. But, he continues, “every lab has certain sets of behavior that need changing or modifying.” And with Scismic and forums like it, “I [see] the opportunity for labs to get better.”

Disseminate the data

Khong learned from the missteps that led her to her first postdoc lab. Now wrapping up three and a half productive years as a postdoc in a second lab, “I asked my second PI all the questions I didn’t ask my first PI,” she says. She is looking forward to starting her new position as a research scientist at a biotech startup in April. But she plans to continue with Scismic as well. “Scismic was almost like a stress outlet for me,” Khong says. “Doing it was very therapeutic. It made me feel like I was working towards a solution for a problem.”

The platform has become a business venture for the team. While the trio currently foots the bills, one co-founder is working on Scismic full time. The team plans to roll out a platform tailored to finding industry scientist positions in the near future as a revenue stream. But right now, the priority is to collect reviews, Khong says. Higgs is also focused on increasing the number of reviews on his website. He was the sole force behind GradPI when it started, but a friend has since joined to help him with marketing and writing grants to grow the website, Higgs says. Like the Scismic team, Higgs pays for the website’s costs himself. He’s considered allowing advertising to offset the costs. But making the website profitable isn’t his goal, he says. “I see it being able to help grad students make a wise decision,” he says of the platform. Higgs is committed to expanding GradPI as he continues through his Ph.D. program. “I know it would have helped me,” he says.

Like Khong, Higgs has firsthand experience with a weak mentor-mentee match. After first starting a Ph.D. program in 2011, his adviser left his institution and he picked a new PI based on funding availability. But the adviser did not provide the support Higgs needed to pass his program’s qualifying exam, he says. Higgs failed the exam and had to withdraw from the program. As he reapplied to Ph.D. programs the following year, he thought carefully about what he wanted from a PI. He also sought information about the labs he was interested in, asking the PI’s mentees about their dynamics with their adviser and visiting his top-choice lab to feel out the environment. He joined the lab as a postgraduate associate and became a graduate researcher when the department accepted him into its Ph.D. program. Now a first-year student in Yale University’s molecular, cellular, and developmental biology program, he’s thriving because of the environment his current adviser creates, he says. He hopes his tool will help grad students “foster the sort of relationship with their PIs similar to the one I have now,” he says.

Follow Science Careers

Search Jobs

Enter keywords, locations or job types to start searching for your new science career.

Top articles in Careers