Eighty percent of their annual budget (which is just under $4 billion) supports 20,000 individual projects. In 1997, 15% of federal resources that went to fund academic R&D came from the NSF. In comparison, 60% came from the National Institutes of Health (NIH), whose sprawling 300-acre campus lies only 10 miles north of its federal counterpart.
The two federal agencies process grant proposals differently: For example, specific activities of the NIH grant review protocol are handled by different personnel, whereas at the NSF, each program officer is responsible for all steps of the review--from assigning proposals to review panels to deciding which ones to fund. Applicants find out how well they've done roughly 6 months after submission.
An Officer and a Proposal
The majority of NSF grants personnel are full-time federal employees. There are also a few VSEEs (Visiting Scientist, Engineer, or Educator), formerly known as "rotators," who are expert scientists recruited from academia to serve 1 to 3 years as NSF program officers and directors.
P. C. Huang, a Johns Hopkins biochemist/molecular biologist in his third year as a rotating program director in Biological Sciences, is enthusiastic about bringing his academic experiences to the federal review system. "We used to sit on the other side of the table, but now we are given the opportunity to bridge the gap," he says.
He may spend an hour or so deciding which reviewers should evaluate the proposal and also which program to assign it to. That's around 8 to 10 proposals sorted per day by each officer. Not a bad turn around, but you have to keep in mind that some directorate deadlines bring in over 3000 proposals! Once they've all been assigned, candidates receive acknowledgement letters informing them where their proposals are going to be reviewed.
'Genetic Engineering'--Biological or Social Sciences?
Since many of today's research projects are multidisciplinary, it is tough to decide who should review a proposal, so "many proposals cross boundaries and therefore get reviewed by multiple panels," explains James L. Edwards, deputy assistant director of Biological Sciences.
There is certainly no shortage of review panels to choose from--NSF is divided into seven directorates that are in turn broken down into specific divisions, programs, and review panels. Each division has its own personnel--Integrative Biology and Neuroscience, for example, has 14 program officers who deal with grant proposals.
They have to be acutely aware of the politics and history of a particular subject area in addition to the actual science. Conflicts of interest between applicants and reviewers (i.e., collaborations or competitions) must be identified before proposals are distributed because these relationships may affect the review. To help out, Edwards recommends that candidates "specify individuals who they do and do not want to review their proposal." Such concerns are treated objectively and fairly before appropriate reviewers are eventually picked.
Days of Review
"Regular research grants receive a two-tier review," explains Huang--an evaluation by mail performed by ad hoc reviewers followed by the panel review session.
In applications that are reviewed by mail, reviewers submit their written comments by mail, facsimile, e-mail, or through FastLane--NSF's electronic submission and review system. These critiques carry a lot of weight because the program officer can decline or recommend an application for an award based solely upon these mail reviews.
The program officer also mails proposals to reviewers who send back their preliminary rating (such as from "excellent" to "poor") before they all convene for the panel review session--the "mail-plus-panel" review. Almost two-thirds of the grant proposals submitted to NSF during FY 1998 were reviewed in this way. A fifth or so were reviewed only by mail and less than two out of 10 submissions received "panel-only" evaluations without prior review mailings.
Picking 12 to 14 reviewers to sit on a panel can be quite a task--NSF has a database of 241,000 scientists and experts from whom they can pick to review grant proposals! Over 50,000 of them reviewed proposals in 1998.
In the Biological Sciences, a program officer makes sure each proposal ends up with two to four principal reviewers who head its discussion once they all meet.
When they do get together, an NSF panel in the Biological Sciences may discuss anywhere from 40 to 120 research proposals over 2 or 3 days. Its purpose is not to make definitive funding decisions but to make recommendations. "We come to a consensus," clarifies Edwards, "we do not vote [on a proposal] and the NSF does not rank proposals."
This is a fundamental difference between NIH's and NSF's selection methods--by the end of the NIH review, applications are ranked alongside other entries according to an overall numerical priority score. At NSF however, proposals are not given a numerical rating but are classified according to written "recommendations."
Fred Stollnitz, program director for Cross-Directorate Activities in the Division of Integrative Biology and Neuroscience at NSF explains further: "When panels review, [the reviewers] put each proposal into categories such as 'outstanding,' 'good and should be funded,' 'not ready in its present form,' or 'decline.' "
A particularly vocal reviewer could influence the final rating of the panel or where the proposal should be classified, but because there is no absolute score, only opinions are noted in the review analysis report--not actual decisions. An opinionated NIH reviewer on the other hand could affect the scores an application receives and so alter its ranking.
I Did It ... I Did It All
NSF panelists convey their opinions and recommendations in a "panel summary." They compose an overall analysis of review for each proposal that incorporate factors such as the panel summary, subject area, available resources, and the potential impact of the research. They then make final award decisions with the division director. Proposals that receive lower classifications by the panel can sometimes be funded over "higher-rated" research proposals because their overall assessment by the program officer is more favorable.
The budgetary consideration also plays a key role in the decision-making process. "The program officer doesn't just make 'yes' or 'no' decisions," explains Stollnitz. "They have to balance all those proposals that should be funded with the actual funds that are available." Sometimes a proposal classified as 'good and should be funded' submitted by an investigator with minimal existing funds may be given the edge over an 'outstanding' proposal submitted by an established and well-funded candidate.
Less (Paper)work, More Work?
To make the submission and administration of grant proposals and awards easier for both future grant-seekers and federal employees, the government is working on its Paperwork Elimination Act, placing more emphasis on using e-mail and the Internet to send and store information electronically.
During the 1980s, because of a 40% increase in the number of proposals submitted to NSF, an electronic network was designed to help process the extra load. In 1994 the electronic efforts resulted in the creation of Fastlane--NSF's electronic administration system. Candidates now also have instant access to information as soon as the assignment decisions have been made instead of having to wait weeks before finding out the status of their proposals.
This year, while gearing up to get the system into operation and publicizing the process to prospective applicants, the NSF predicted that only a tenth of submissions would be processed electronically. The system surpassed itself--over 40% of proposals were processed. Tom Quarles, deputy director of the division of Biological Infrastructure, is excited about NSF's technological achievements.
"We recently put out a request for pre-proposals and prepared the panels to give recommendations," he says. For new award initiatives, candidates can submit pre-proposals to explain their preliminary research plans to the NSF. If those plans are approved they would then be asked to submit a more detailed grant proposal.
"The panel discussed the pre-proposals for 3 days and by Thursday, one day after the panel ended, everybody had the results of their review!" he exclaims, describing how program officers used e-mail and other electronic resources to notify the applicants.
But despite the apparent ease of using new technology, postal and courier services will remain in business--for legal reasons, inked signatures are still required. But NSF is striving to stay ahead of the game and expedite the electronic system: In a handful of pilot experiments initiated by their Senior Management Integration Group, they are currently assessing the possibility of accepting electronically scanned "signature pages," thereby circumventing the need for hard copies. How well the process works will be assessed next year.
Congress may try to eliminate paperwork, but they will still have a hard time trying to eliminate human nature. "Procrastination is still the name of the game," says Quarles of applicants who wait until the eleventh hour to prepare and submit their proposals.
Carolyn Miller, Chief of the External Systems Branch at NSF recalls a recent July deadline which confirmed Quarles' observation--over a thousand applicants attempted to simultaneously "prepare, print [for their personal files], and submit their proposals within the last few hours of the deadline." The NSF server couldn't cope and the electronic delivery system froze up!
Click Here ...
Before spending months, weeks, or as is usually the case, a few frenetic days preparing an proposal, contact the appropriate NSF office or directorate and share your ideas with a program officer to gain valuable, time-saving advice.
Read the NSF Grant Proposal Guide
Find further information about FastLane