Granting Researchers Success

The stakes for research grant applications are high in today’s competitive funding environment. Yet applications are often submitted to external funding agencies before they pass any kind of internal review process. A new study from Columbia University’s School of Nursing suggests that institutions benefit from helping researchers write better grants. Specifically, it found that pilot grant applications that underwent an internal review were twice as likely as nonreviewed applications to receive funding.

“Over a five-year time frame, our school’s intramural pilot grant program produced peer-reviewed publications, conference presentations and subsequent external grant funding that likely would not have otherwise been generated,” the study says. “Given the resources required to prepare grant applications, internal finding and reviews can enhance return on investment.”

The paper, now in press with Nursing Outlook, is based on outcome data on 14 intramural pilot grants and 88 external grant applications from 2012 to 2016. In all, researchers found that pilot grants produced 16 peer-reviewed articles, 33 presentations and 11 grants. Some 42 percent (20 out of 48) applications that saw any type of internal review received funding, compared to 20 percent of grants (eight out of 40) that were not reviewed internally prior to submission.

Columbia Nursing’s $127,000 investment in funding their review processes over five years led to $3 million in external funding. That doesn’t take into account the time and effort associated with such processes. However, the study says, “we believe it still represents a sound investment for the school and for launching the research careers of postdoctoral fellows and junior faculty.”

In 2012, to address what the study describes as a critical need for research infrastructure, Columbia Nursing launched an intramural pilot grant program open to postdoctoral fellows and faculty members, with a preference for funding junior professors and early-stage investigators. Three such grants are available each year, of up to $10,000 each; the idea is that they’ll fund preliminary data collection or scholarly preparation for bigger grant proposals. Pilot grant recipients are required to submit interim and final progress reports, make a formal presentation to faculty members and students on the results, and share their findings in a publishable manuscript, abstract or other conference presentation.

Other funding priorities include projects that enhance collaboration among nursing faculty members, particularly clinical and research scholars, and those that incorporate interdisciplinary and translational research, the paper notes.

At the same time, the school’s Office of Scholarship and Research Development began coordinating a two-part internal review process for grant applications: peer-to-peer meetings called Specific Objectives and Aims Review (SOAR), followed by a live mock review. The programs are optional but strongly encouraged for faculty members and mandatory for postdocs.

SOAR protocols are scheduled for two to three months prior to proposal submissions. “Because a SOAR is designed to be preparatory to writing the proposal, only the specific aims are distributed and discussed,” the study says. “The purpose of the SOAR is to assure, before development of the protocol, that the aims are clear, logical, important, well articulated, feasible and include a cogent and logical rationale.” Sessions are informal and dialogue based, taking approximately 30 minutes. Investigators are required to contact their peer reviewers via email following the meeting to share what changes they plan to make in their proposals.

As for mock reviews, the study says that even if an investigator “feels that the grant is as near ‘perfect’ as possible, fresh eyes can pick up ways to improve the flow, make the presentation more motivating or identify flaws not previously recognized.” Grant seekers submit their mock review requests several weeks in advance and identify a group of peer experts and nonexperts to participate. Ideally, investigators agree to let graduate students and other faculty members watch the mock review, which is moderated by a staff or faculty member.

Over the five years studied, 19 intramural pilot grant applications were submitted. Fourteen were funded, with five senior principal investigators, seven junior professors, one associate research scientist and one postdoctoral fellow (three were repeat recipients). Pilot grants averaged $9,100 per project. The grants provided data for 14 subsequent grant applications, 11 of which were funded externally (two are pending and one was not funded).

Of the 88 grant applications that underwent mock reviews, 72 were submitted to federal agencies and 16 were submitted to foundations. Seventy-five applications were submitted by faculty, 22 of which were funded. Six of the 13 applications submitted by postdocs or graduate students were funded. Twenty-seven underwent at least one type of internal review. Seven applications only had a SOAR review, two of which were funded. Twenty-three applications had only a mock review, of which 12 were funded. Eighteen applications had both a SOAR and a mock review, six of which were funded. Forty applications did not have a SOAR or a mock review, and only eight of these were funded. (Note: The previous sentence has been updated from an earlier version to correct the nature and outcome of these applications.)

For external grant applications that underwent any type of internal review, whether SOAR, mock review or both, compared with those that did not participate, 42 percent received funding as compared with 20 percent that did not participate.

‘Broadly Applicable’ Findings

Lead author Kristine M. Kulage, director of research and scholarly development in nursing at Columbia, said Tuesday that she thought the study’s findings were “broadly applicable” not just to other health-care fields but to those across the sciences. (Kulage co-wrote the study with Elaine L. Larson, associate dean for research at Columbia Nursing.)

“High-quality grant writing is essential for all scientific disciplines to secure research grants in this highly competitive funding environment,” Kulage said. “Grant applications must not only be scientifically significant, innovative and rigorous with a high potential to improve health outcomes, but must also be well written to ensure accuracy, clarity, organization and a logical flow of arguments.”

It’s “absolutely essential that sound infrastructure be in place to support scientific investigators in both their grant writing and research endeavors,” Kulage added. At Columbia Nursing, for example, even relatively small investments in critical infrastructure -- such as financial support for preliminary data collection for future grants -- make a difference.

Nathan L. Vanderford, an assistant professor of toxicology and cancer biology at the University of Kentucky and assistant director for research at the campus’s Markey Cancer Center, said it’s rare for researchers or professors to participate in internal vetting procedures when they lack a strong internal peer network or institutional support services.

“Some faculty within an institution work in a vacuum and they simply don’t connect with other faculty that could help them vet or pre-review grants and manuscripts,” he said. Possible reasons for not participating might be “pure stubbornness” or internal political issues, he added. Yet Vanderford said he’s seen researchers succeed when they tap in to an internal peer network or use institutional support services. The cancer center’s Research Communications Office, for example, offers help with grant writing. The cancer center also offers a “grant swap” mechanism, by which professors can have their grant applications reviewed by other professors with similar expertise, and an integrated pilot funding program. (The College of Medicine also has a National Institutes of Health primary grant consultation service.)

Vanderford said the pilot mechanisms have been successful: in the last five years, 19 funded projects have yielded 11 external grants totaling $6,583,106 -- a return of $13.86 per dollar invested -- and 12 peer-reviewed publications.

It’s “all about faculty engaging with their peers, being willing to obtain peer feedback and utilizing the services provided by an institution,” he said.