Abstract

Detailed feedback on draft research proposals can be among the most valuable pre-award support RMAs can offer to researchers. However, it requires skills that may take years to develop, and is time-consuming even with the right skills. Moreover, requests for pre-award support are often concentrated in short periods of time before major funder deadlines. Therefore, many research support offices do not give feedback on proposals, or provide it only for select researchers. To help solve these problems, we trained an AI application to critically evaluate fundamental research proposals and to suggest improvements. We fine-tuned an AI chatbot using many example research proposals from a major European funder and their evaluations by panels of experts, from across scientific disciplines. We also provided the bot with commonly agreed upon guidelines for scientific writing. In parallel, we engineered elaborate static and dynamic prompts based on our own experience with for instance common mistakes. The resulting chatbot provides a detailed evaluation of a research proposal, as well as suggestions to improve structure, content, and language where necessary. Based on its own suggestions, it may rewrite individual sentences or restructure and rewrite whole sections of a proposal. We tested the application on research proposals that we ourselves provided feedback on, and found that within minutes, it provided criticism and suggestions that largely coincided with our own. The most important difference is probably that we as human beings sometimes questioned content (e.g. claims that a method is novel, or that a problem has been overlooked), while the AI assumes that proposals are factually correct. Our AI tool allows RMAs with limited time and/or experience to provide quality feedback on draft research proposals to many researchers.