Abstract

As Artificial Intelligence (AI) continues to shape the research landscape, the potential for unconscious bias to influence decision-making has become a pressing concern, particularly in grant writing processes. With a view to developing a policy for Equality, Diversity, and Inclusion (EDI) for responsible research, the University of Galway is exploring the extent to which the use of AI compounds unconscious bias in grant application writing processes. The program will focus on raising awareness of EDI principles into AI-driven tools and systems, equipping researchers and research managers with the skills to identify and mitigate unconscious bias. This discussion will explore the intersection of EDI, AI, and research management, offering practical steps for implementing a sustainable and ethical policy within the research ecosystem. Participants of this discussion table will engage in a collaborative dialogue on how to design, develop, and deliver effective unconscious bias and the use of AI training tailored to researchers and research managers.

Methodology

The initiative will involve a targeted anonymous survey and interviews with researchers, research managers and EDI professionals and identify potential scenarios where unconscious bias in AI tools could affect applications. The interview guides will be developed in collaboration with AI experts, EDI practitioners, and research managers. The results will inform the design and delivery of training modules, which will focus on how AI algorithms are designed and how to audit them for biases. The ultimate aim is to develop a policy that will offer practical applications for identifying and addressing these issues in the grant-writing processes. The participants will be encouraged to contribute their insights, creating a collaborative learning environment where best practices can be shared and applied.

Possible questions for the discussion table.

1. Exploring challenges:
a. How can we tell if AI is causing unconscious bias in research proposal/grant writing?
b. What are the main challenges in making AI tools fair?
c. What do researchers and managers need to know to identify and address unconscious bias in AI?
d. Overcoming challenges - Training Needs: What do researchers and research managers need to know to identify and address unconscious bias in AI?
e. Policy design: what policy recommendations should be considered to ensure unconscious bias is considered and responsibly addressed in the use of AI.

2. Training and learning
a. How can we create and deliver training on unconscious bias that meets the needs of researchers and managers?
b. Are there good examples of successful bias training in research (benchmarking best practice)

Key learning outcomes of the discussion table

• Understand how AI impacts fairness in research
• Learn how to reduce bias in AI tools
• Generate practical ideas to make AI use more inclusive

References:

1. European Commission, 2022. Ethics Guidelines for Trustworthy AI. European Union Publications Office.
2. OECD, 2022. AI in the Research Lifecycle: Addressing Bias and Fairness in Scientific Funding. OECD Science, Technology, and Innovation Policy Papers.
3. West, S.M., Whittaker, M. & Crawford, K., 2023. Discriminating Systems: Gender, Race, and Power in AI. AI Now Institute. Available at: [https://ainowinstitute.org/publication/discriminating-systems-gender-race-and-power-in-ai-2 L] [Accessed 18 September 2024].