CCC Workshop Report: The Future of Information Retrieval
Report, 2024 Computing Community Consortium, CRA Workshop
Citation: Allan, J., Choi, E., Lopresti, D., & Zamani, H. (2024). Future of Information Retrieval Research in the Age of Generative AI CCC Workshop Report. Washington, D.C.: Computing Research Association (CRA).
Forty-four experts in areas of information retrieval, artificial intelligence, and human-computer interaction came together to explore the use and impact of generative AI on information retrieval activities. The Computing Community Consortium (a sub-group of the Computing Research Association) sponsored this workshop, held in mid-2024. The result of the break-out sessions and discussions are provided in this report.
As explained in this version of the 40-page report, organizers of the July 2024 workshop "gathered immediately after the workshop, read the reports from each breakout session, and discussed the outline and potential content of the report. The first draft of the report was produced solely by the organizers using all the material produced by each breakout session (i.e., notes, summaries, comments, challenges, questions, recommendations) and extending or modifying them as needed. The draft of the report was shared with CCC and workshop participants for further feedback and it was revised to shape the report at hand. This report has been reviewed internally by a member of the CCC council and externally by 2 members of the IR/AI research community."
Extracted from the Executive Summary
In the fast-evolving field of information retrieval (IR), the integration of generative AI technologies such as large language models (LLMs) is transforming how users search for and interact with information. Recognizing this paradigm shift at the intersection of IR and generative AI (IR-GenAI), a visioning workshop supported by the Computing Community Consortium (CCC) was held in July 2024 to discuss the future of IR in the age of generative AI. This workshop convened 44 experts in information retrieval, natural language processing, human-computer interaction, and artificial intelligence from academia, industry, and government to explore how generative AI can enhance IR and vice versa, and to identify the major challenges and opportunities in this rapidly advancing field.
Research Directions: This report outlines eight research directions for IR-GenAI systems with high intellectual merits and broader impact:
- Evaluation challenges and needs in IR-GenAI;
- Learning from implicit and explicit human feedback for solving complex problems that may require reasoning;
- Understanding and modeling users for the evolving generative AI-powered information access systems;
- Challenges and potential solutions to address or mitigate socio-technical issues raised by the new technologies in IR-GenAI;
- Methods for developing personalized IR-GenAI systems;
- Efficiency considerations when scaling compute, data, and human eorts in developing IR-GenAI methods;
- The role of information retrieval in enhancing AI agents; and
- Developing foundation models specically for information access and discovery.
This report contains a summary of discussions as potentially important research topics and contains a list of recommendations for academics, industry practitioners, institutions, evaluation campaigns, and funding agencies.