Isaac Newton described science as the process of “standing on the shoulders of giants”—building on earlier discoveries to attain higher levels of knowledge. Synthesis of existing scientific literature is among the ways that researchers climb atop those giants' shoulders. But that system is getting shaky as interdisciplinary work gets more complex and the volume of global scientific output expands.
The very notion of a comprehensive review of the scientific literature is growing untenable. The Web of Science Core Collection alone now indexes over 271 million publications from more than 34,000 journals, representing a 47% growth from 2016 to 2022. In interdisciplinary domains such as ours—hazards and disaster research—achieving true comprehensiveness may be impossible already. Relevant findings are dispersed across thousands of outlets representing dozens of disciplines. And each discipline uses its own concepts, theories, and methodological orientation.
These problems are illustrated by the ongoing work of the Third Assessment of Natural Hazards and Disaster Research. Led by the University of Colorado Boulder's Natural Hazards Center, one of the nation's preeminent disaster research institutes, this is a massive project to synthesize 25 years of global research on the technical, social, and policy aspects of natural hazards and disasters.
The very notion of a comprehensive review of the scientific literature is growing untenable.
Four structural challenges will confront the assessment team, or anyone who wants to make scientific contributions in an interdisciplinary field.
Accessing the past remains a barrier. The assessment targets research published since 2000. This is a practical boundary for policy relevance, but it risks excluding the research that forms the intellectual bedrock of the field. Older studies often elude major databases such as Web of Science or survive only in print. Similar limitations became evident during previous efforts to identify emergency management scholarship; then, even “comprehensive” reviews omitted key historical texts. The result is an incomplete lineage of evidence that weakens the cumulative, self-correcting nature of science.
Interdisciplinary language frustrates comprehensiveness. Disaster-related terms are used inconsistently across disciplines—and diverge from policy definitions. Some scholars use “reconstruction” similarly to how others use “recovery.” Others equate “mitigation” with “climate adaptation.” “Resilience” may signify preparedness, response, or recovery depending on the author's disciplinary lens. These inconsistencies complicate both database searches and automated text analyses.
Boolean searches to capture all relevant permutations require substantial time and expertise, and the person searching still may fail to include key terms. Advanced algorithms or AI can help, but even they struggle to overcome linguistic ambiguity. How great is this challenge? For unrelated emergency management research that we conducted in 2024, we initially discovered more than 1.4 million articles using Boolean searches of strings of emergency management terms. After training and using machine learning models, we eventually isolated 141,857 that were actually relevant to the field. In other words, nearly 90 percent of what the Boolean searches had yielded was noise. Sifting through it would have been impossible without AI. AI might be further leveraged to help improve identification of articles for interdisciplinary fields. But who would fund, build, maintain and host such a tool?
Obtaining the full text at scale can be a profound challenge. Bibliometric metadata provide titles, abstracts, and keywords but not the substantive content needed for synthesis. The Natural Hazards Center's Third Assessment team has identified bibliometric records for 82,658 publications (another 41,900 are still under review). Assuming seven minutes each, simply retrieving and downloading 82,658 papers would require approximately 9,600 hours, or 241 workweeks for one person. Tools such as EBSCO, ScienceDirect, and EndNote's Find Full Text function are designed to assist in document retrieval. Yet when we recently conducted another literature review of 1,370 publications, automated bulk retrieval via these tools achieved only 49 percent success. Time constraints and access issues made manual retrieval of the remaining texts infeasible. The Natural Hazards Center team is likely to confront similar problems. And yet any synthesis based solely on accessible materials will be incomplete and risks systematic bias.
Synthesis remains a formidable undertaking even when texts are accessible. At an average of 20 minutes per article, reviewing 82,658 publications would require more than a year of full-time effort from a team of ten people. That's just to ingest the information. Synthesis requires further interpretive judgment—identifying patterns, reconciling contradictions, and drawing meaningful conclusions across disciplines.
Again, AI tools offer potential assistance, but few meet the demanding criteria for rigorous literature synthesis so far. Popular AI models such as ChatGPT, Gemini, Claude, and Notebook LLM limit file volume and size. Academic tools like Scholarly, Perplexity, and SciSpace, have their own additional limitations. Effective AI tools would require bulk upload capacity, automated extraction, customizable analysis, cross-disciplinary comprehension, and transparent traceability of outputs. AI may automate and speed up components of the process, but it's not yet a substitute for the discernment of experts.
AI tools offer potential assistance, but few meet the demanding criteria for rigorous literature synthesis so far.
Indeed, the pursuit of interdisciplinary comprehensiveness is no longer merely a technical problem that can be solved; it is a structural condition of knowledge itself.
We cannot assume that AI will simply fix this epistemic crisis. We also need new knowledge infrastructure to collate research that is spread across epistemic communities, data systems, and languages. And we need all of this quickly proliferating research to be discoverable, interpretable, and actionable.
The research communities working on disasters and natural hazards could lead the way by:
- Disseminating guidance for scholars on designing titles, keywords, and abstracts for discovery and connection to the broader body of research
- Working with key publishers, like Elsevier and Taylor & Francis, and academic databases, like Web of Science, to develop solutions for usefully tagging hazards and disaster research for discovery and impact
- Developing and promulgating a research agenda for the field
- Organizing and disseminating regular state-of-science reports covering shorter time periods, e.g., five years
- Supporting the creation and maintenance of a specialized disasters and natural hazards research database that is supported by AI tools
Supported by such knowledge infrastructure, hazard and disaster research can be more cumulative and policy-relevant, even when it cannot be exhaustive. For our own and comparable interdisciplinary policy fields, the solution lies not just in AI, but also in developing systems that facilitate continuous, collective learning.
