The Air Force Transformational Capabilities Office aims to foster capabilities across initiatives. To show how human-centered, data-enhanced decision processes can be used to determine which concepts to advance into the capability pipeline, the authors used a multimethod qualitative approach and developed a data science tool to extract information from free-text descriptions. They demonstrate the tool and foresight methods in three case studies.

Data-Enabled Approaches for Enhancing the Air Force Transformational Capability Pipeline

Matthew Walsh, Lance Menthe, Jonathan L. Brosmer, Julie George, Christine Kistler LaCoste, Eric Hastings, Sherrill Lingel, Éder M. Sousa

ResearchPublished Mar 23, 2023

Cover: Data-Enabled Approaches for Enhancing the Air Force Transformational Capability Pipeline
Download PDF
Additional Downloads

Includes Research Summary

A key goal for the U.S. Air Force's Transformational Capabilities Office (TCO) is fostering transformational capabilities across a variety of initiatives. To propose, develop, and select which concepts to advance into the transformational capability pipeline, the TCO must extract information from many data sources. Machine learning and natural language processing can be used to extract information from text sources; however, subject matter expertise must also be applied and leveraged effectively to provide creative insight and make the best use of extracted information.

To understand how human-centered, data-enhanced (HCDE) decision processes can be used to determine which concepts to advance into the pipeline, the authors used a multimethod qualitative approach that included a review of the relevant literature on development planning and interviews with senior leaders, technical experts, and subject matter experts from the Air Force and the defense community. The synthesis of their analysis revealed opportunities for the TCO to use data science tools to extract information from vast databases of capability gaps, capability needs, and technology solutions and to use a more diverse set of future-focused decision methods — called foresight methods — to leverage human expertise and creativity. They developed and implemented the proof-of-concept Semantic Clustering Analysis and Thematic Exploration Tool to extract information from free-text descriptions of capability gaps and technologies and combined data extraction with foresight methods as part of an HCDE decision process. The authors demonstrate the data science tool and foresight methods in three case studies.

Key Findings

  • The TCO's exceptionally broad mandate calls for tools and methods different from those used by other Department of the Air Force (DAF) and Department of Defense organizations.
  • Some data sources for capability gaps are widely referenced, but they are not centrally managed; data sources for science and technology solutions are far more varied and diverse, and the volume of data contained across these sources is vast.
  • No software tools are systematically used to parse, extract, and summarize the content of capability gap and technology solution sources.
  • Modern data science techniques can be used to extract information from free-text descriptions contained in these sources.
  • Development planning is a human-centered endeavor that depends on domain knowledge, creativity, and social networks.
  • Foresight methods can be used to leverage human expertise and creativity.
  • Data science techniques and foresight methods can be integrated to form an HCDE decision process.

Recommendations

  • The Air Force Research Laboratory (AFRL) and the TCO should use the concept development and selection process described in this report.
  • The AFRL and the TCO should use a software tool, like the one described in this report, to extract information from natural language data sources. As they do so, they should conduct user testing and validation studies to improve the software tool.
  • The AFRL should explore alternate natural language processing methods to maximize the utility of information extracted from free-text sources.
  • The DAF should curate and standardize key operational capability gap data sources.
  • The AFRL, the DAF, and the TCO should enrich key science and technology data sources by purchasing or developing capabilities to cleanse records and merge them with metadata.
  • The TCO should expand the use of creative, interactive, expert-driven, and evidence-based foresight methods.
  • As a stepping stone to reach full curation and standardization of HCDE capability development planning, the AFRL and the TCO should record human-generated technology pairings for capability gaps.

Topics

Document Details

Citation

Chicago Manual of Style

Walsh, Matthew, Lance Menthe, Jonathan L. Brosmer, Julie George, Christine Kistler LaCoste, Eric Hastings, Sherrill Lingel, and Éder M. Sousa, Data-Enabled Approaches for Enhancing the Air Force Transformational Capability Pipeline. Santa Monica, CA: RAND Corporation, 2023. https://www.rand.org/pubs/research_reports/RRA952-1.html.
BibTeX RIS

Research conducted by

The research reported here was commissioned by the AFRL's TCO (AFRL/RS) and conducted by the Force Modernization and Employment Program within RAND Project AIR FORCE.

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.