Preventing sexual assault, sexual harassment, suicide, and domestic violence among service members is essential to maintain a ready force. This report describes the development, pilot test, and validity testing of new prevention capability metrics for use by the Department of Defense to improve prevention. These metrics, as part of an integrated process of risk identification and assessment, will help identify gaps in prevention capability.

Novel Methods to Assess the Military's Evolving Prevention Capabilities

Development and Pilot Test of the On-Site Installation Evaluation Process

Joie D. Acosta, Matthew Chinman, Samer Atshan, Jack Baker, Charles Barton, Armenda Bialas, Tara Laila Blagg, Mallika Bhandarkar, Ingrid Estrada-Darley, Paul Flaspohler, et al.

ResearchPublished Jun 28, 2023

Service members experience a variety of harmful behaviors, such as sexual assault, sexual harassment, domestic abuse, and suicide, that can affect their overall health and quality of life and be detrimental to force readiness. While response and treatment are vital services to address these harms, a robust prevention system is also needed.

The Department of Defense (DoD) asked RAND Corporation researchers to develop a prevention capabilities assessment process for use across the active and reserve forces and to pilot the process during visits at 20 installations that were at higher risk for, or had a greater number of protective factors to mitigate, these harmful behaviors. This report describes how the prevention capabilities assessment process was developed, pilot tested, and assessed for validity.

The metrics assessed prevention capabilities in nine areas covering the extent to which military installations prioritized, were prepared to promote, or are currently promoting integrated primary prevention and efforts to create healthy and protective environments and are engaging with service members around these efforts. In pilot tests, Office of the Secretary of Defense (OSD) leaders, researchers, and some installation staff found the metrics beneficial in terms of capturing and providing useful and nuanced information about prevention practices and areas for growth, and the analyses generally supported the validity of the metrics.

Moving forward, DoD should continue to validate and refine these metrics as prevention efforts evolve and grow. These metrics, as part of an integrated process of risk identification and assessment, will help DoD track its prevention capability.

Key Findings

  • DoD compliance assessments did not capture the extent to which military installations prioritized, were prepared to promote, or are currently promoting integrated primary prevention and efforts to create healthy and protective environments or engaging with service members around these efforts.
  • RAND developed prevention capability metrics in nine key areas to address these gaps.
  • Researchers, OSD leads, and some site installation staff reported finding that the prevention capability assessment visits provided useful and nuanced information about prevention practices and areas for growth.
  • Analyses generally supported the validity of the RAND military prevention capability metrics. Protective percentiles were significantly positively correlated with capability ratings in three dimensions, and risk percentile scores were negatively correlated with capability ratings in four dimensions.
  • External research partners conducted respectful and sensitive focus groups and interviews and facilitated meaningful discussions that bolstered trust and provided opportunities for candid, confidential feedback from site personnel.
  • On-site data collection — particularly the focus groups and the tabletop exercise — generated important insights into prevention-focused activities and the climate at sites. In particular, the engagement of enlisted personnel and officers in focus groups provided crucial information.
  • Data collection was adaptive to differences in the availability of site personnel and researchers, creating environments for candid feedback while protecting respondent confidentiality.
  • OSD leads and installation staff viewed the visits as a chance to review their practices and promote change at the installations.

Recommendations

  • DoD should continue using the refined military prevention capabilities assessment through in-person, on-site installation visits. The military prevention capabilities assessment provided useful information to both the installation and DoD that could be used to inform future prevention plans and support. Applying these metrics as part of an integrated process of risk identification and assessment will help DoD proactively identify gaps in prevention capability.
  • Qualified and appropriate individuals are needed to facilitate the discussions and the tabletop exercises that are part of the capabilities assessment. DoD will need to carefully choose future site visit staff who have skills in qualitative research, experience with military contexts, and prevention expertise and who are perceived as coming from outside the military to encourage candor during the assessment.
  • Future site visit staff should be provided with structured and regular training on how to use the prevention capabilities metrics and how to adjust the assessment protocols to avoid redundancy and provide opportunities for flexibility.
  • To optimize the assessment while on-site, site visit staff should begin the planning process as early as possible, improve familiarity with each installation's existing prevention structures and established communications processes prior to visits, make accommodations for notetakers to lessen the physical burden (e.g., share notetaking duties, use speech recognition software), and strategically schedule on-site and supplemental data collection to contact difficult-to-reach target respondents.
  • Moving forward, DoD should continue to validate and refine these metrics as prevention efforts evolve and grow.

Topics

Document Details

Citation

Chicago Manual of Style

Acosta, Joie D., Matthew Chinman, Samer Atshan, Jack Baker, Charles Barton, Armenda Bialas, Tara Laila Blagg, Mallika Bhandarkar, Ingrid Estrada-Darley, Paul Flaspohler, Belinda Hernandez, Stephanie Brooks Holliday, Pamela Imm, Christine Kistler LaCoste, Sarita D. Lee, Sarah O. Meadows, Laura L. Miller, Ellen M. Pint, Keller Scholl, Shoshana R. Shelton, Ivy Todd, Abraham Wandersman, Amber Watson, N. Peter Whitehead, Deborah Zajdman, and Katie Zenger, Novel Methods to Assess the Military's Evolving Prevention Capabilities: Development and Pilot Test of the On-Site Installation Evaluation Process. Santa Monica, CA: RAND Corporation, 2023. https://www.rand.org/pubs/research_reports/RRA1353-1.html.
BibTeX RIS

Research conducted by

This research was sponsored by the Office of Force Resiliency (OFR), within the Office of the Secretary of Defense, and conducted within the Personnel, Readiness, and Health Program of the RAND National Security Research Division (NSRD).

This publication is part of the RAND research report series. Research reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND research reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

RAND is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors.