Practice as You Play: Using Homeland Security Exercise and Evaluation Program (HSEEP) Exercises to Evaluate a Storm Decision Support Tool

Abstract

We discuss the novel application of homeland security functional exercises to evaluate emergency managers’ use of simulation-based decision support tools for response to major coastal storms (e.g., hurricanes, tropical storms, nor’easters), such as Hurricane Katrina. The U.S. Department of Homeland Security’s “Homeland Security Exercise and Evaluation Program” (HSEEP) is the universally accepted standard for emergency management exercises in the United States and provides a templated format familiar to emergency management practitioners. Using HSEEP in a scenario-based approach to end-user evaluation provides a plausible, realistic environment that encourages open and honest player feedback. We present the results of two HSEEP-based functional exercises run as part of implementation research with the Coastal Hazards Analysis, Modeling, and Prediction (CHAMP) system. CHAMP combines high-resolution storm models with a database of critical infrastructure vulnerabilities to predict storm consequences and aid decision-making. Findings suggest that the HSEEP exercise format provides a ready-made process for evaluating emergency management tools in a format comfortable and familiar to participants. User feedback and observation data collected during the CHAMP workshops and exercises are used to inform real-world activation protocols and to guide ongoing development of the CHAMP system.