Training Exercises Fall Short?

Government Executive published a long article that examines the means by which governments at all levels train themselves for disasters. The article finds the training lacking:

"Exercises are not all created equal," says Michael Wermuth, director of homeland security programs at the nonprofit RAND Corp. "There are a lot of different kinds of exercises, a lot of different methodologies used to conduct exercises. There are exercises that sometimes seem to be destined to ensure success or at least a successful outcome in the exercise."

All but the first TOPOFF have occurred with advance warning. Emergency responders learn more valuable information for free from real-life false alarms - for example, a nerve gas scare in the Senate in February 2006 or a student pilot flying within three miles of the White House in May 2005 - critics say.

Problems also can be excluded or hidden if participants are less than honest. ... That was a problem during [the] Hurricane Pam [simulation, prior to Hurricane Katrina]. FEMA turned out to be unable to deliver on many of the promises it made during the simulation. "It happened a lot - the conversation would stop over something like generators or ice, and a FEMA guy would say, 'Look, don't worry about that, we've got contracts in place, you'll get your million gallons of water a day or whatever,' " recounted one Pam participant in Disaster: Hurricane Katrina and the Failure of Homeland Security by Christopher Cooper and Robert Block (Times Books, 2006).

"All these exercises don't mean anything unless there is some type of after-action report, [but] some people in some agencies see the exercise as the end in itself rather than a means to an end," says Carl Osaki, a clinical associate professor of environmental and occupational health at the University of Washington, who has designed several simulations. "A lot of times the findings of the after-action reports require additional training or policy. Sometimes [producing the reports is] time-consuming, or they're costly. So once they hit some of those barriers, the after-action report is sometimes seen as an academic exercise." That appears to have been a large problem with Hurricane Pam. Though many envisioned the tabletop simulation as the beginning of a conversation, according to Cooper and Block, FEMA canceled much of the follow-up work - including answering questions about moving emergency evacuees from short-term housing at the Superdome - for lack of funds.

Despite the notorious response failures during Hurricane Katrina, Hurricane Pam had some success. According to Cooper and Block, contraflow schemes - reversing one side of a divided highway so all lanes lead out of the city - historically had worked poorly in Louisiana, but evacuations improved markedly after Pam.

And others say that in addition to raising general awareness of potential threats, these exercises are successful because enduring a simulated crisis is a more effective way of learning than reading a lengthy evaluation or report. The most important benefit, they say, is the opportunity for first responders and public health workers to meet and get to know those with whom they would be working during a crisis. "Ninety percent of what you learn in an exercise you learn before the day of the exercise," says Fosher. "Sitting down with sister agencies and going through each other's plans, expectations and assumptions: 'I thought you were bringing the N95 [gas] masks.' 'No, it says you bring the N95 masks.' "

Two thoughts on this: First, the lack of follow-up after a simulation is an all-too-real occurrence, in all fields of endeavor. This is what happens when you use a training exercise as an opportunity to check off an item on a "to-do" list rather than as an opportunity to really improve what you're doing. Second, it makes sense to me that a significant benefit of these exercises is the simple opportunity for responders to get to know one another and work with one another. Creating collaborative relationships is not something you can do easily. An expensive and time-consuming training exercise may sometimes be necessary.