Decisions Blog

False Missile Alarms – A Workflow and Rule Problem?

In recent weeks, both the State of Hawaii and Japan have sent out mass alarms to their entire populations that have proven to be human error. In follow up news coverage from Business Insider on the Hawaii incident we’ve seen the user interface that supported the error (below, credit: Business Insider) and we’ve seen photographs of the workstation which happened to include a post it note of the system password (also below, credit: Business Insider).

While media coverage might raise the assumption that this is an uncommon event, anybody working in IT knows that it is all too common. In a progressively automated and web-enabled IT ecosystem, vendors are giving more and more capability to less sophisticated users. Many of these vendors are delivering that capability with a less than desirable user interface. In other words, you don’t have to know as much to hang yourself and the company, and the UI isn’t helping.

Everybody talking about these pictures agrees that the UI contributed heavily to this problem. A quick look suggests that a slip of the finger was all it took to send out a mass public warning! While I agree that the UI is a contributing factor – and we really care about the UI/UX here at Decisions – I wonder if there is another perspective that we could take here. I’d like to consider this as a Workflow and Business Rule problem.

As congress begins their inquiry into this event we will likely learn more about the role of the people involved in the actual button clicking and approvals that were required to run both the test and to initiate the mass alert… but for the sake of this conversation – let’s assume it was a junior professional who was running a routine test. What controls were present in this loose workflow to prevent this from happening? What business rules were governing the approval chain for these types of alerts? How are we balancing the speed to alert with the gathering of approvals?  

We’ve helped many of our customers solve a similar class of  problem. In fact, we’ve helped the largest business supply retailer in the world create a workflow with rules that let junior IT personnel create all of the configurations required to deploy new point of sale systems en masse – while controlling that actual impact they can have on production systems. With a technology stack that automates the provisioning and deployment of new POS systems, who are we going to trust to configure and push the “GO” button?

It was not only an automation project, but a user enablement project.

Senior IT professionals can easily become bottlenecks in development projects as they are asked to review and confirm many decisions from development through go live. More and more IT organizations are using Business Rule Engines to automate some of this decision making, while others are combining Business Rule Engines with Workflow Engines to simplify and enhance the review and approval process.

These Business Rule and Workflow enabled organizations are able to leverage junior resources more heavily by giving them access to key systems to do configuration and other work – but not letting these changes move forward until an approval has been granted – either by the Rule Engine or the human approval. We have a great eBook on this topic related to Systems Management at decisions.com/ebooks.

In other scenarios, like Credit Scoring in Financial Services, this capability yields a significant cost savings. Those familiar with the industry understand that web service calls to a credit bureau carry a hard cost. Using Business Rules and Workflow to create an as needed call given certain business conditions can be a quick source of improvement.

As a final aside, I learned yesterday that a major credit bureau using our technology saves $40M annually due to Business Rules and Workflows that help them recapture failed credit card transactions. It’s interesting that we are used on both sides of that equation. More to come on that story later. 

I’m sure we’ll learn more about the Hawaii and Japan incidents, and I’m sure there are competent professionals who are going to lose their job over the event. We’ll continue to work with folks to put Business Rules and Workflows in place that can prevent these types of events.