Fully autonomous systems and fallback to human control

When you design your fully autonomous systems it is important to have a good plan for how the system will fallback to human control when needed. See what happened to this driverless Waymo.

Photo from Wikimedia

Google Maps Leads About 100 Drivers Into A 'Muddy Mess' In Colorado

Many of us use car navigation systems these days, and we developed some trust working with these systems over time.
Here is what can happen when we have too much trust - Google Maps Leads About 100 Drivers Into A 'Muddy Mess' In Colorado.
Now imagine how this type of incident would develop in the world of automation in healthcare.

Learning from Autopilot Problems

What can the healthcare industry learn from the risks introduced by the rise of increasingly automated airplanes?

Automation is identified as one of the top 10 health technology hazards for the first time

The ECRI Institute have published its Top 10 Health Technology Hazards for 2017, and Automation is identified as one of the top 10 technology hazards for the first time.

What Can Happen When Operators Rely Too Much on Automation?

I already referred to the danger in relying on automation. The Asiana Airlines Boeing 777 passenger jet, that crashed while landing at San Francisco International Airport on July 2013, is a good example for what can go wrong when operators rely on automation.

In a June 24 press release said NTSB acting chairman Christopher A. Hart: "In this accident, the flight crew over-relied on automated systems without fully understanding how they interacted. Automation has made aviation safer. But even in highly automated aircraft, the human must be the boss.".

This lesson must be learned before we start implementing automation in healthcare.

Switching to manual mode

When the air traffic controllers' system in southwestern United States went out on April 30, 2014, the operators at the regional center had to switched to a back-up system so they could see the planes on their screens. "Paper slips and telephones were used to relay information about planes to other control centers."

This is a good example for the human operator role in complex systems - improvising and taking control when the technology part of the system fail. This also demonstrate how important it is to maintain the operators' skills to run the system without the technology that support them during normal operation.

This lesson should be learned when we implement new technologies and automation in healthcare.

(image source: http://en.wikipedia.org/wiki/File:Air_traffic_controller_schiphol_tower.jpg)

Do We Rely Too Much on Automation?

There are many things we can learn from the experience of others.
This time, what pilots think about automation.

 Since aviation started to implement automation some time ago, it is interesting to see what are the current challenges they are experiencing with commercial airline automation.

"Relying too heavily on ... automation ... pose the biggest threats to airliner safety world-wide"
 (The Wall Street Journal).