Marine Mass Casualty Events

Relevant Events and Links:

• 2016 Summer Institute on Marine Mass Casualty Events
• 2016 IEEE Safety, Security and Rescue Robotics Workshop on Unmanned Marine Vehicles for Marine Mass Casualty Events (see video of invited talks here)
• CRASAR YouTube channel with videos

Motivation and Overview

To date in 2016, over 4,500 migrants have drowned in the Mediterranean Sea this year attempting to reach Europe, Al Jeezera computes this as 14 deaths a day or one every 100 minutes. Since January 10, 2016, the Texas A&M Engineering Experiment Station's Center for Robot-Assisted Search and Rescue has been assisting the Hellenic (Greek) Coast Guard and Hellenic Red Cross in preventing refugee drownings with two Hydronalix EMILY unmanned marine vehicles. EMILY is a miniature teleoperated jet-ski covered with a flotation device sufficient to support five people. EMILY It is in operation by beach lifeguards in 10 countries to quickly reach people trapped in rip currents and to transfer a line to boats or kayaks trapped on rocky shores. To date, EMILY was used by Hellenic Coast Guard patrol boat 618 in February 2016 to pull refugees washed up on inaccessible rocks in high seas to safety. Patrol boat 618 could not get close enough to the trapped refugees without EMILY; the high seas were so extreme that one of the Coast Guard sailors fell overboard and the Captain was severely injured rescuing the sailor. The Hellenic Coast Guard, Red Cross, and volunteer life guard teams have used EMILY to escort dozens of boats carrying over 500 refugees to safe shores.

This ongoing project is adding autonomous capabilities and an aerial robot, Kitey, to amplify what lifeguards can do in marine mass casualty events. Lifeguards normally work with one to two other lifeguards plus an EMILY to save one or two people (3:1 ratio) but now are trying to save 50 people (1:16 ratio) at a time. Under funding from the National Science Foundation and private donations, we are implementing artificial intelligence capabilities. These capabilities are enabling the lifeguards to task EMILY to autonomously go to a group of refugees who are in a lower degree of distress, freeing the lifeguards to spend their time and attention on people who are at higher risk of drowning. Texas A&M students have programmed autonomous navigation capabilities and increased the lifeguard’s confidence in EMILY by adding a return to home upon loss of signal. Kitey, a Fotokite tethered UAV, provides situation awareness. Lifeguards can designate victims to assist on the UAV video feed and the UAV can direct EMILY to autonomously navigate to them. Kitey can be easily flown from a response boat and will automatically drift and move with the boat, requiring no attention. Moreover, its tether can provide unlimited power. Added LTE networking capabilities allowed incident commanders at a recent disaster exercise to see streaming video from EMILY’s and the Kitey’s cameras over 100 miles away.

Technical Objectives

The ultimate goal of this work in progress is to enable any number and combination of EMILYs and Kiteys to be used for all types of marine incidents, from a person falling overboard from a cruise ship to a mass event such as refugee boat, ferry, or cruise ships sinking. This technology has been offered to the Italian Coast Guard and Irish Navy for use with refugees coming from Africa, which has become the more popular but more dangerous route. We are actively working with the US Coast Guard, lifeguard agencies in Arizona, California, Oregon, and Texas, volunteer lifeguard organizations, and other US and European water agencies to identify and test new improvements.

The tandem combination of EMILY and Kitey is not just a theoretical idea, but a system that has and will continue to save lives. It is important to remember that the robots do not replace lifeguards but rather the intent is create tools that will help the lifeguards help others. EMILY and Kitey can be used independently or together, plus a single Kitey can be used with multiple EMILYs.

Further research, testing, and implementation is needed to help the robots reach their full potential as assistant lifeguards. The project was initially funded by donations from Hydronalix and private contributions through GoFundMe and is now partially funded by a National Science Foundation grant. The scientific challenges in creating robots that are easy for the lifeguards to use and aren’t scary to the victims in the water were much bigger than anyone suspected. We continue to uncover new problems such as human-robot interaction, coordinating multiple marine and aerial vehicles, and maintaining cybersecurity yet allowing ad hoc teams to work together.

The two biggest issues that we are addressing are control and human-robot interaction with the victims. The lifeguards have trouble teleoperating EMILY to a group of people because of lack of depth perception and interference from waves. As a result they may stop too far from the victim. Worse yet, from a human-robot interaction perspective is the lifeguard may drive too fast at the person or drive over the person; while a collision won’t injure the person, it adds more fear and stress to their situation. Therefore, we have programmed Kitey to control EMILY, using the UAV’s higher altitude viewpoint to make sure EMILY gets to the person. We are adding a thermal camera to EMILY and creating heat seeking algorithms that will allow EMILY to track and follow victims even while drifting in currents. EMILY is also being programmed to slow down as she approaches a victim and turn so that her long side of flotation and handles are directly facing the victim.

The Team

This project is conducted through the Roboticists Without Borders (RWB) program of the Center for Robot-Assisted Search and Rescue. The RWB program brings together industry, agencies and academia.

The project is under the direction of Dr. Robin Murphy and Dr. Walt Magnussen. Dr. Murphy is a professor of computer science and engineering at Texas A&M, and director of the Center for Robot-Assisted Search and Rescue. Dr. Magnussen is the director of the Internet2 Technology Evaluation Center and is leading the effort to add LTE wireless to this life saving system.

The two industry partners are Hydronalix and Fotokite. Hydronalix, under the direction of Tony Mulligan, has donated EMILY robots to the Hellenic Coast Guard, Red Cross, and Texas A&M and paid for multiple trips to Greece, Italy, and Houston to test new developments. In addition, two water rescue experts, Fernando Boiteux and John Simms, have donated their time to join us in Greece and other exercises. Sergei Lupashin and Christopher McCall at Fotokite donated parts and labor, plus always placed a high priority on the work despite them rolling out a new product.

Over 40 undergraduate students and 5 graduate students at Texas A&M have worked on EMILY, Kitey, or some aspect of the refugee problem (such as natural language smartphone apps to let lifeguards speak to the refugees in any language) as part of their courses in sensor capstone design and robotics. The incorporation of real-world needs is part of the service-based learning philosophy at Texas A&M. The students who have been the most directly involved in the fielding of EMILY and Kitey are:

• Tim Woodbury (PhD student Aerospace Engineering) and Xuesu Xiao (PhD student Computer Science and Engineering), in charge of autonomous navigation and onboard control
• Jan Dufek (PhD student Computer Science and Engineering),, in charge of the UAV and computer vision to control EMILY. Jan’s work won the best field research paper at the 2016 IEEE Symposium on Safety, Security and Rescue Robotics in Lausanne, Switzerland.
• Grant Wilde (PhD student Computer Science and Engineering), in charge of experimentation and evaluation
• Jesus Orozco (MS student Aerospace Engineering), and Rebecca Schofield (undergraduate student, honors thesis Computer Science and Engineering), in charge of developing onboard thermal sensing to track people in the water.

In addition to Texas A&M, the Technical University of Athens, University of Zagreb, University of Genoa, and University of Cassino are involved in the project, with interest from the Technical University of Delft and University of Limmerick.