Problem:

Emergency workers routinely enter hazardous and compromised environments without hesitation to save lives. However, as tragedies such as Grenfell demonstrated, the men and women working in these enclosed spaces need better situational awareness. Agencies that provide humanitarian disaster relief and emergency services and, most importantly, those in Fire and Rescue, require new tools and new thinking to improve operational safety inside high-risk buildings.

The U.K. Government Digital Service (GDS), the department of the U.K. Cabinet Office tasked with the digital transformation of the British Government, asked for proposals from European innovation and defence firms to provide tools using Augmented Reality that improved situational awareness for U.K. fire brigades. The competition was a two-step bid, with an initial £50,000 offered to 5 finalists to showcase a minimal level of research, followed by an award grant of £500,000 and a contract to supply the resulting product to the U.K. government. Established players like Airbus Defense and SDS Defense, which already had contracts with the government, were expected to win.

Challenge:

Brandwidth, as a clear outsider to this challenge, trusted that our engineering and development teams could be competitive against the favorites across the Continent. We relied on our expertise in product development and user experience research to stand out over the other proposals. From the start, we felt that there were three components to this challenge:

1. A volumetric tracker that would record an emergency worker's movement in the X-, Y-, and Z-axis.
2. A radio communication system that would relay the volumetric location data from inside the building to a local server (a laptop or Toughbook), located in the command structure outside.
3. An Augmented Reality front-end application, developed primarily for a handheld tablet as requested by the government team.

Each of these components had several challenges that were considered unknown at the point of proposal submission.

Volumetric Tracker Challenges:

Many engineers have attempted to build military or civilian versions of an accurate volumetric tracker for tracking inside a compromised building. Tracking in volumetric space is possible using a variety of sensors, including camera-based systems and magnetic resonance. Each tracker type has its pros and cons, mostly related to the type, size, and processing requirement of data collected.

My innovation team focused primarily on using an array of Inertial Movement Unit (I.M.U.) sensors to track X, Y, and Z-axis movement without the need for cameras, expensive processors, heavy batteries or large data pipeline requirements to the local server. These I.M.U. trackers, however, have a trade-off in terms of tracking accuracy. They have a known difficulty with positional drift, where a small error in tracking becomes multiplied over distance and time. Over time, this error leads to a large deviation between the reported position of the tracker and the actual tracked object. We needed to create a volumetric tracking solution that utilized the small data requirements of IMUs, without the problematic drift.

Radio & Network Communication Challenges:

Competition requirements limited us from using 3G/4G/ or 5G networks, as there could be no guarantee that these would be operable in any location across the United Kingdom. Because of our I.M.U.-based trackers, we were lucky to have a low data rate, which would have been impossible if we had chosen a visual tracking system. Thus, we were able to consider using extreme long-distance radio frameworks, such as LoRA or Zigbee. Our requirements stated that the radio networked must work through any construction material from a distance of up to 400 meters.

Server & Augmented Reality Front-End Challenges:

We had to redefine what Augmented Reality is for life-threatening environments. Information overload, in this instance, could cost lives. Equally, this A.R. front-end needed to utilize graphics and UI elements familiar to those working in Fire and Rescue.

We felt that by leaning on elements of Skeuomorphic Design and Natural User Interfaces, we felt we could create an intuitive information overlay for the rescue commander. A project requirement was for the platform to work without existing CAD data for the buildings, and for an interior map to be built in real-time during emergency worker deployment.

Development:

Our R&D phase uncovered that complex and overly engineered solutions yield multiple failure points that only introduce risk to the environment. Solutions that require too much additional training or are prohibitively expensive will limit distribution. We needed to create a best-in-class platform that was affordable, simple, and easy-to-use for the emergency worker inside the building, while accurately and dynamically improving the situational awareness for the emergency commander outside the building.

Sensor Development:

Our prototype sensor was designed to be lightweight, low-impact, simple to use, with multiple redundancies on all critical parts and requiring no further training on the part of the emergency worker.

A major challenge remained in discovering a solution to IMU-based drift. Brandwidth Innovation relies on the multidisciplinary nature of our team to uncover solutions in areas normally not covered by traditional engineering or defence firms. The solution emerged in the user experience discovery that emergency workers never enter a compromised environment alone.

Each tracker on each emergency worker would include an array of IMU sensors. By connecting these sensors into a local mesh network of other emergency workers in the team, each node would be able to self-correct. We researched ways to average the data of one tracker against others in the network, as well as looking at time-of-flight numbers from nearby sensors. If one tracker reported that it was 20m away from the others, but the time-of-flight calculations reported that it was no more than 3m, it had very likely drifted and needed to be adjusted back into alignment.

Radio & Network Engineering Development:

In the R&D pilot prototype, we created an array of radio prototypes to test signal strength and transmission ability in a variety of challenging environments. By using LoRA radios, tuned for the emergency OFCOM frequencies, we were able to send data from our sensors through nearly every building material type tested, up to distances of nearly 400m.

Server & Augmented Reality Front-End Development:

We created the local server and front-end AR platform in consultation with our partners in the Fire & Rescue Service to ensure it reflects the user interface elements already familiar to the emergency response officer. In our Pilot MVP platform, our local server synced with a nearby tablet device over Bluetooth, sending pre-rendered data that overlays on top of the existing real-world view.

Result:

My team was honored to be selected not only as a semifinalist, beating competitors like Airbus Defense, but also as the sole organization to be awarded the full £500,000 grant (GDS offered the other £500k grant to a consortium of two other finalists). In close collaboration with our partners at the Mid & West Wales Fire & Rescue Service, we focused heavily on building a robust yet simple system. As with my previous product builds, I have brought on specialist consultants for specific tasks that are outside of Brandwidth's specialty, including a consultant network engineer, hardware radio engineer, electrical engineers, and mechanical engineers.

We are in the development stage of our engineering production prototypes, intending to conclude our government grant-funded portion of the project with a series of manufacturing samples. As the government grant explicitly precludes the manufacturing budget, we are looking at several avenues for producing our trackers and fulfilling our contract to supply the British government and UK Fire & Rescue service with our platform.