XR Firefighter Sim
Using the same setup and technology as the XR Hockey Sim, a similar simulation can be done
for firefighting. Using the headset to track the users movement and experience through the training, it is intended to help firefighters better
familiarize themselves with the search and rescue sweep pass techniques. Spacial audio and sounds are also utilized to indicate many things that would be heard during
a real training or search/rescue situation, such as: low air warning, PASS idle alarm, radio communication and codes, victim shouts and other environmental
sound effects.
The Quest controllers (or other VR set controllers) can be used standalone or in attachment with tools to change the experience. As shown above, a quest controller is connected to a fire axe which will help simulate carrying a tool with you during a search pass, and to help monitor sweeping motions to locate potential victims! Controllers can also be strapped to each hand or held in hand (a bit hard to hold controllers while crawling on hands...) to simulate traditional gloves, or hand tracking can be used without controllers to do the same thing. Although sometimes the gloves aren't properly tracked depending on which gloves are used. I have also tested using IMU chips to track other tools in addition to the controllers. While I have not connected external devices directly to the VR headset, IMU chips can be used standalone and if one of them is attached near the headset or controller it can be tracked relative to the other controllers. In the case of testing addition gear, I used 3 Adafruit Industries TDK InvenSense ICM-20948 9-DoF IMU placed along a fire hose, to track 3 different points. Using libraries and assets such as Uduino, the movement of these chips can be assessed to get a good enough reading to track a virtual hose in-game. Hoses are often utilized in fire rescue to track which way they are coming from or need to move to exit the building.
The Quest controllers (or other VR set controllers) can be used standalone or in attachment with tools to change the experience. As shown above, a quest controller is connected to a fire axe which will help simulate carrying a tool with you during a search pass, and to help monitor sweeping motions to locate potential victims! Controllers can also be strapped to each hand or held in hand (a bit hard to hold controllers while crawling on hands...) to simulate traditional gloves, or hand tracking can be used without controllers to do the same thing. Although sometimes the gloves aren't properly tracked depending on which gloves are used. I have also tested using IMU chips to track other tools in addition to the controllers. While I have not connected external devices directly to the VR headset, IMU chips can be used standalone and if one of them is attached near the headset or controller it can be tracked relative to the other controllers. In the case of testing addition gear, I used 3 Adafruit Industries TDK InvenSense ICM-20948 9-DoF IMU placed along a fire hose, to track 3 different points. Using libraries and assets such as Uduino, the movement of these chips can be assessed to get a good enough reading to track a virtual hose in-game. Hoses are often utilized in fire rescue to track which way they are coming from or need to move to exit the building.
XR FF Helmet HUD
(work in progress)
After showing my initial training VR prototype to some members of my fire company, they were interested in it. My chief was curious if the same technology
could be used in a real-world situation as well. Training helps but can only go so far, and sometimes you can't train for events that we face so new technology
to make real calls safer is always sought after. Obvious a VR headset would not be practical, and most firefighters don't want to change helmets, even if they are safer...
However, what if there was a way to attach a screen or projector onto current helmets to show this HUD (heads up display) right on the visor. Similar to the AR glasses by Mira, a screen could reflect information onto the lenses the firefighter looks through, which are already included on most helmets!
/cdn.vox-cdn.com/uploads/chorus_asset/file/8833235/jbareham_170710_1837_0024.jpg "Mira AR Headset")
Whether this screen would be attached inside or outside the helmet, I am not sure yet. Obviously this will come down to safety, another possibility that has worked as well is
to have a screen clip on to an upper pocket on the jacket. Although this is less reliable and can be blocked much more easily. These screens could be turned on/off with the click
of a button, and display valuable information to the firefighter. Some information that could be displayed on either visor from the screen could be:
Current temperature information
How long they have been inside
3D visual representation of structure
2D map overlay of floorplans
Indicators (2D and 3D) of other FFs
Indicators pointing to the exit or hydrants
Remaining oxygen levels
(already indicated on o2 masks but only as 4 lights, not specific time)
As well as the possible ability to communicate further information, such as displaying radio dialog or marking a location in 3D for others to see. (Fire source, victim, tool, exit, etc...)
These features would not impend with the FF activities but would provide an additional level of information without requiring any training.
However, what if there was a way to attach a screen or projector onto current helmets to show this HUD (heads up display) right on the visor. Similar to the AR glasses by Mira, a screen could reflect information onto the lenses the firefighter looks through, which are already included on most helmets!