Engineers at Harvard University have developed a smartphone navigation app designed to help visually impaired pedestrians steer around obstacles in real time. The software, named Mobilio, runs on standard consumer phones, using onboard cameras, motion sensors and auditory signals to cut indoor collisions by 41 per cent during clinical trials published this week.
Key points
- Researchers at Harvard created Mobilio, an app that uses everyday smartphone cameras and sensors to map paths and flag hazards for visually impaired pedestrians.
- In trials with 14 visually impaired users, participants reached destinations 13 per cent faster and experienced 41 per cent fewer indoor collisions compared to using Google Maps with a cane.
- The software uses computer vision trained on pedestrian sidewalk footage rather than vehicle road data, planning movement patterns similar to an autonomous vehicle.
- The system avoids the need for expensive specialised hardware by running computer vision models locally on standard smartphones.
How Mobilio tracks paths and hazards
Existing travel aids for people with visual impairments range from traditional white canes to specialised hardware such as sensor-packed mobility canes and wearable cameras. Many digital navigation tools rely on satellite mapping services designed for road transport. Standard mapping tools do not provide the granular street-level detail or hazard warnings required to safely guide someone walking along a crowded corridor or pavement.
Mobilio was developed by Raymond Liu, a doctoral student, and Patrick Slade, an assistant professor of bioengineering at the Harvard John A. Paulson School of Engineering and Applied Sciences. Details of the system appeared in a research paper published in Nature Biomedical Engineering.
Rather than requiring users to buy bespoke assistance devices, the software functions directly on standard mobile hardware. The app processes data from the phone’s optical camera, GPS receiver, inertial measurement units and LiDAR sensors where available. A local computer vision model inspects live video frames from the camera to detect oncoming hazards, mapping out an unobstructed route directly ahead of the user.
Directional prompts are delivered through customised audio beeps. The software times these cues to match the user’s specific walking cadence, warning them of turns, surface changes and physical obstructions without overwhelming them with spoken commentary.
Pedestrian data over driver maps
Before writing code, the team conducted a needs assessment involving more than 100 people who have visual impairments or blindness. Participants reported three consistent priorities: continuous direction along straight stretches, accurate turn-by-turn routing, and prompt alerts for physical hazards.
The engineering team addressed those needs by rethinking how the visual model interprets the surrounding world. Most commercial navigation tools use visual models trained on road environments recorded from cars. Mobilio uses semantic segmentation models trained on imagery captured from the eye level of people walking on pavements, plazas and pedestrian crossings.
Liu, whose interest in visual accessibility stems from his older brother being blind, stressed the importance of collecting relevant visual perspectives. As Liu explained: “There’s a need for more data that comes from the perspective of humans walking along sidewalks, or walking outdoors.”
By analysing live scenes from a pedestrian perspective, the software calculates safe walking paths using mathematical trajectory planning similar to algorithms found in self-driving cars. The app projects a dynamic path forward, continuously adjusting guidance as the person moves through changing environments.
Trial results and obstacle avoidance
The Harvard team tested Mobilio in controlled experiments with 14 visually impaired participants who navigated complex routes using their standard mobility cane alongside the app. The researchers compared these journeys against trials where participants walked the same paths using a cane paired with Google Maps.
Users guided by Mobilio reached their destinations 13 per cent faster than those relying on standard digital maps. The system also reduced indoor physical collisions with obstacles by 41 per cent.
User feedback during testing drove adjustments to the app’s sound design and responsiveness. Liu said direct testing with visually impaired individuals was essential to understand practical device use: “Having actual people test out your device is absolutely crucial, because it’s impossible to predict how somebody is going to react to your device.”
Hardware accessibility and field trials
Specialised accessibility hardware can cost thousands of pounds, creating steep barriers to adoption. A primary goal for the researchers was ensuring Mobilio operated on commercial smartphones without requiring external processing boxes or high-end desktop hardware.
Running real-time image segmentation and trajectory calculations on an ordinary mobile processor required streamlining the underlying computer vision architecture. By keeping processing on the device, the app maintains low latency between visual hazard detection and audio feedback, which is necessary when warning a pedestrian of immediate obstacles.
The Harvard SEAS team plans to expand trials to wider outdoor public spaces. The next phase of testing will track how the software adapts to unpredictable pedestrian foot traffic, weather conditions and the varied daily routines of its users.
