Multimodal Data Integration for Real-Time Indoor Navigation Using a Smartphone

<p>We propose an integrated solution of indoor navigation using a smartphone, especially for assisting people with special needs, such as the blind and visually impaired (BVI) individuals. The system consists of three components: hybrid modeling, real-time navigation, and client-server architecture. In the<em> hybrid modeling</em> component, the hybrid model of a building is created region by region and is organized in a graph structure with nodes as destinations and landmarks, and edges as traversal paths between nodes. A Wi-Fi/cellular-data connectivity map, a beacon signal strength map, a 3D visual model (with destinations and landmarks annotated) are collected while a modeler walks through the building, and then registered with the floorplan of the building. The <em>client-server architecture</em> allows the scale-up of the system to a large area such as a college campus with multiple buildings, and the hybrid models are saved in the cloud and only downloaded when needed. In the <em>real-time navigation</em> component, a mobile app on the user’s smartphone will first download the beacon strength map and data connectivity map, and then use the beacon information to put the user in a region of a building. After the visual model of the region is downloaded to the user’s phone, the visual matching module will localize the user accurately in the region. A path planning algorithm takes the visual, connectivity and user preference information into account in planning a path for the user’s current location to the selected destination, and a scheduling algorithm is activated to download visual models of neighboring regions considering the connectivity information. Our current implementation uses ARKit on an iPhone to create local visual models and perform visual matching. User interfaces for both modeling and navigation are developed using visual, audio and haptic displays for our targeted users. Experimental results in real-time navigation are provided to validate our proposed approach.</p>

Name thesis:degree_name

Master of Science (M.S.)

Level thesis:degree_level

Thesis

Discipline thesis:degree_discipline

Computer Science

Year dc:date.available

2020

Author dc:creator

• Chang, Yaohua

Contributors dc:contributor

• Zhigang Zhu

• Indoor navigation
• blind and visually impaired
• hybrid modeling
• route planning algorithm
• task scheduling algorithm
• ARKit
• Other Computer Sciences
• Systems Architecture
• Theory and Algorithms

Repository record dc:identifier

https://academicworks.cuny.edu/cc_etds_theses/857

OAI identifier oai:identifier

oai:academicworks.cuny.edu:cc_etds_theses-1911