Walk from the department to the bakery with LOMOBI

Symbolic picture for the article. The link opens the image in a large view. — The LOMOBI system can be carried like a backpack. It is equipped with a camera on the front which feeds the system with data about the surroundings in real-time. Pic: FAU

FAU chair FAPS developing adaptive assist system for visually impaired people

August 17, 2023

The project “LOMOBI – Learning assistance system for orientation and mobility of visually impaired persons in everyday life” was launched this year at the Institute for Factory Automation and Production Systems (FAPS). Led by medical engineer Sina Martin, researchers at FAU will develop a system within the next two years that is able to navigate people over different surfaces and through unfamiliar situations. The final trial run should be approximately two kilometers long and guide test persons from the Institute to a bakery.

LOMOBI follows on from ESI 2.0, which FAPS conducted together with the Interdisciplinary Center for Embedded Systems (IZ ESI) from 2013 to 2014. The aim of ESI 2.0 was to allow visually impaired athletes to jog independently on gravel or forest paths using an assistance system. The researchers developed a rucksack equipped with an integrated navigation system. Feedback signals such as vibrations or sequences of sounds emitted by bone conduction headphones warn joggers of obstacles that are detected in real time using 3D sensors.

LOMOBI: Flexible, independent of GPS, predictive

Now the FAU researchers hope to expand the area of application of the older assistance system: LOMOBI should be able to work without GPS coverage, allowing users to move in buildings as well as outside. A further aim is to allow people to cope smoothly and at no risk to their safety with a wide range of different surfaces and transitions. It is currently not one of the main focal points of the project, but it is a valuable aspect for users: the system should learn typical human walking behavior to allow it to plan a way round people approaching from the opposite direction.

Under the leadership of Sina Martin, Matthias Kalenberg focuses predominantly on navigation, whilst his colleague Helmut Engelhardt is responsible for image processing.

Continual learning process

LOMOBI should continually improve and learn. It is important for the researchers that even unfamiliar situations do not pose a problem for the assistance system. By the time the project ends in 2025, they hope to establish a web platform that can be accessed both by LOMOBI users and those with no visual impairments in order to share images of surroundings. The individual systems will integrate these data and learn new paths. In addition, sighted people can correct data on the platform for surroundings that have proven difficult for people with visual impairments.

User-centered development

Whilst valuable insights were gained during the project ESI 2.0, the current research is centered on users’ experiences. To ensure that LOMOBI really does meet the needs of blind people and those with visual impairments, Matthias Kalenberg and Helmut Engelhardt worked in collaboration with the Center for Self-Determined Living with Disabilities (Zentrum für selbstbestimmtes Leben Behinderter e.V., ZSL Erlangen) to exchange ideas with its members during a workshop. Discussions focused on ethical, legal and social aspects. Would users be worried about stigmatization? What should the system’s design look like? How will data protection and the security of those using the system be taken into account?

At a later stage, there will be extensive studies involving LOMOBI being used in practice. The final test will involve walking from the FAPS Institute to a bakery roughly two kilometers away using the LOMOBI rucksack. The way there basically includes everything LOMOBI is expected to cope with: sections that are not covered by GPS, changing surfaces such as carpet, tiles, asphalt or gravel, obstacles such as steps and skirting boards and people approaching from the opposite direction.

The LOMOBI project is receiving funding of approximately 465,000 euros from the Federal Ministry of Education and Research (BMBF).

The LOMOBI system can be carried like a backpack. It is equipped with a camera on the front which feeds the system with data about the surroundings in real-time. Pic: FAU