Be My Eyes launches digital visual assistant powered by GPT-4

App allows volunteers to assist blind and visually-impaired users with online shopping and visual identification tasks

The Virtual Volunteer feature allows blind and visually impaired users to post tasks that a sighted volunteer can complete in their own time. Photo: Be My Eyes

Be My Eyes, an app that connects blind and visually-impaired people with sighted volunteers, provides users with remote help.

The digital visual assistant, called Be My Eyes Virtual Volunteer, is powered by the latest language model from OpenAI, GPT-4, and allows sighted volunteers to assist blind and visually impaired users with tasks that don't require physical presence.

These tasks could include things like online shopping, reading labels, and identifying objects in photos.

The tool, which launched on Monday, March 14, aligns with OpenAI's principles on responsible AI, and will provide transformative resources to improve navigation, independence and everyday needs.

Powered by OpenAI's GPT-4 language model, the tool allows users to send images for immediate visual assistance, offering a deeper level of understanding and conversational ability not yet seen in digital assistants, with the option to connect to a sighted volunteer.

Be My Eyes plans to expand the beta testing group for the Virtual Volunteer feature, and hopes to make it available to the blind and low-vision community.

Expansion to a powerful tool

Virtual Tool is an expansion of Be My Eyes' existing platform, which allows users to connect with sighted volunteers via a video call for assistance with tasks in real-time.

With the Virtual Volunteer feature, blind and visually impaired users can post tasks that require assistance and a sighted volunteer can complete them in their own time.

According to Hans Jorgen Wiberg, founder of Be My Eyes, the virtual volunteer feature was created in response to feedback from users who expressed a need for assistance with tasks that didn't require real-time interaction.

“We have been listening to our users and we heard the need for a platform where we can help more people with more tasks. The Virtual Volunteer feature will help us connect users with volunteers for tasks that can be done on their own time,” said Mr Wiberg.

The Virtual Volunteer feature is available to all users of the Be My Eyes app, which currently has more than four million volunteers and 300,000 visually impaired users.

Be My Eyes is a free app that was launched in 2015 and is available for both iOS and Android devices.

The app has received numerous awards and has been widely praised for its innovative approach to connecting blind and visually impaired people with sighted volunteers.

In addition to the Virtual Volunteer feature, Be My Eyes also recently partnered with Google to integrate Google Assistant into the app.

This integration will allow blind and visually impaired users to get assistance with tasks like checking the weather, getting direction, and making phone calls using voice commands.