Smart glove for sign language recognition using AI

Project Reference :

AISG-GC-2019-002

Institution :

National University of Singapore (NUS)

Principal Investigator :

Professor Lee Chengkuo Vincent

Technology Readiness :

6 (Technology demonstrated in relevant environment)

Technology Categories :

AIoT

Background/Problem Statement

Amid the COVID-19 crisis, the global market for Disabled and Elderly Assistive Technology estimated at US$23.3 Billion in the year 2020, is projected to reach a revised size of US$32 Billion by 2026, of which, communication aids, is projected to record a 5.6% CAGR and reach US$12.8 Billion by the end of the analysis period.

Sign language recognition is of paramount importance for removing the communication barrier between speech/hearing impaired persons and normal individuals. Current sign language recognition systems rely on extremely limited features such as amplitude, bandwidth, and polarity to identify simple and discrete gestures for numbers or letters. This is far from the translation requirement of sentences that are frequently used in the daily life of the speech/hearing impaired. Besides, most reported works only achieve unidirectional delivery of recognition results without allowing two-way interaction between the speech/hearing-impaired user and a normal user.

Solution

A patent-pending, practical sign language recognition system using smart gloves that enables sentence identification based on a unique segmentation method and artificial intelligence (AI) and allows mutual communication based on a virtual reality (VR) interface.

Due to the novel segmentation method for sentence recognition, the signers are allowed to directly use their familiar sign language that is identified and translated into audio and text to start a conversation in the client end of VR interface. The non-signers respond to the signers by typing through the VR interface.

Benefits

  1. Unique segmentation method enables connection between basic word units and sentences.
  2. New sentence recognition feature enables rapid expansion of sentence database.
  3. High accuracy recognition using machine learning
  4. Immersive bidirectional interaction through VR interface.

Potential Application(s)

Sign language recognition system for facilitating two-way communication between speech/hearing impaired individuals and healthy individuals who do not understand sign language.

We welcome interest from the industry for collaboration/ co-development / customisation of the technology into a new product or service. If you have any enquiries or are keen to collaborate, please contact us.