Dialogue enabling speech-to-text user assistive agent system for hearing-impaired person

Seongjae Lee, Sunmee Kang, David K. Han, Hanseok Ko

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

A novel approach for assisting bidirectional communication between people of normal hearing and hearing-impaired is presented. While the existing hearing-impaired assistive devices such as hearing aids and cochlear implants are vulnerable in extreme noise conditions or post-surgery side effects, the proposed concept is an alternative approach wherein spoken dialogue is achieved by means of employing a robust speech recognition technique which takes into consideration of noisy environmental factors without any attachment into human body. The proposed system is a portable device with an acoustic beamformer for directional noise reduction and capable of performing speech-to-text transcription function, which adopts a keyword spotting method. It is also equipped with an optimized user interface for hearing-impaired people, rendering intuitive and natural device usage with diverse domain contexts. The relevant experimental results confirm that the proposed interface design is feasible for realizing an effective and efficient intelligent agent for hearing-impaired.

Original languageEnglish
Pages (from-to)915-926
Number of pages12
JournalMedical and Biological Engineering and Computing
Volume54
Issue number6
DOIs
Publication statusPublished - 2016 Jun 1

Fingerprint

Audition
Cochlear implants
Hearing aids
Intelligent agents
Transcription
Noise abatement
Speech recognition
Surgery
User interfaces
Acoustics
Communication

Keywords

  • Beamforming
  • Hearing-impaired person
  • Keyword spotting
  • Speech-to-text transcription
  • User interface

ASJC Scopus subject areas

  • Biomedical Engineering
  • Computer Science Applications

Cite this

Dialogue enabling speech-to-text user assistive agent system for hearing-impaired person. / Lee, Seongjae; Kang, Sunmee; Han, David K.; Ko, Hanseok.

In: Medical and Biological Engineering and Computing, Vol. 54, No. 6, 01.06.2016, p. 915-926.

Research output: Contribution to journalArticle

@article{df5ad43e49644bc3b5153a30658c019e,
title = "Dialogue enabling speech-to-text user assistive agent system for hearing-impaired person",
abstract = "A novel approach for assisting bidirectional communication between people of normal hearing and hearing-impaired is presented. While the existing hearing-impaired assistive devices such as hearing aids and cochlear implants are vulnerable in extreme noise conditions or post-surgery side effects, the proposed concept is an alternative approach wherein spoken dialogue is achieved by means of employing a robust speech recognition technique which takes into consideration of noisy environmental factors without any attachment into human body. The proposed system is a portable device with an acoustic beamformer for directional noise reduction and capable of performing speech-to-text transcription function, which adopts a keyword spotting method. It is also equipped with an optimized user interface for hearing-impaired people, rendering intuitive and natural device usage with diverse domain contexts. The relevant experimental results confirm that the proposed interface design is feasible for realizing an effective and efficient intelligent agent for hearing-impaired.",
keywords = "Beamforming, Hearing-impaired person, Keyword spotting, Speech-to-text transcription, User interface",
author = "Seongjae Lee and Sunmee Kang and Han, {David K.} and Hanseok Ko",
year = "2016",
month = "6",
day = "1",
doi = "10.1007/s11517-015-1447-8",
language = "English",
volume = "54",
pages = "915--926",
journal = "Medical and Biological Engineering and Computing",
issn = "0140-0118",
publisher = "Springer Verlag",
number = "6",

}

TY - JOUR

T1 - Dialogue enabling speech-to-text user assistive agent system for hearing-impaired person

AU - Lee, Seongjae

AU - Kang, Sunmee

AU - Han, David K.

AU - Ko, Hanseok

PY - 2016/6/1

Y1 - 2016/6/1

N2 - A novel approach for assisting bidirectional communication between people of normal hearing and hearing-impaired is presented. While the existing hearing-impaired assistive devices such as hearing aids and cochlear implants are vulnerable in extreme noise conditions or post-surgery side effects, the proposed concept is an alternative approach wherein spoken dialogue is achieved by means of employing a robust speech recognition technique which takes into consideration of noisy environmental factors without any attachment into human body. The proposed system is a portable device with an acoustic beamformer for directional noise reduction and capable of performing speech-to-text transcription function, which adopts a keyword spotting method. It is also equipped with an optimized user interface for hearing-impaired people, rendering intuitive and natural device usage with diverse domain contexts. The relevant experimental results confirm that the proposed interface design is feasible for realizing an effective and efficient intelligent agent for hearing-impaired.

AB - A novel approach for assisting bidirectional communication between people of normal hearing and hearing-impaired is presented. While the existing hearing-impaired assistive devices such as hearing aids and cochlear implants are vulnerable in extreme noise conditions or post-surgery side effects, the proposed concept is an alternative approach wherein spoken dialogue is achieved by means of employing a robust speech recognition technique which takes into consideration of noisy environmental factors without any attachment into human body. The proposed system is a portable device with an acoustic beamformer for directional noise reduction and capable of performing speech-to-text transcription function, which adopts a keyword spotting method. It is also equipped with an optimized user interface for hearing-impaired people, rendering intuitive and natural device usage with diverse domain contexts. The relevant experimental results confirm that the proposed interface design is feasible for realizing an effective and efficient intelligent agent for hearing-impaired.

KW - Beamforming

KW - Hearing-impaired person

KW - Keyword spotting

KW - Speech-to-text transcription

KW - User interface

UR - http://www.scopus.com/inward/record.url?scp=84954193404&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84954193404&partnerID=8YFLogxK

U2 - 10.1007/s11517-015-1447-8

DO - 10.1007/s11517-015-1447-8

M3 - Article

C2 - 26753778

AN - SCOPUS:84954193404

VL - 54

SP - 915

EP - 926

JO - Medical and Biological Engineering and Computing

JF - Medical and Biological Engineering and Computing

SN - 0140-0118

IS - 6

ER -