Retrieving tract variables from acoustics: A comparison of different machine learning strategies

Vikramjit Mitra, Hosung Nam, Carol Y. Espy-Wilson, Elliot Saltzman, Louis Goldstein

Research output: Contribution to journalArticle

37 Citations (Scopus)

Abstract

Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed speech-inversion. This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.

Original languageEnglish
Article number5570879
Pages (from-to)1027-1045
Number of pages19
JournalIEEE Journal on Selected Topics in Signal Processing
Volume4
Issue number6
DOIs
Publication statusPublished - 2010 Dec 1
Externally publishedYes

Fingerprint

Learning systems
Acoustics
Trajectories
Neural networks
Supervised learning
Speech recognition

Keywords

  • Articulatory phonology
  • articulatory speech recognition (ASR)
  • artificial neural networks (ANNs)
  • coarticulation
  • distal supervised learning
  • mixture density networks
  • speech inversion
  • task dynamic and applications model
  • vocal-tract variables

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Retrieving tract variables from acoustics : A comparison of different machine learning strategies. / Mitra, Vikramjit; Nam, Hosung; Espy-Wilson, Carol Y.; Saltzman, Elliot; Goldstein, Louis.

In: IEEE Journal on Selected Topics in Signal Processing, Vol. 4, No. 6, 5570879, 01.12.2010, p. 1027-1045.

Research output: Contribution to journalArticle

Mitra, Vikramjit ; Nam, Hosung ; Espy-Wilson, Carol Y. ; Saltzman, Elliot ; Goldstein, Louis. / Retrieving tract variables from acoustics : A comparison of different machine learning strategies. In: IEEE Journal on Selected Topics in Signal Processing. 2010 ; Vol. 4, No. 6. pp. 1027-1045.
@article{8cce4264f8fd404889db0d7eb11ed983,
title = "Retrieving tract variables from acoustics: A comparison of different machine learning strategies",
abstract = "Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed speech-inversion. This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.",
keywords = "Articulatory phonology, articulatory speech recognition (ASR), artificial neural networks (ANNs), coarticulation, distal supervised learning, mixture density networks, speech inversion, task dynamic and applications model, vocal-tract variables",
author = "Vikramjit Mitra and Hosung Nam and Espy-Wilson, {Carol Y.} and Elliot Saltzman and Louis Goldstein",
year = "2010",
month = "12",
day = "1",
doi = "10.1109/JSTSP.2010.2076013",
language = "English",
volume = "4",
pages = "1027--1045",
journal = "IEEE Journal on Selected Topics in Signal Processing",
issn = "1932-4553",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "6",

}

TY - JOUR

T1 - Retrieving tract variables from acoustics

T2 - A comparison of different machine learning strategies

AU - Mitra, Vikramjit

AU - Nam, Hosung

AU - Espy-Wilson, Carol Y.

AU - Saltzman, Elliot

AU - Goldstein, Louis

PY - 2010/12/1

Y1 - 2010/12/1

N2 - Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed speech-inversion. This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.

AB - Many different studies have claimed that articulatory information can be used to improve the performance of automatic speech recognition systems. Unfortunately, such articulatory information is not readily available in typical speaker-listener situations. Consequently, such information has to be estimated from the acoustic signal in a process which is usually termed speech-inversion. This study aims to propose and compare various machine learning strategies for speech inversion: Trajectory mixture density networks (TMDNs), feedforward artificial neural networks (FF-ANN), support vector regression (SVR), autoregressive artificial neural network (AR-ANN), and distal supervised learning (DSL). Further, using a database generated by the Haskins Laboratories speech production model, we test the claim that information regarding constrictions produced by the distinct organs of the vocal tract (vocal tract variables) is superior to flesh-point information (articulatory pellet trajectories) for the inversion process.

KW - Articulatory phonology

KW - articulatory speech recognition (ASR)

KW - artificial neural networks (ANNs)

KW - coarticulation

KW - distal supervised learning

KW - mixture density networks

KW - speech inversion

KW - task dynamic and applications model

KW - vocal-tract variables

UR - http://www.scopus.com/inward/record.url?scp=78649390043&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78649390043&partnerID=8YFLogxK

U2 - 10.1109/JSTSP.2010.2076013

DO - 10.1109/JSTSP.2010.2076013

M3 - Article

AN - SCOPUS:78649390043

VL - 4

SP - 1027

EP - 1045

JO - IEEE Journal on Selected Topics in Signal Processing

JF - IEEE Journal on Selected Topics in Signal Processing

SN - 1932-4553

IS - 6

M1 - 5570879

ER -