Using model trees for evaluating dialog error conditions based on acoustic information

Abe Kazemzadeh; Sungbok Lee; Shrikanth Narayanan

doi:10.1145/1178745.1178763

Back

Conference proceeding

Using model trees for evaluating dialog error conditions based on acoustic information

Abe Kazemzadeh, Sungbok Lee and Shrikanth Narayanan

Proceedings of the 1st ACM international workshop on Human-centered multimedia, pp.109-114

ACM Conferences

MM06: The 14th ACM International Conference on Multimedia 2006

10/27/2006

DOI: https://doi.org/10.1145/1178745.1178763

Abstract

Computing methodologies -- Artificial intelligence -- Philosophical/theoretical foundations of artificial intelligence -- Cognitive science

Hardware -- Communication hardware, interfaces and storage -- Sound-based input -- output

Human-centered computing -- Human computer interaction (HCI)

Human-centered computing -- Human computer interaction (HCI) -- HCI design and evaluation methods

Human-centered computing -- Human computer interaction (HCI) -- Interaction devices -- Sound-based input -- output

Human-centered computing -- Human computer interaction (HCI) -- Interaction paradigms -- Natural language interfaces

Human-centered computing -- Interaction design -- Interaction design process and methods -- User centered design

Information systems -- Information retrieval -- Document representation

Information systems -- Information retrieval -- Search engine architectures and scalability -- Search engine indexing

This paper examines the use of model trees for evaluating user utterances for response to system error in dialogs from the Communicator 2000 corpus. The features used by the model trees are limited to those which can be automatically obtained through acoustic measurements. These features are derived from pitch and energy measurements. The curve of the model tree output versus dialog turn is interpreted to be a measure of the level of user activation in the dialog. We test the premise that user response to error at the utterance level is related to user satisfaction at the dialog level. Several different evaluation tasks are investigated: on an utterance level we applied the model tree output to detecting response to error and on the dialog level we analyzed the relation of model tree output to estimating user satisfaction. For the former, we achieve 65% precision and 63% recall and for the latter our predictions show significant .48 correlation with user surveys.

Metrics

3 Record Views

Details

Title: Using model trees for evaluating dialog error conditions based on acoustic information
Author/Creator: Abe Kazemzadeh - University of Southern California
Sungbok Lee - University of Southern California
Shrikanth Narayanan - University of Southern California
Publication Details: Proceedings of the 1st ACM international workshop on Human-centered multimedia, pp.109-114
Conference: MM06: The 14th ACM International Conference on Multimedia 2006
Series: ACM Conferences
Publisher: ACM
Academic Unit: Software Engineering and Data Science
Language: English
Resource Type: Conference proceeding
Record Identifier: 991015166274803691

Using model trees for evaluating dialog error conditions based on acoustic information

Abstract

Related links

Metrics

Details