Advances in Natural Multimodal Dialogue SystemsJan van Kuppevelt, Laila Dybkjær, Niels Ole Bernsen Springer Science & Business Media, 6 dic 2005 - 373 páginas References 74 Part II Annotation and Analysis of Multimodal Data: Speech and Gesture 4 FORM 79 Craig H. Martell 1. Introduction 79 2. Structure of FORM 80 3. Annotation Graphs 85 4. Annotation Example 86 5. Preliminary Inter-Annotator Agreement Results 88 6. Conclusion: Applications to HLT and HCI? 90 Appendix: Other Tools, Schemes and Methods of Gesture Analysis 91 References 95 5 97 On the Relationships among Speech, Gestures, and Object Manipulation in Virtual Environments: Initial Evidence Andrea Corradini and Philip R. Cohen 1. Introduction 97 2. Study 99 3. Data Analysis 101 4. Results 103 5. Discussion 106 6. Related Work 106 7. Future Work 108 8. Conclusions 108 Appendix: Questionnaire MYST III - EXILE 110 References 111 6 113 Analysing Multimodal Communication Patrick G. T. Healey, Marcus Colman and Mike Thirlwell 1. Introduction 113 2. Breakdown and Repair 117 3. Analysing Communicative Co-ordination 125 4. Discussion 126 References 127 7 131 Do Oral Messages Help Visual Search? Noëlle Carbonell and Suzanne Kieffer 1. Context and Motivation 131 2. Methodology and Experimental Set-Up 134 3. Results: Presentation and Discussion 141 4. Conclusion 153 References 154 Contents vii 8 159 Geometric and Statistical Approaches to Audiovisual Segmentation Trevor Darrell, John W. Fisher III, Kevin W. Wilson, and Michael R. Siracusa 1. Introduction 159 2. Related Work 160 3. Multimodal Multisensor Domain 162 4. Results 166 5. Single Multimodal Sensor Domain 167 6. |
Índice
NATURAL AND MULTIMODAL INTERACTIVITY ENGINEERING DIRECTIONS AND NEEDS | 1 |
2 Chapter Presentations | 2 |
3 NMIE Contributions by the Included Chapters | 7 |
4 Multimodality and Natural Interactivity | 16 |
References | 19 |
MAKING DIALOGUES MORE NATURAL EMPIRICAL WORK AND APPLIED THEORY | 21 |
SOCIAL DIALOGUE WITH EMBODIED CONVERSATIONAL AGENTS | 23 |
2 Embodied Conversational Agents | 25 |
THE PSYCHOLOGY AND TECHNOLOGY OF TALKING HEADS APPLICATIONS IN LANGUAGE LEARNING | 183 |
2 Facial Animation and Visible Speech Synthesis | 184 |
3 Speech Science | 191 |
4 Language Learning | 194 |
5 Research on the Educational Impact of Animated Tutors | 197 |
6 Summary | 210 |
References | 211 |
EFFECTIVE INTERACTION WITH TALKING ANIMATED AGENTS IN DIALOGUE SYSTEMS | 215 |
3 Social Dialogue | 29 |
4 Related Work | 32 |
5 Social Dialogue in REA | 36 |
6 A Study Comparing ECA Social Dialogue with AudioOnly Social Dialogue | 40 |
7 Conclusion | 48 |
References | 49 |
A FIRST EXPERIMENT IN ENGAGEMENT FOR HUMANROBOT INTERACTION IN HOSTING ACTIVITIES | 55 |
2 Hosting Activities | 56 |
3 What is Engagement? | 57 |
A Pointing Robot | 59 |
5 Making Progress on Hosting Behaviours | 62 |
6 Engagement for HumanHuman Interaction | 63 |
7 Computational Modelling of HumanHuman Hosting and Engagement | 70 |
8 A Next Generation Mel | 73 |
9 Summary | 74 |
ANNOTATION AND ANALYSIS OF MULTIMODAL DATA SPEECH AND GESTURE | 77 |
FORM | 79 |
2 Structure of FORM | 80 |
3 Annotation Graphs | 85 |
4 Annotation Example | 86 |
5 Preliminary InterAnnotator Agreement Results | 88 |
Applications to HLT and HCI? | 90 |
Other Tools Schemes and Methods of Gesture Analysis | 91 |
References | 95 |
ON THE RELATIONSHIPS AMONG SPEECH GESTURES AND OBJECT MANIPULATION IN VIRTUAL ENVIRONMENTS INITIAL EVIDENCE | 97 |
2 Study | 99 |
3 Data Analysis | 101 |
4 Results | 103 |
5 Discussion | 106 |
7 Future Work | 108 |
Questionnaire MYST III EXILE | 110 |
References | 111 |
ANALYSING MULTIMODAL COMMUNICATION | 113 |
2 Breakdown and Repair | 117 |
3 Analysing Communicative Coordination | 125 |
4 Discussion | 126 |
References | 127 |
DO ORAL MESSAGES HELP VISUAL SEARCH? | 131 |
2 Methodology and Experimental SetUp | 134 |
Presentation and Discussion | 141 |
4 Conclusion | 153 |
References | 154 |
GEOMETRIC AND STATISTICAL APPROACHES TO AUDIOVISUAL SEGMENTATION | 159 |
2 Related Work | 160 |
3 Multimodal Multisensor Domain | 162 |
4 Results | 166 |
5 Single Multimodal Sensor Domain | 167 |
6 Integration | 175 |
References | 178 |
ANIMATED TALKING HEADS AND EVALUATION | 181 |
2 The KTH Talking Head | 217 |
3 Effectiveness in Intelligibility and Information Presentation | 219 |
4 Effectiveness in Interaction | 223 |
5 Experimental Applications | 231 |
6 The Effective Agent as a Language Tutor | 235 |
7 Experiments and 3D Recordings for the Expressive Agent | 237 |
References | 239 |
CONTROLLING THE GAZE OF CONVERSATIONAL AGENTS | 245 |
2 Functions of Gaze | 248 |
3 The Experiment | 252 |
4 Discussion | 258 |
5 Conclusion | 260 |
ARCHITECTURES AND TECHNOLOGIES FOR ADVANCED AND ADAPTIVE MULTIMODAL DIALOGUE SYSTEMS | 263 |
MIND A CONTEXTBASED MULTIMODAL INTERPRETATION FRAMEWORK IN CONVERSATIONAL SYSTEMS | 265 |
2 Related Work | 267 |
4 Example Scenario | 268 |
5 SemanticsBased Representation | 270 |
6 ContextBased Multimodal Interpretation | 277 |
7 Discussion | 282 |
References | 283 |
A GENERAL PURPOSE ARCHITECTURE FOR INTELLIGENT TUTORING SYSTEMS | 287 |
2 An Intelligent Tutoring System for Damage Control | 288 |
3 An Architecture for Multimodal Dialogue Systems | 294 |
4 Activity Models | 295 |
5 Dialogue Management Architecture | 298 |
6 Benefits of ACI for Intelligent Tutoring Systems | 301 |
7 Conclusion | 302 |
References | 303 |
MIAMM A MULTIMODAL DIALOGUE SYSTEM USING HAPTICS | 307 |
1 Introduction | 308 |
2 Haptic Interaction in a Multimodal Dialogue System | 309 |
3 Visual Haptic Interaction Concepts in MIAMM | 313 |
4 Dialogue Management | 319 |
5 The Multimodal Interface Language MMIL | 326 |
6 Conclusion | 331 |
ADAPTIVE HUMANCOMPUTER DIALOGUE | 333 |
2 Overview of Language Acquisition | 334 |
3 Dialogue Systems | 337 |
4 Language Knowledge Representation | 340 |
5 Dialogue Adaptation | 341 |
6 Experiments | 347 |
353 | |
MACHINE LEARNING APPROACHES TO HUMAN DIALOGUE MODELLING | 355 |
2 Modality Independent Dialogue Management | 357 |
3 Learning to Annotate Utterances | 362 |
Data Driven Dialogue Discovery | 366 |
5 Discussion | 367 |
368 | |
371 | |
Otras ediciones - Ver todo
Advances in Natural Multimodal Dialogue Systems Jan van Kuppevelt,Laila Dybkjær,Niels Ole Bernsen Vista previa restringida - 2006 |
Advances in Natural Multimodal Dialogue Systems Jan van Kuppevelt,Laila Dybkjær,Niels Ole Bernsen No hay ninguna vista previa disponible - 2005 |
Términos y frases comunes
activities analysis animated annotation application architecture articulation audio audiovisual autism Baldi beamforming behaviour Bickmore Cassell chapter collaboration communication computer vision concept classes context cues database developed dialogue act dialogue manager dialogue move discourse display domain editors effects embodied conversational agents engagement environment evaluation example experiment face face-to-face feedback Figure function gaze gesture goal grammar Granström graphical haptic hosting human Human-Computer Interaction human-human initial intelligent tutoring system interface interpretation Karin language learning linguistic machine learning Massaro MIAMM modalities module MP condition Multimodal Dialogue Systems multimodal interaction mutual information n-gram natural NMIE objects output parameters participants presented Proceedings repair representation robot scenes segments semantic Sidner signal small talk social dialogue spatial speaker Speech Perception speech synthesis structure student subjects talking head target task Technology text-to-speech synthesis tion user inputs user's utterance visible speech visual search vocabulary