Class-Level Spectral Features for Emotion Recognition

Thumbnail Image
Penn collection
Departmental Papers (CIS)
Degree type
Computer Sciences
Grant number
Copyright date
Related resources

The most common approaches to automatic emotion recognition rely on utterance-level prosodic features. Recent studies have shown that utterance-level statistics of segmental spectral features also contain rich information about expressivity and emotion. In our work we introduce a more fine-grained yet robust set of spectral features: statistics of Mel-Frequency Cepstral Coefficients computed over three phoneme type classes of interest – stressed vowels, unstressed vowels and consonants in the utterance. We investigate performance of our features in the task of speaker-independent emotion recognition using two publicly available datasets. Our experimental results clearly indicate that indeed both the richer set of spectral features and the differentiation between phoneme type classes are beneficial for the task. Classification accuracies are consistently higher for our features compared to prosodic or utterance-level spectral features. Combination of our phoneme class features with prosodic features leads to even further improvement. Given the large number of class-level spectral features, we expected feature selection will improve results even further, but none of several selection methods led to clear gains. Further analyses reveal that spectral features computed from consonant regions of the utterance contain more information about emotion than either stressed or unstressed vowel features. We also explore how emotion recognition accuracy depends on utterance length. We show that, while there is no significant dependence for utterance-level prosodic features, accuracy of emotion recognition using class-level spectral features increases with the utterance length.

Date of presentation
Conference name
Departmental Papers (CIS)
Conference dates
Conference location
Date Range for Data Collection (Start Date)
Date Range for Data Collection (End Date)
Digital Object Identifier
Series name and number
Volume number
Issue number
Publisher DOI
Journal Issue
<p>Bitouk, D., Verma, R., & Nenkova, A., Class-Level Spectral Features for Emotion Recognition, <em>Speech Communication, Volume 52, Issues 7-8</em>, July-August 2010, doi: <a href="" target="_blank" title="10.1016/j.specom.2010.02.010">10.1016/j.specom.2010.02.010</a></p>
Recommended citation