基于深度情感唤醒网络的多模态情感分析与情绪识别
DOI:
作者:
作者单位:

河北大学数学与信息科学学院

作者简介:

通讯作者:

中图分类号:

TP181

基金项目:

国家自然科学基金项目(面上项目,重点项目,重大项目),


Deep Emotional Arousal Network for Multimodal Sentiment Analysis and Emotion Recognition
Author:
Affiliation:

College of Mathematics and Information Science

Fund Project:

The National Natural Science Foundation of China (General Program, Key Program, Major Research Plan)

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    随着网络平台上各类图像、视频数据的快速增长,多模态情感分析与情绪识别已成为一个日益热门的研究领域.相比于单模态情感分析,多模态情感分析的模态融合策略是一个亟待解决的关键问题.本文受到认知科学中情感唤起模型的启发,提出一种能够模拟人类处理多通道输入信息机制的深度情感唤醒网络(Deep Emotional Arousal Network, DEAN),该网络可实现多模态信息有机融合,既能处理情绪的连贯性,又避免了融合机制的选择问题.深度情感唤醒网络主要由以下三部分组成:跨模态Transformer模块,用以模拟人类知觉分析系统的功能;多模态LSTM系统,用以模拟认知比较器;多模态门控模块,用以模拟情感唤起模型的激活结构.在多模态情感分析与情绪识别的三个数据集上进行的比较实验结果表明,深度情感唤醒网络在各数据集上均超越了目前最先进情感分析模型的性能.

    Abstract:

    With a large number of emerging of various images and videos on internet, multimodal sentiment analysis and emotion recognition have become an increasingly popular research topic. Compared with the unimode sentiment analysis, one of the key issues in multimodal sentiment analysis is the fusion strategy. Inspired by the emotional arousal model in cognitive science, a Deep Emotional Arousal Network (DEAN), which can simulate emotional coherence, is proposed in this paper. The proposed DEAN consists of three parts: a crossmodal Transformer module that simulates the functions of a human perception analysis system; a Multimodal BiLSTM system to simulate the cognitive comparator; and a multimodal gating module that is used to simulate the activation mechanism in physiological emotional arousal model. Extensive experimental comparisons on three benchmarks for multimodal sentiment analysis and emotion recognition are conducted and the experimental results show that DEAN outperforms the state-of-the-art methods.

    参考文献
    相似文献
    引证文献
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-05-05
  • 最后修改日期:2021-08-02
  • 录用日期:2021-08-09
  • 在线发布日期:
  • 出版日期: