East China Jiaotong University
The National Natural Science Foundation of China (General Program, Key Program, Major Research Plan),Jiangxi Province Graduate Innovation Special Fundation of China
图像情感分析是机器视觉领域的研究热点,它面临的关键问题：标注者的主观差异性导致情感标签明确的高质量样本匮乏,且异构图像特征间跨模态语义未有效利用。提出基于主动样本精选与跨模态语义挖掘的图像情感分析模型ASRF2 (Active Sample Refinement Feature Fusion)：融合主动学习与样本精选思想,设计主动样本精选策略,优选情感标签明确的高质量样本；对异构图像特征执行判别相关分析,生成能准确刻画图像情感内容的低维跨模态语义；采用跨模态语义训练Catboost模型,实现图像情感分析。在Twitter 1与FI数据集上验证ASRF2模型,对应识别准确率分别达90.06%和75.77%,优于主流基线且实时效率较好。相比基线,ASRF2模型仅需两类特征,参数调制简单,故易于复现,可部署到相应终端以对接实际应用。
Image sentiment analysis is a research focus in the computer vision field. However, we are faced with the following key problems: First, due to the subjective differences of annotators, high-quality samples with definite sentimental annotations are very scarce. Second, the implicit cross-modal semantics among heterogeneous features has not been fully explored. To address these two problems, we propose a novel but effective model called active sample refinement feature fusion (ASRF2) based on active sample refinement and cross-modal semantics mining: a new active sample refinement algorithm is designed by fusing active learning and sample refinement ideas. High-quality samples with definite sentimental annotations are obtained in turn. Then, the state-of-the-art discriminant correlation analysis (DCA) algorithm is used to fully mine the cross-modal correlations among heterogeneous features. Low-dimensional but more discriminant cross-modal semantics that can better depict the key sentimental contents of images are generated. The cross-modal semantics is used to train a Catboost classifier and complete image sentiment analysis. We validate the proposed ASRF2 model on the Twitter 1 and FI datasets. The corresponding accuracies reach about 90.06% and 75.77%, respectively, which outperform state-of-the-art baselines as well as the real-time efficiency. More importantly, compared with baselines, the proposed model only needs two features, and it is easy to tune the model. This means that it is easy to reproduce the ASRF2 model. Hence, the ASRF2model can be employed in some terminal devices to better dock many real applications.