Robust auxiliary modality is beneficial for video-based cloth-changing person re-identification

Youming Chen, Ting Tuo, Lijun Guo*, Rong Zhang, Yirui Wang, Shangce Gao

*この論文の責任著者

研究成果: ジャーナルへの寄稿総説査読

1 被引用数 (Scopus)

抄録

The core of video-based cloth-changing person re-identification is the extraction of cloth-irrelevant features, such as body shape, face, and gait. Most current methods rely on auxiliary modalities to help the model focus on these features. Although these modalities can resist the interference of clothing appearance, they are not robust against cloth-changing, which affects model recognition. The joint point information of pedestrians was considered to better resist the impact of cloth-changing; however, it contained limited pedestrian discrimination information. In contrast, the silhouettes had rich pedestrian discrimination information and could resist interference from clothing appearance but were vulnerable to cloth-changing. Therefore, we combined these two modalities to construct a more robust modality that minimized the impact of clothing on the model. We designed different usage methods for the temporal and spatial aspects based on the characteristics of the fusion modality to enhance the model for extracting cloth-irrelevant features. Specifically, at the spatial level, we developed a guiding method retaining fine-grained, cloth-irrelevant features while using fused features to reduce the focus on cloth-relevant features in the original image. At the temporal level, we designed a fusion method that combined action features from the silhouette and joint point sequences, resulting in more robust action features for cloth-changing pedestrians. Experiments on two video-based cloth-changing datasets, CCPG-D and CCVID, indicated that our proposed model outperformed existing state-of-the-art methods. Additionally, tests on the gait dataset CASIA-B demonstrated that our model achieved optimal average precision.

本文言語英語
論文番号105400
ジャーナルImage and Vision Computing
154
DOI
出版ステータス出版済み - 2025/02

ASJC Scopus 主題領域

  • 信号処理
  • コンピュータ ビジョンおよびパターン認識

引用スタイル