Band-Aid Backdoor: A Dual-Branch Framework for Physical Backdoor Attacks on Face Recognition Systems

Abstract

A physical backdoor attack framework against face recognition systems, using band-aid stickers as visually plausible triggers, is proposed. The proposed method introduces a dual-branch architecture that decouples identity recognition from trigger classification, enabling the model to maintain high recognition accuracy on clean inputs while inducing targeted misclassification when a specific physical trigger is present. To ensure photorealistic integration, the band-aid undergoes a 3D curvature-aware warping and illumination adjustment process before being applied to facial images. Experiments conducted under both digital and real-world conditions demonstrate the effectiveness and specificity of the attack. Compared to a standard single-task model, the dual-branch model achieves significantly higher attack success rates and lower false positive rates, while remaining robust to unseen trigger types. However, results also reveal a performance gap between digital and physical scenarios, emphasizing the importance of visual distinctiveness and domain consistency. This work highlights the practical feasibility of physical backdoor attacks and provides insights into their design and mitigation.

Lab Members

Patrick Chan

Associate Professor, Vice Dean

Patrick Chan works on machine learning, deep learning, image processing, adversarial learning, and secure machine learning.