TY - JOUR
T1 - Self-supervised scheme for generalizing GAN image detection
AU - Jeong, Yonghyun
AU - Kim, Doyeon
AU - Kim, Pyounggeon
AU - Ro, Youngmin
AU - Choi, Jongwon
N1 - Publisher Copyright:
© 2024 Elsevier B.V.
PY - 2024/8
Y1 - 2024/8
N2 - Although the recent advancement in generative models brings diverse advantages to society, it can also be abused with malicious purposes, such as fraud, defamation, and fake news. To prevent such cases, vigorous research is conducted to distinguish the generated images from the real images, but challenges still remain to distinguish the generated images outside of the training settings. Such limitations occur due to data dependency arising from the model's overfitting issue to the specific Generative Adversarial Networks (GANs) and categories of the training data. To overcome this issue, we adopt a self-supervised scheme. Our method is composed of the artificial artifact generator reconstructing the high-quality artificial artifacts of GAN images, and the GAN detector distinguishing GAN images by learning the reconstructed artificial artifacts. To improve the generalization of the artificial artifact generator, we build multiple autoencoders with different numbers of upconvolution layers. With numerous ablation studies, the robust generalization of our method is validated by outperforming the generalization of the previous state-of-the-art algorithms, even without utilizing the GAN images of the training dataset.
AB - Although the recent advancement in generative models brings diverse advantages to society, it can also be abused with malicious purposes, such as fraud, defamation, and fake news. To prevent such cases, vigorous research is conducted to distinguish the generated images from the real images, but challenges still remain to distinguish the generated images outside of the training settings. Such limitations occur due to data dependency arising from the model's overfitting issue to the specific Generative Adversarial Networks (GANs) and categories of the training data. To overcome this issue, we adopt a self-supervised scheme. Our method is composed of the artificial artifact generator reconstructing the high-quality artificial artifacts of GAN images, and the GAN detector distinguishing GAN images by learning the reconstructed artificial artifacts. To improve the generalization of the artificial artifact generator, we build multiple autoencoders with different numbers of upconvolution layers. With numerous ablation studies, the robust generalization of our method is validated by outperforming the generalization of the previous state-of-the-art algorithms, even without utilizing the GAN images of the training dataset.
KW - Deepfake detection
KW - GAN detector
KW - Self-supervised learning
UR - http://www.scopus.com/inward/record.url?scp=85197808609&partnerID=8YFLogxK
U2 - 10.1016/j.patrec.2024.06.030
DO - 10.1016/j.patrec.2024.06.030
M3 - Article
AN - SCOPUS:85197808609
SN - 0167-8655
VL - 184
SP - 219
EP - 224
JO - Pattern Recognition Letters
JF - Pattern Recognition Letters
ER -