Picked from the software, topic, and level of “Class Deepfake Detection Vision Transformer #5: Comparison with CNN EfficientNet & JPEG Compression Resistance Test”.
At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
Here is the final result, the resilience graph of two deepfake detectors when faces are compressed with JPEG, such as when photos are sent via WhatsApp or social media. EfficientNet Angelop accuracy drops from 87 to 74%, while Vision Transformer remains stable above 38%. Let's get started. Hello, welcome to episode five of the Vision Transformer-based deepfake detection series. In research, proposed models must be compared with strong baselines. We are using EfficientNet B0, a modern and efficient CNN. Then we test something very important in the real world. Deepfake is almost always spread after being compressed by messaging and social media apps. Does our detector still work?
Step 1: EfficientNet-B0 as a benchmark
Create file lati underscore cnn.py The function buat CNN loads EfficientNet B0 pre-trained on ImageNet from TorchVision, and replaces the last layer with a linear 2-class layer. We are using the same training function as in episode 4, 8 epochs, Adamwe, cosine annealing, and label smoothing. The same protocol is a requirement for a fair comparison.
Step 2: VIT vs CNN Results
RUN EfficientNet B0 has 4.01 million parameters, slightly smaller than this model, but each epoch is almost 3 times longer, about 46 seconds, because of convolutions on full resolution. Its best validation accuracy is 87.2%, below NMIT's 92.2%. The graph compares the validation accuracy curves of both.
Step 3: JPEG Compression Simulation
Now create file uji underscore kompresi dot py. The function JPEG makes a transformation that saves the image as JPEG with a certain quality in memory, then opens it back up. We load the two best models, calculate test data accuracy without compression, then at JPEG quality 100, 90, 75, 50, and 30.
Step 4: Compression Resistance
RUN Without compression, NMIT 86.7% and EfficientNet 85% on test data, comparable. But look when compression gets heavier. At quality 75, EfficientNet drops to 77%, and at quality 30 only 74%. NMIT remains in the range of 84-87%. The explanation is that CNN tends to rely on fine pixel patterns that are removed by JPEG compression, while self-attention NMIT looks at global relationships between PECs that are more compression-resistant. Findings like these are very valuable for research discussion chapters.
2 Done!
Done. We have fairly compared vision transformer with EfficientNet CNN, and tested their resistance to JPEG compression. NMIT excels in both validation accuracy and resistance. In the final episode, we look at attention maps to interpret NMIT's decisions, and create a deepfake checker web application. Currently working on a thesis, dissertation, or AI research in the field of deepfake detection and image forensics? Cerdas Inovasi Digital helps with deep learning implementation, from face detection, vision transformer, comparison with CNN, compression resistance testing, up to attention maps and web applications, as well as opening deep learning courses with PyTorch. Want to master the entire series, from face detection sets to deepfake checker applications? All 6 episodes, complete with step-by-step PDF guides and source code, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or one-time payment of Rp999.000 for lifetime access. Want a single class only? Buy single classes with lifetime access. Subscribe now on the premium page cerdasinovasidigital.com. Contact our WhatsApp, or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
52 video classes1,276 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
