Picked from the software, topic, and level of “Class Deepfake Detection Vision Transformer #4: Fine-Tuning Pretrained ViT for Original vs Deepfake Classification”.
At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, the Vision Transformer Pre-Trained model we fine-tuned to distinguish between real faces and Style Gen generated faces, with 92% validation accuracy in just 8 epochs on CPU. There is a small trick that increased the accuracy from 86% to 92%. Let's get started. Welcome to episode four of the Vision Transformer-based deepfake detection series. In the previous episode, we wrote E6 IT from scratch. Training E6 IT from scratch requires millions of images, so now we are using this feature from the team's library, which was trained on ImageNet with 21,000 classes, then performing fine-tuning for a new task, distinguishing between real faces and deepfakes.
Step 1: Dataset without resizing
Create the dataset underscore face.py file There is one important decision here. We do not resize the images and do not use random resize crop, just horizontal flip. The reason goes back to episode 1. The gain trace is in the high frequencies, and every resize smooths out that trace. In the experiment, resizing to 128 pixels only gave about 86% accuracy, while the original resolution of 192 gave 92%. Normalization is mean and standard deviation of 0.5, according to this E6 IT pre-trained model. Images are loaded once into memory. Now create the train underscore E6 IT.py file.
Step 2: AdamW training function & label smoothing
The accuracy function calculates precision on a loader. The train function uses the AdamW optimizer with Weight Decay, and cosine annealing scheduler, and cross entropy loss with label smoothing 0.1, so the model doesn't become too confident and generalizes more easily. The model with the best validation accuracy is saved. This function is also used for CNN in the next episode. The E6 IT function loads Fitini Pack 16 from the team with pretrinet weights,
Step 3: Pretrained ViT-Tiny from timm
two-class classification head, and Imsai 192. The team automatically adjusts the positional embedding from 224 to 192 pixels, so there is 12x12, 144 pack.
Step 4: Fine-tuning results
Run. Fitini has 5.5 million parameters, 12 blocks, and 3 attention heads. Each epoch takes about 17 seconds on CPU. Validation accuracy at 79% in the first epoch, rises to 92.2% in the third epoch, then stabilizes around 90. The left graph shows the loss decreasing and leveling off near the lower bound of label smoothing. The right graph shows the validation accuracy. The best model is saved as NMIT best.pt.
Done!
Done. We have performed fine-tuning of the NMIT pretrinet for deepfake detection, understood why resizim breaks the footprint, and used adamwe cosine annealing, as well as label smoothing. In the next episode, we will compare it with the EfficientNet CNN. and test the resilience of both models when images are compressed as JPEG, as is common on social media. Working on a thesis, thesis, Or are you doing AI research in the field of deepfake detection and image forensics? Cerdas Inovasi Digital helps with deep learning implementation, from face detection, vision transformer, comparison with CNN, test compression resilience, until the attention map and web application, as well as opening the deeplearning course with PyTor. Want to master the entire series? From face datasets to deepfake checker applications? All 6 episodes, complete with a step-by-step PDF guide and source code, Available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, Save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy single class with lifetime access. Subscribe now on the premium page cerdasinovasidigital.com. Contact our WhatsApp or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,286 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
