Picked from the software, topic, and level of “Class Deepfake Detection Vision Transformer #6: Attention Map for Interpretation & Deepfake Checker Application”.
At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
Here is the final result, a web-based deepfake detector application. Select or upload a photo, the face is automatically detected, then the Vision Transformer evaluates whether the face is real or AI-generated, and the attention map shows the areas the model focuses on when making a decision. Let's get started. Hello, welcome to the final episode of the Vision Transformer-based deepfake detection series. A real-world deepfake detector must be explainable, not just a black box. Today we are creating an attention map using the attention rollout technique, and packaging the model into a web application that anyone can use.
Step 1: Capturing attention weights
Create a file named underscore attention dot pay. We load the six best ITs from episode 4. So that we can read the attention weights, the fucet attention mode in each block is turned off, then we set a forward hook on attend drop in the twelve blocks to capture the attention matrix every time the model runs. The rollout function is the core of the interpretation.
Step 2: Attention rollout
For each block, attention is averaged across heads, plus the identity matrix to represent residual connections, normalized, then multiplied in sequence from the first block to the last. The CLS token row from the result of this multiplication shows how much each pack influences the final decision. The result is a 12x12 grid that we normalize to 0 to 1.
Step 3: Attention map for real and fake faces
We visualize three real faces and three fake faces from the test data, complete with fake probability and attention maps above them. Run The three original faces received low fake probabilities, 0.02 up to 0.46, and the three fake faces 0.9 up to 0.96. Everything is correct. What's interesting, the model pays much attention to the image edges, background, and hair, not the center of the face. This aligns with research that artifact generation often appears in the background and object boundaries. At the end, six face examples are saved to the examples folder for the application.
Step 4: Streamlit Application: loading model & input
Now the application. Create a file app underscore deepfake.py with streamlib. The model is loaded once with minimal resource usage. The face cropping function uses the image as-is if it's already 192 in size, to prevent genetic trace damage, or crops the face from a free photo. The page contains an example select box and file uploader. In the inspection section,
Step 5: Inspection & running the application
faces are detected using units from episode 2, then classified by NMIT. The left column displays the photo with a face box, the right column shows the results Red box if deepfake is likely Green box if original Plus progress bar for fake probability and model limitations note We run it with streamlit run Then open it in the browser at local host port 8501
Step 6: Trying the application in the browser
The application appears First example Original face 1 Evaluated as original with confidence around 73% We select example fake face 1 And the application immediately evaluates it as likely deepfake with probability around 94% This model is trained specifically for face style gain So for other types of deepfake additional training data is needed
Done!
Done And this series is also done We have built a complete deepfake detection research Original face dataset and style gain Gain trace in frequency domain Face detection unit Vision transformer from 0 Fine tuning NMIT pretrinet Comparison with CNN and compression test Up to attention map and web application For further research Try face swap datasets like face forensics video, or a larger model. Thank you for following this series. Currently working on a thesis, dissertation, or AI research in the field of deepfake detection and image forensics? Cerdas Inovasi Digital helps implement deep learning, from face detection, vision transformer, comparison with CNN, compression resilience testing, up to attention maps and web applications, as well as opening deep learning courses with PyTor. Want to master the entire series, from face datasets to deepfake checker applications? All 6 episodes, complete with step-by-step PDF guides and source code, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want a single class only? Buy a single class with lifetime access. Subscribe now at the premium page cerdas inovasi digital.com. Contact us via WhatsApp, or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,290 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
