Picked from the software, topic, and level of “Class Deepfake Detection Vision Transformer #1: Deepfake Concepts, Manipulation Types & Real vs Fake Face Dataset”.
At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
Here is the final result, a series of human faces, half of them real and half of them AI-generated, almost impossible to distinguish with the naked eye. But in the frequency domain, fake faces leave subtle traces. This is the first step in our series to build a deepfake detector using Vision Transformer. Let's get started. Hello, welcome to the AI research tutorial series, Deepfake and Digital Image Manipulation Detection using Vision Transformer. Deepfake is an image or video of a face created or altered by AI, and it is now used for fraud, hoaxes, and identity impersonation. There are several types of manipulation, including faces entirely generated by generative models, facewap which swaps faces, and face reenagement which animates expressions. In this 6-episode series, we will build the detector, dataset, face detection, understand Vision Transformer, fine-tuning, comparison with CNN, compression testing, attention maps, and web application. In this first episode, we will get to know the dataset. We open the terminal through the terminal menu, new terminal, and then install the libraries for this series, Torque, Torch Vision, Theme, Open Pit Home, Scikit Learn, Matplotlib, and Streamlit.
Step 1: Installing the library & dataset
Theme is a collection of Vision models including Vision Transformer Pretrinet. The dataset comes from 140,000 real and fake faces, real faces from FFHQ owned by Nvidia, and fake faces generated by style gen. We use 2,400 faces, available in a special customer premium project file.
Step 2: Listing files and splitting data
Create a file named data_underscore_face.py. The folder structure is simple, a folder for real and a folder for fake. The list_files function generates pairs of each photo along with its label, 0 for real and 1 for fake. The split_data function divides the data into 70, 15, 15% stratified with a fixed random state, so that each part is balanced and the results are reproducible.
Step 3: Counting and splitting data
Now create a file named exploration_underscore_face.py and run it. There are 2,400 images, 1,200 real and 1,200 fake, each of size 192 pixels. The split is 1,680 for training, 360 for validation, and 360 for testing, and each part has exactly 50% fake.
Step 4: Real vs fake faces
Let's take a look at an example. Top row is real faces, bottom row is fake faces. Try to guess before seeing the label, almost impossible. Style gain produces skin, hair, and lighting that are very convincing. Sometimes there are small clues like an unusual background, asymmetrical earrings, or melting hair texture, but they are not reliable. Where is the trace?
Step 5: GAN traces in the frequency domain
We calculate the frequency spectrum of each face using the Fourier transform, then average it for 300 real faces and 300 fake faces. At first glance, they are similar, but the difference map on the right shows a subtle pattern in high frequencies, the residual of the upsampling process inside the generator gain. This trace is too subtle for the human eye, but it can be learned by neural networks. Remember also one important thing, reducing or compressing images can remove this trace.
Done!
Done We have now learned about deepfake and types of manipulation, creating a dataset of real faces FFHQ and fake style gain, splitting data in a stratified manner, and finding the gain trace in the frequency domain. In the next episode, we will detect and align faces using the UNET detector from OpenCA6. Currently working on a thesis, dissertation, or AI research in the field of deepfake detection and image forensics? Cerdas Inovasi Digital helps implement deep learning, from face detection, vision transformer, comparison with CNN, testing compression resistance, up to attention maps and web applications, as well as opening deep learning courses with Pytor. Want to master the entire series, from face dataset to deepfake checker application? All 6 episodes, complete with PDF step-by-step guides and source code, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy a single class with lifetime access. Subscribe now at the premium page cerdasinovasidigital.com Contact us via WhatsApp, or visit cerdasinovasidigital.com Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,281 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
