Picked from the software, topic, and level of “Face Occlusion Robust Recognition Introduction #3: Multi-Scale Feature Extraction with Feature Pyramid Network (FPN)”.
At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
Here is the final result, one face viewed at 4 scales simultaneously, from a 28-pixel feature map capturing fine details such as eyes and skin texture, up to a 4-pixel map capturing the global meaning of the face. This is the Feature Pyramid Network we built from scratch on top of ResNet 18. Let's start. Hello, welcome to episode three of the Occlusion-Tolerant Face Recognition series. A standard CNN only uses features from the last layer, which has low resolution but rich semantics. Fine details from the early layers are discarded. The Feature Pyramid Network combines both, each scale gets meaning from the deeper layers as well as detail from the shallow layers. This idea was originally for object detection, and today we are using it for faces.
Step 1: Backbone ResNet18 & lateral connections
Create file fpn.py Backbone is ResNet 18 with Pretrained ImageNet weights. We split into stem, which is the first convolution and Max Pooling, then 4 stages, C2 to C5. Each stage reduces resolution by 2 times and increases channels, 64, 123, 256, and 512. Lateral connections are 1x1 convolutions that equalize all stages to 128 channels, and 3x3 convolutions to smooth the combined results.
Step 2: Bottom-up & top-down paths
The forward function runs two paths. The bottom-up path goes through the stem and C2 to C5 like a standard CNN. The top-down path starts from C5, the most semantically rich layer. The result is upscaled with bilinear interpolation to the size of the stage below, then added to the lateral version of that stage. This continues until C2. The final result is four feature maps, P2 to P5, each smoothed with a 3x3 convolution.
Step 3: Size of each pyramid level
Create file check underscore fpn.py We take one face from the already aligned dataset, normalize to -1 to 1, and run fpn. Run. P2 is 28x28, one pixel represents 4 original pixels. P3 is 14, P4 is 7, and P5 is 4x4. All levels have 128 channels, making them easy to combine. In total, about 11.9 million parameters, almost all from ResNet 18.
Step 4: Feature map visualization of P2-P5
Let's look at the feature map by averaging the absolute values of all channels. P2 still shows the face shape with detailed eyes, eyebrows, and nose. As we go to P5, the map becomes coarser and more abstract, representing the overall face context. For a face covered by a mask, the detailed eyes at the fine scale are still available even though the lower part of the face is missing. That's why we want to combine all scales, which we will do in the next episode.
Done!
Done. We have built the feature pyramid network on top of the ResNet 18 Pretrained model, understood the bottom-up path, lateral connections, and top-down path, as well as viewed the feature maps at 4 scales. In the next episode, we will combine the 4 scales into a face embedding and train it with the art face loss. Currently working on a thesis, dissertation, or AI research in the field of face recognition and biometrics? Cerdas Inovasi Digital helps with deep learning implementation, from synthetic occlusion, face alignment, feature pyramid, art face, verification, and ROC, up to face attendance applications, as well as opening deep learning courses with PyTor. Want to master the entire series, from face datasets to attendance applications? All 6 episodes, complete with PDF step-by-step guides and source code, are available to premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy a single class with lifetime access. Subscribe now at the premium page cerdasinovasidigital.com Contact our WhatsApp or visit cerdasinovasidigital.com Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,290 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
