1. Gradient Clipping – a critically important component.
The most contrasting result was obtained without gradient norm clipping (no_grad_clip). The Directional CLIP Loss graph (Fig. 1) shows that without Gradient Clipping, the error remains high (0.78–0.85) with no steady decrease, whereas in the baseline it drops to 0.66–0.72.
StyleGAN2 has an enormous number of parameters, and the gradients can “explode” when computing the CLIP loss. Gradient Clipping limits their norm, preventing the weights from drifting into a random state. Without it, training becomes unstable, and the generator cannot learn the direction of the text.
2. Adaptive layer unfreezing – a compromise between the target and the original image.
The no_adaptive configuration (without adaptive layer selection) shows the best result of all on the Directional CLIP Loss graph (pink line, Fig. 1): the error drops to 0.62–0.69.
However, when looking at the LPIPS Stability graph (Fig. 2), no_adaptive has the highest LPIPS values (0.37–0.39), whereas the baseline values remain at 0.30–0.34.
Training all layers allows the generator to better match the text prompt (werewolf), but severely distorts the original facial structure. Adaptive unfreezing (selecting only those layers that change the most in W+) preserves human identity and prevents the generator from drifting entirely into the target domain.This is a trade-off between the magnitude of the changes and stability.
It is also worth noting that the no_grad_clip configuration has the lowest LPIPS values. However, this does not indicate high quality—the generator simply failed to adapt properly.
3. Impact of regularizations (L2, Patch, LPIPS).
If we look at the overall Directional CLIP Loss graph (Fig. 1), the no_l2, no_patch, and no_lpips configurations track the baseline almost exactly. At first glance, they do not provide a significant improvement in the target metric. However, their contribution is evident in the details:
L2 regularization (Mean grad norms per block graph, Fig. 3): without it, the gradients in the generator’s later layers (b1024) are noticeably higher (0.43 versus 0.36 for the baseline). L2 constrains weight drift and makes training more stable.
LPIPS regularization: without it, stronger artifacts appear in the generated images, and the LPIPS Stability value increases.
Patch-based loss: without it, local structures (hair, etc.) become more blurred.
4. CLIP ensemble and multi-prompts.
Disabling the second CLIP model (no_clip) produces results very close to the baseline. Disabling multi-prompts (no_prompt) also does not significantly affect the quality of training and generation. This confirms that these components improve stability but are not critical to the functioning of the method.
5. Gradient analysis by block.
The Mean grad norms per block graph (Fig. 3) shows that, when using the baseline, the largest gradients are observed in the later blocks (b512, b1024).This is logical because low-level blocks are responsible for geometry and texture, while high-level blocks are responsible for details (in our case, the appearance of wolf features). In the no_adaptive configuration, the gradients in the early blocks (b4, b8) are several times higher, confirming that without adaptive selection, all layers are trained, including those that should not change.
The Diversity values for all configurations remain stable (0.60–0.63), indicating that the generator does not lose diversity during adaptation.
Figures 4 and 5 present a visual comparison of the generated images at the 300th epoch. It can be seen that the baseline preserves the facial structure but adds wolf features. The no_adaptive configuration distorts the face more strongly, while no_grad_clip demonstrates weak adaptation. The absence of Patch-based loss and L2 and LPIPS regularization leads to slightly more artifacts and “noise.” Disabling the second CLIP model and multiprompting also does not significantly affect the quality of generation.
For clarity, a summary table with the key metrics is provided below. The 100th epoch (the minimum Directional CLIP Loss for most configurations) and the 300th epoch (the completion of training) were selected as comparison points.
Conclusion
In this study, I implemented the StyleGAN-NADA method and conducted a comprehensive ablation analysis of its components.The results clearly demonstrated that the quality and stability of text adaptation in generative models depend not only on the underlying architecture, but also on the proper configuration of auxiliary mechanisms.
Key findings:
Gradient Clipping is an essential component. Without gradient norm clipping, training becomes unstable: the Directional CLIP Loss error does not decrease, and the generator barely adapts to the target domain. This is visually confirmed by the generated images remaining almost unchanged.
Adaptive layer unfreezing is a control mechanism. It reduces the strength of text adaptation in order to preserve the original image structure. This is a trade-off between text alignment and preserving human identity.
Regularization methods (L2, Patch, LPIPS) are critical for visual quality. Although they do not yield a significant improvement in the target CLIP metric, their absence leads to more artifacts. L2 regularization also further restrains weight drift in the generator’s later layers, making training more stable.
The CLIP ensemble and multi-prompts improve stability but are not critical. Disabling them does not cause a sharp decline in quality; however, they make training somewhat more robust.
The optimal stopping point is around the 100th epoch.Further training leads to a plateau or even degradation in the metrics.
The project showed that properly tuning the auxiliary mechanisms and understanding the gradients play a major role in the final result.
The project code and full experimental results are available in the StyleGAN-NADA-reimplementation repository on GitHub.
References
Original paper by researchers from Tel Aviv University and NVIDIA Research: StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators.
Authors’ repository: github.com/rinongal/StyleGAN-nada.
Original CLIP paper: Learning Transferable Visual Models From Natural Language Supervision.
CLIP repository: github.com/openai/CLIP.
Original StyleGAN paper: A Style-Based Generator Architecture for Generative Adversarial Networks.
StyleGAN repository: github.com/NVlabs/stylegan.
Original StyleGAN2 paper: Analyzing and Improving the Image Quality of StyleGAN.
StyleGAN2 repository: github.com/NVlabs/stylegan2.
StyleGAN-NADA-reimplementation repository: github.com/KBaron34/StyleGAN-NADA-reimplementation.
Literature review of the paper StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generators: habr.com/ru/articles/943034.
