3. Activation heatmap3.1. Capturing activations with hooksThe model is loaded in BF16, the LoRA adapter is merged into the weights (merge_and_unload), and a forward_pre_hook is attached to the input of each mlp.down_proj. The input to down_proj is precisely the vector h:modules = [(name, m) for name, m in merged.named_modules() if name.endswith("mlp.down_proj")] def make_hook(name): def hook(module, args): hidden = args[0] # [batch, seq, 17408] flat = hidd