def _set_lora_trainable(self):
        """Set only the lora parameters trainable."""
        for name, param in self.named_parameters():
            if '.lora_' in name:
                param.requires_grad = True
            else:
                param.requires_grad = False
Set only the lora parameters trainable.
mmpretrain/models/peft/lora.py