AdapterModule¶
- class torchtune.modules.peft.AdapterModule(*args, **kwargs)[source]¶
Interface for an nn.Module containing adapter weights. Note that an adapter module does not have to explicitly implement this protocol, but it must define the
adapter_params(self)
method.- adapter_params() List[str] [source]¶
Return a list of strings corresponding to the names of the nn.Parameters in the model coming from the adapter. E.g. if an nn.Module has adapter
self.proj = nn.Linear(in_dim, out_dim)
, then adapter_params should return['proj.weight', 'proj.bias']
.See LoRALinear’s
adapter_params()
for an example.