Shortcuts

AdapterModule

class torchtune.modules.peft.AdapterModule(*args, **kwargs)[source]

Interface for an nn.Module containing adapter weights. Note that an adapter module does not have to explicitly implement this protocol, but it must define the adapter_params(self) method.

adapter_params() List[str][source]

Return a list of strings corresponding to the names of the nn.Parameters in the model coming from the adapter. E.g. if an nn.Module has adapter self.proj = nn.Linear(in_dim, out_dim), then adapter_params should return ['proj.weight', 'proj.bias'].

See LoRALinear’s adapter_params() for an example.

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources