qwen2_5_1_5b_base

torchtune.models.qwen2_5.qwen2_5_1_5b_base() → TransformerDecoder

Builder for creating a Qwen2.5 base model initialized w/ the default 1.5B parameter values from https://huggingface.co/Qwen/Qwen2.5-1.5B

Returns:

Instantiation of Qwen2.5 1.5B model

Return type:

TransformerDecoder

Note

The base and instruct versions have slightly different architectures for all Qwen2.5 model sizes except 0.5B and 3B. Make sure to select the correct model builder for the weights.

Note

Qwen2.5 0.5B-3B model builders will enable tie_word_embeddings by default (see qwen2()).