.. role:: hidden :class: hidden-section .. currentmodule:: torchtune.modules.transforms.tokenizers TikTokenBaseTokenizer ===================== .. autoclass:: TikTokenBaseTokenizer :members: