format_content_with_images¶
- torchtune.data.format_content_with_images(content: str, *, image_tag: str, images: List[PIL.Image.Image]) List[Dict[str, Any]] [source]¶
Given a raw text string, split by the specified
image_tag
and form into list of dictionaries to be used in theMessage
content field:[ { "role": "system" | "user" | "assistant", "content": [ {"type": "image", "content": <PIL.Image.Image>}, {"type": "text", "content": "This is a sample image."}, ], }, ... ]
- Parameters:
- Raises:
ValueError – If the number of images does not match the number of image tags in the content
Examples
>>> content = format_content_with_images( ... "<|image|>hello <|image|>world", ... image_tag="<|image|>", ... images=[<PIL.Image.Image>, <PIL.Image.Image>] ... ) >>> print(content) [ {"type": "image", "content": <PIL.Image.Image>}, {"type": "text", "content": "hello "}, {"type": "image", "content": <PIL.Image.Image>}, {"type": "text", "content": "world"} ]