torchtext.utils

reporthook

torchtext.utils.reporthook(t)

https://github.com/tqdm/tqdm.

download_from_url

torchtext.utils.download_from_url(url, path=None, root='.data', overwrite=False, hash_value=None, hash_type='sha256')

Download file, with logic (from tensor2tensor) for Google Drive. Returns the path to the downloaded file. :param url: the url of the file from URL header. (None) :param path: path where file will be saved :param root: download folder used to store the file in (.data) :param overwrite: overwrite existing files (False) :param hash_value: hash for url (Default: None). :type hash_value: str, optional :param hash_type: hash type, among “sha256” and “md5” (Default: "sha256"). :type hash_type: str, optional

Examples

>>> url = 'http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/validation.tar.gz'
>>> torchtext.utils.download_from_url(url)
>>> url = 'http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/validation.tar.gz'
>>> torchtext.utils.download_from_url(url)
>>> '.data/validation.tar.gz'

extract_archive

torchtext.utils.extract_archive(from_path, to_path=None, overwrite=False)

Extract archive. :param from_path: the path of the archive. :param to_path: the root path of the extracted files (directory of from_path) :param overwrite: overwrite existing files (False)

Returns:

List of paths to extracted files even if not overwritten.

Examples

>>> url = 'http://www.quest.dcs.shef.ac.uk/wmt16_files_mmt/validation.tar.gz'
>>> from_path = './validation.tar.gz'
>>> to_path = './'
>>> torchtext.utils.download_from_url(url, from_path)
>>> torchtext.utils.extract_archive(from_path, to_path)
>>> ['.data/val.de', '.data/val.en']
>>> torchtext.utils.download_from_url(url, from_path)
>>> torchtext.utils.extract_archive(from_path, to_path)
>>> ['.data/val.de', '.data/val.en']