ts package



ts.arg_parser module

This module parses the arguments given through the torchserve command-line. This is used by model-server at runtime.

class ts.arg_parser.ArgParser[source]

Bases: object

Argument parser for torchserve and torchserve-export commands TODO : Add readme url

static extract_args(args=None)[source]
static model_service_worker_args()[source]

ArgParser for backend worker. Takes the socket name and socket type. :return:

static ts_parser()[source]

Argument parser for torchserve start service

ts.context module

Context object of incoming request

class ts.context.Context(model_name, model_dir, manifest, batch_size, gpu, mms_version, limit_max_image_pixels=True, metrics=None, model_yaml_config=None)[source]

Bases: object

Context stores model relevant worker information Some fixed during load times and some

get_all_request_header(idx: int)Dict[str, str][source]
get_request_header(idx: int, key: str)Optional[str][source]
get_request_id(idx: int = 0)Optional[str][source]
get_response_content_type(idx: int)Optional[str][source]
get_response_headers(idx: int)Dict[str, str][source]
get_response_status(idx: int)Tuple[int, str][source]
property metrics
property request_processor
set_all_response_status(code: int = 200, phrase: str = '')None[source]

Set the status code of individual requests :param phrase: :param code: :return:

set_response_content_type(idx: int, value: str)None[source]
set_response_header(idx, key, value)[source]
set_response_status(code: int = 200, phrase: str = '', idx: int = 0)[source]

Set the status code of individual requests :param phrase: :param idx: The index data in the list(data) that is sent to the handle() method :param code: :return:

property system_properties
class ts.context.RequestProcessor(request_header: dict)[source]

Bases: object

Request processor

add_response_property(key: str, value: str)None[source]
get_request_property(key: str)Optional[str][source]
get_response_header(key: str)Optional[str][source]
report_status(code, reason_phrase=None)None[source]

ts.model_loader module

ts.model_server module

File to define the entry point to Model Server


Read properties file into map.


This is the entry point for model server :return:

ts.model_service_worker module

ts.service module

ts.version module

This is the current version of TorchServe

Module contents

This module does the following: a. Starts model-server. b. Creates end-points based on the configured models. c. Exposes standard “ping” and “api-description” endpoints. d. Waits for servicing inference requests.


Access comprehensive developer documentation for PyTorch

View Docs


Get in-depth tutorials for beginners and advanced developers

View Tutorials


Find development resources and get your questions answered

View Resources