Shortcuts

ts.torch_handler.request_envelope package

Submodules

ts.torch_handler.request_envelope.base module

Base class for all RequestEnvelope.

A request envelope reformats the inputs/outputs of a call to a handler. It translates from formats specific to a model orchestrator like Seldon or KServe to a set of flat Python items, and vice versa.

class ts.torch_handler.request_envelope.base.BaseEnvelope(handle_fn)[source]

Bases: abc.ABC

Interface for all envelopes. Derive from this class, replacing the abstract methods

abstract format_output(data)[source]
handle(data, context)[source]

The Input Requests and Response are handled here.

abstract parse_input(data)[source]

ts.torch_handler.request_envelope.body module

Default class for all handlers. Grabs the data out of the request body.

class ts.torch_handler.request_envelope.body.BodyEnvelope(handle_fn)[source]

Bases: ts.torch_handler.request_envelope.base.BaseEnvelope

Gets the key “body” from the input data, returns a raw list

format_output(data)[source]
parse_input(data)[source]

ts.torch_handler.request_envelope.json module

Uses JSON formatted inputs/outputs, following the structure outlined if __name__ == ‘__main__’: https://www.tensorflow.org/tfx/serving/api_restmain()

class ts.torch_handler.request_envelope.json.JSONEnvelope(handle_fn)[source]

Bases: ts.torch_handler.request_envelope.base.BaseEnvelope

Implementation. Captures batches in JSON format, returns also in JSON format.

format_output(data)[source]
parse_input(data)[source]

ts.torch_handler.request_envelope.kserve module

The KServe Envelope is used to handle the KServe Input Request inside Torchserve.

class ts.torch_handler.request_envelope.kserve.KServeEnvelope(handle_fn)[source]

Bases: ts.torch_handler.request_envelope.base.BaseEnvelope

This function is used to handle the input request specified in kserve format and converts it into a Torchserve readable format.

Parameters

Format (data - List of Input Request in kserve) –

Returns

Returns the list of the Input Request in Torchserve Format

Return type

[list]

format_output(data)[source]

Returns the prediction response and captum explanation response of the input request.

Parameters

outputs (List) – The outputs arguments is in the form of a list of dictionaries.

Returns

The response is returned as a list of predictions and explanations

Return type

(list)

parse_input(data)[source]

ts.torch_handler.request_envelope.kservev2 module

Module contents

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources