Shortcuts

ts.model_service package

Submodules

ts.model_service.model_service module

ModelService defines an API for base model service.

class ts.model_service.model_service.ModelService(model_name, model_dir, manifest, gpu=None)[source]

Bases: object

ModelService wraps up all preprocessing, inference and postprocessing functions used by model service. It is defined in a flexible manner to be easily extended to support different frameworks.

handle(data, context)[source]

Backward compatible handle function.

Parameters
  • data

  • context

Returns

abstract inference(data)[source]

Wrapper function to run pre-process, inference and post-process functions.

Parameters

data (list of object) – Raw input from request.

Returns

data to be sent back

Return type

list of outputs to be sent back to client.

initialize(context)[source]

Internal initialize ModelService.

Parameters

context – MMS context object

Returns

abstract ping()[source]

Ping to get system’s health.

Returns

A message, “health”: “healthy!”, to show system is healthy.

Return type

String

signature()[source]

Signature for model service.

Returns

Model service signature.

Return type

Dict

class ts.model_service.model_service.SingleNodeService(model_name, model_dir, manifest, gpu=None)[source]

Bases: ts.model_service.model_service.ModelService

SingleNodeModel defines abstraction for model service which loads a single model.

inference(data)[source]

Wrapper function to run preprocess, inference and postprocess functions.

Parameters

data (list of object) – Raw input from request.

Returns

data to be sent back

Return type

list of outputs to be sent back to client.

Module contents

Model services code

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources