Shortcuts

ts.model_service package

Submodules

ts.model_service.model_service module

ModelService defines an API for base model service.

class ts.model_service.model_service.ModelService(model_name, model_dir, manifest, gpu=None)[source]

Bases: object

ModelService wraps up all preprocessing, inference and postprocessing functions used by model service. It is defined in a flexible manner to be easily extended to support different frameworks.

handle(data, context)[source]

Backward compatible handle function.

Parameters:
  • data

  • context

Returns:

abstract inference(data)[source]

Wrapper function to run pre-process, inference and post-process functions.

Parameters:

data (list of object) – Raw input from request.

Returns:

data to be sent back

Return type:

list of outputs to be sent back to client.

initialize(context)[source]

Internal initialize ModelService.

Parameters:

context – MMS context object

Returns:

abstract ping()[source]

Ping to get system’s health.

Returns:

A message, “health”: “healthy!”, to show system is healthy.

Return type:

String

signature()[source]

Signature for model service.

Returns:

Model service signature.

Return type:

Dict

class ts.model_service.model_service.SingleNodeService(model_name, model_dir, manifest, gpu=None)[source]

Bases: ModelService

SingleNodeModel defines abstraction for model service which loads a single model.

inference(data)[source]

Wrapper function to run preprocess, inference and postprocess functions.

Parameters:

data (list of object) – Raw input from request.

Returns:

data to be sent back

Return type:

list of outputs to be sent back to client.

Module contents

Model services code

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources