ts.model_service package

Submodules

ts.model_service.model_service module

ModelService defines an API for base model service.

class ts.model_service.model_service.ModelService(model_name, model_dir, manifest, gpu=None)

Bases: object

ModelService wraps up all preprocessing, inference and postprocessing functions used by model service. It is defined in a flexible manner to be easily extended to support different frameworks.

handle(data, context)

Backward compatible handle function.

Parameters

• data –

• context –

Returns

abstract inference(data)

Wrapper function to run pre-process, inference and post-process functions.

Parameters

data (list of object) – Raw input from request.

Returns

data to be sent back

Return type

list of outputs to be sent back to client.

initialize(context)

Internal initialize ModelService.

Parameters

context – MMS context object

Returns

abstract ping()

Ping to get system’s health.

Returns

A message, “health”: “healthy!”, to show system is healthy.

Return type

String

signature()

Signature for model service.

Returns

Model service signature.

Return type

Dict

class ts.model_service.model_service.SingleNodeService(model_name, model_dir, manifest, gpu=None)

Bases: ts.model_service.model_service.ModelService

SingleNodeModel defines abstraction for model service which loads a single model.

inference(data)

Wrapper function to run preprocess, inference and postprocess functions.

Parameters

data (list of object) – Raw input from request.

Returns

data to be sent back

Return type

list of outputs to be sent back to client.

Module contents

Model services code