Shortcuts

ts.metrics package

Submodules

ts.metrics.dimension module

Dimension class for model server metrics

class ts.metrics.dimension.Dimension(name, value)[source]

Bases: object

Dimension class defining key value pair

to_dict()[source]

return an dictionary

ts.metrics.metric module

Metric class for model server

class ts.metrics.metric.Metric(name, value, unit, dimensions, request_id=None, metric_method=None)[source]

Bases: object

Class for generating metrics and printing it to stdout of the worker

reset()[source]

Reset Metric value to 0

to_dict()[source]

return an Ordered Dictionary

update(value)[source]

Update function for Metric class

Parameters:

value (int, float) – metric to be updated

ts.metrics.metric_collector module

Single start point for system metrics and process metrics script

ts.metrics.metric_encoder module

Metric Encoder class for json dumps

class ts.metrics.metric_encoder.MetricEncoder(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, sort_keys=False, indent=None, separators=None, default=None)[source]

Bases: JSONEncoder

Encoder class for json encoding Metric Object

default(obj)[source]

Override only when object is of type Metric

ts.metrics.metrics_store module

Metrics collection module

class ts.metrics.metrics_store.MetricsStore(request_ids, model_name)[source]

Bases: object

DEPRECATED Class for creating, modifying different metrics. And keep them in a dictionary

add_counter(name, value, idx=None, dimensions=None)[source]

Add a counter metric or increment an existing counter metric

Parameters:
  • name (str) – metric name

  • value (int) – value of metric

  • idx (int) – request_id index in batch

  • dimensions (list) – list of dimensions for the metric

add_error(name, value, dimensions=None)[source]

Add a Error Metric :param name: metric name :type name: str :param value: value of metric, in this case a str :type value: str :param dimensions: list of dimensions for the metric :type dimensions: list

add_metric(name, value, unit, idx=None, dimensions=None)[source]

Add a metric which is generic with custom metrics

Parameters:
  • name (str) – metric name

  • value (int, float) – value of metric

  • idx (int) – request_id index in batch

  • unit (str) – unit of metric

  • dimensions (list) – list of dimensions for the metric

add_percent(name, value, idx=None, dimensions=None)[source]

Add a percentage based metric

Parameters:
  • name (str) – metric name

  • value (int, float) – value of metric

  • idx (int) – request_id index in batch

  • dimensions (list) – list of dimensions for the metric

add_size(name, value, idx=None, unit='MB', dimensions=None)[source]

Add a size based metric

Parameters:
  • name (str) – metric name

  • value (int, float) – value of metric

  • idx (int) – request_id index in batch

  • unit (str) – unit of metric, default here is ‘MB’, ‘kB’, ‘GB’ also supported

  • dimensions (list) – list of dimensions for the metric

add_time(name, value, idx=None, unit='ms', dimensions=None)[source]

Add a time based metric like latency, default unit is ‘ms’

Parameters:
  • name (str) – metric name

  • value (int) – value of metric

  • idx (int) – request_id index in batch

  • unit (str) – unit of metric, default here is ms, s is also accepted

  • dimensions (list) – list of dimensions for the metric

ts.metrics.process_memory_metric module

Collect process memory usage metrics here Pass a json, collection of pids and gpuID

ts.metrics.process_memory_metric.check_process_mem_usage(stdin)[source]
Returns:

mem_utilization

Return type:

float

ts.metrics.process_memory_metric.get_cpu_usage(pid)[source]

use psutil for cpu memory :param pid: str :return: int

ts.metrics.system_metrics module

Module to collect system metrics for front-end

ts.metrics.system_metrics.collect_all(mod, num_of_gpu)[source]

Collect all system metrics.

Parameters:
  • mod

  • num_of_gpu

Returns:

ts.metrics.system_metrics.cpu_utilization()[source]
ts.metrics.system_metrics.disk_available()[source]
ts.metrics.system_metrics.disk_used()[source]
ts.metrics.system_metrics.disk_utilization()[source]
ts.metrics.system_metrics.gpu_utilization(num_of_gpu)[source]

Collect gpu metrics.

Parameters:

num_of_gpu

Returns:

ts.metrics.system_metrics.memory_available()[source]
ts.metrics.system_metrics.memory_used()[source]
ts.metrics.system_metrics.memory_utilization()[source]

ts.metrics.unit module

Module to define Unit mappings

class ts.metrics.unit.Units[source]

Bases: object

Define a unit of elements

Module contents

This is a folder for all python worker metrics.

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources