Shortcuts

Serve

These components aim to make it easier to interact with inference and serving tools such as torchserve.

torchx.components.serve.torchserve(model_path: str, management_api: str, image: str = 'ghcr.io/pytorch/torchx:0.8.0dev0', params: Optional[Dict[str, object]] = None, dryrun: bool = False) AppDef[source]

Deploys the provided model to the given torchserve management API endpoint.

>>> from torchx.components.serve import torchserve
>>> torchserve(
...     model_path="s3://your-bucket/your-model.pt",
...     management_api="http://torchserve:8081",
... )
AppDef(name='torchx-torchserve', ...)
Parameters:
  • model_path – The fsspec path to the model archive file.

  • management_api – The URL to the root of the torchserve management API.

  • image – Container to use.

  • params – torchserve parameters. See https://pytorch.org/serve/management_api.html#register-a-model

  • dryrun – Start the app, but does not perform actual work

Returns:

the TorchX application definition

Return type:

specs.AppDef

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources