Struct Device¶

Defined in File torch_tensorrt.h

Nested Relationships¶

Nested Types¶

Class Device::DeviceType

Struct Documentation¶

struct Device¶

Setting data structure for Target device.

Public Functions

inline Device()¶: Constructor for Device structure

Public Members

DeviceType device_type¶: Setting data structure for device This struct will hold Target device related parameters such as device_type, gpu_id, dla_core.

int64_t gpu_id¶

int64_t dla_core¶

bool allow_gpu_fallback¶: (Only used when targeting DLA (device)) Lets engine run layers on GPU if they are not supported on DLA

class DeviceType

Supported Device Types that can be used with TensorRT engines

This class is compatable with c10::DeviceTypes (but will check for TRT support) but the only applicable value is at::kCUDA, which maps to DeviceType::kGPU

To use the DataType class itself, interface using the enum vs. normal instatination

ex. torch_tensorrt::DeviceType type = DeviceType::kGPU;

Public Types

enum Value

Underlying enum class to support the DeviceType Class

In the case that you need to use the DeviceType class itself, interface using this enum vs. normal instatination

ex. torch_tensorrt::DeviceType type = DeviceType::kGPU;

Values:

enumerator kGPU: Target GPU to run engine.

enumerator kDLA: Target DLA to run engine.

Public Functions

DeviceType() = default: Construct a new Device Type object.

inline constexpr DeviceType(Value t): Construct a new Device Type object from internal enum.

DeviceType(c10::DeviceType t)

Construct a new Device Type object from torch device enums Note: The only valid value is torch::kCUDA (torch::kCPU is not supported)

Parameters: t –

inline operator Value() const

Get the internal value from the Device object.

Returns: Value

explicit operator bool() = delete

inline constexpr bool operator==(DeviceType other) const

Comparison operator for DeviceType.

Parameters

other –

Returns

true

false

inline constexpr bool operator!=(DeviceType other) const