Struct Device

• Defined in File torch_tensorrt.h

Nested Relationships

Nested Types

• Class Device::DeviceType

Struct Documentation

struct Device

Setting data structure for Target device.

Public Functions

inline Device()

Constructor for Device structure

Public Members

DeviceType device_type

Setting data structure for device This struct will hold Target device related parameters such as device_type, gpu_id, dla_core.

int64_t gpu_id

int64_t dla_core

bool allow_gpu_fallback

(Only used when targeting DLA (device)) Lets engine run layers on GPU if they are not supported on DLA

class DeviceType

Supported Device Types that can be used with TensorRT engines

This class is compatable with c10::DeviceTypes (but will check for TRT support) but the only applicable value is at::kCUDA, which maps to DeviceType::kGPU

To use the DataType class itself, interface using the enum vs. normal instatination

ex. torch_tensorrt::DeviceType type = DeviceType::kGPU;

Public Types

enum Value

Underlying enum class to support the DeviceType Class

In the case that you need to use the DeviceType class itself, interface using this enum vs. normal instatination

ex. torch_tensorrt::DeviceType type = DeviceType::kGPU;

Values:

enumerator kGPU

Target GPU to run engine.

enumerator kDLA

Target DLA to run engine.

Public Functions

DeviceType() = default

Construct a new Device Type object.

inline constexpr DeviceType(Value t)

Construct a new Device Type object from internal enum.

DeviceType(c10::DeviceType t)

Construct a new Device Type object from torch device enums Note: The only valid value is torch::kCUDA (torch::kCPU is not supported)

Parameters

t –

inline operator Value() const

Get the internal value from the Device object.

Returns

Value

explicit operator bool() = delete

inline constexpr bool operator==(DeviceType other) const

Comparison operator for DeviceType.

Parameters

other –

Returns

true

Returns

false

inline constexpr bool operator!=(DeviceType other) const

Comparison operator for DeviceType.

Parameters

other –

Returns

true

Returns

false