Template Class Int8Calibrator

• Defined in File ptq.h

Inheritance Relationships

Base Type

• private Algorithm

Class Documentation

template<typename Algorithm, typename DataLoaderUniquePtr>
class torch_tensorrt::ptq::Int8Calibrator : private Algorithm

Generic Int8Calibrator implementation based on a specified TensorRT calibration algorithm and a LibTorch DataLoader.

Template Parameters

• Algorithm, : – class nvinfer1::IInt8Calibrator (Default: nvinfer1::IInt8EntropyCalibrator2) - Algorithm to use

• DataLoaderUniquePtr, : – std::unique_ptr<torch::data::DataLoader> - DataLoader type

Public Functions

inline Int8Calibrator(DataLoaderUniquePtr dataloader, const std::string &cache_file_path, bool use_cache)

Construct a new Int8Calibrator object.

Using the provided DataLoader, construct a calibrator that can be used for PTQ with Torch-TensorRT

Parameters

• dataloader, : – std::unqiue_ptr<torch::data::DataLoader> - A unique pointer to the DataLoader, should be what is returned from the make_data_loader factory

• cache_file_path, : – const std::string& - A path to store / find the calibration cache

• use_cache – : bool - Whether to use the cache (if it exists)

inline int getBatchSize () const noexceptoverride

Get the Batch Size for the next batch (always 1 due to issues with TRT and explicit batch)

Returns

int

inline bool getBatch (void *bindings[], const char *names[], int nbBindings) noexceptoverride

Get the next Batch.

Parameters

• bindings, : – void*[] - An array of binding pointers (fed in from TensorRT calibrator), these buffers should be filed with batch data for each input

• names, : – const char*[] - Names of bindings

• nbBindings, : – int - Number of bindings

Returns

true - There is a new batch for the calibrator to consume

false - There is not a new batch for the calibrator to consume

inline const void * readCalibrationCache (size_t &length) noexceptoverride

Read calibration cache.

How to read from the calibration cache, only enabled if use_cache is set

Parameters

length –

Returns

const void* - Pointer to cache data

inline void writeCalibrationCache (const void *cache, size_t length) noexceptoverride

Write calibration cache.

Write a the calibration cache provided by TensorRT to a specified file

Parameters

• cache, : – const void* - cache data

• length, : – size_t - length of cache

inline operator nvinfer1::IInt8Calibrator*()

operator to cast to nvinfer1::IInt8Calibrator*

Convience function to convert to a IInt8Calibrator* to easily be assigned to the ptq_calibrator field in CompileSpec

Returns

nvinfer1::IInt8Calibrator*