Shortcuts

Template Class Int8CacheCalibrator

Inheritance Relationships

Base Type

  • private Algorithm

Class Documentation

template<typename Algorithm>
class torch_tensorrt::ptq::Int8CacheCalibrator : private Algorithm

Generic Int8Calibrator implementation based on a specified TensorRT calibration algorithm that only reads from a calibration file.

Template Parameters

Algorithm – class nvinfer1::IInt8Calibrator (Default: nvinfer1::IInt8EntropyCalibrator2) - Algorithm to use

Public Functions

inline Int8CacheCalibrator(const std::string &cache_file_path)

Construct a new Int 8 Cache Calibrator object.

Parameters

cache_file_path

inline int getBatchSize() const noexcept override

Get the Batch Size for the next batch (always 1 due to issues with TRT and explicit batch)

Returns

int

inline bool getBatch(void *bindings[], const char *names[], int nbBindings) noexcept override

Get the next Batch.

Not used always returns false

Parameters
  • bindings – void*[] - An array of binding pointers (fed in from TensorRT calibrator), these buffers should be filed with batch data for each input

  • names – const char*[] - Names of bindings

  • nbBindings – int - Number of bindings

Returns

false

inline const void *readCalibrationCache(size_t &length) noexcept override

Read calibration cache.

How to read from the calibration cache, only enabled if use_cache is set

Parameters

length

Returns

const void* - Pointer to cache data

inline void writeCalibrationCache(const void *cache, size_t length) noexcept override

Write calibration cache.

Write a the calibration cache provided by TensorRT to a specified file

Parameters
  • cache – const void* - cache data

  • length – size_t - length of cache

inline operator nvinfer1::IInt8Calibrator*()

operator to cast to nvinfer1::IInt8Calibrator*

Convience function to convert to a IInt8Calibrator* to easily be assigned to the ptq_calibrator field in CompileSpec

Returns

nvinfer1::IInt8Calibrator*

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources