Template Class Int8CacheCalibrator¶
Defined in File ptq.h
Inheritance Relationships¶
Base Type¶
private Algorithm
Class Documentation¶
-
template<typename Algorithm>
class Int8CacheCalibrator : private Algorithm¶ Generic Int8Calibrator implementation based on a specified TensorRT calibration algorithm that only reads from a calibration file.
- Template Parameters
Algorithm – class nvinfer1::IInt8Calibrator (Default: nvinfer1::IInt8EntropyCalibrator2) - Algorithm to use
Public Functions
-
inline Int8CacheCalibrator(const std::string &cache_file_path)¶
Construct a new Int 8 Cache Calibrator object.
- Parameters
cache_file_path –
-
inline int getBatchSize() const noexcept override¶
Get the Batch Size for the next batch (always 1 due to issues with TRT and explicit batch)
- Returns
int
-
inline bool getBatch(void *bindings[], const char *names[], int nbBindings) noexcept override¶
Get the next Batch.
Not used always returns false
- Parameters
bindings – void*[] - An array of binding pointers (fed in from TensorRT calibrator), these buffers should be filed with batch data for each input
names – const char*[] - Names of bindings
nbBindings – int - Number of bindings
- Returns
false
-
inline const void *readCalibrationCache(size_t &length) noexcept override¶
Read calibration cache.
How to read from the calibration cache, only enabled if use_cache is set
- Parameters
length –
- Returns
const void* - Pointer to cache data
-
inline void writeCalibrationCache(const void *cache, size_t length) noexcept override¶
Write calibration cache.
Write a the calibration cache provided by TensorRT to a specified file
- Parameters
cache – const void* - cache data
length – size_t - length of cache
-
inline operator nvinfer1::IInt8Calibrator*()¶
operator to cast to nvinfer1::IInt8Calibrator*
Convience function to convert to a IInt8Calibrator* to easily be assigned to the ptq_calibrator field in CompileSpec
- Returns
nvinfer1::IInt8Calibrator*
