Template Class Int8Calibrator
Defined in File ptq.h
Inheritance Relationships
Base Type
private Algorithm
Class Documentation
-
template<typename Algorithm, typename DataLoaderUniquePtr>
class Int8Calibrator : private Algorithm Generic Int8Calibrator implementation based on a specified TensorRT calibration algorithm and a LibTorch DataLoader.
- Template Parameters
Algorithm – class nvinfer1::IInt8Calibrator (Default: nvinfer1::IInt8EntropyCalibrator2) - Algorithm to use
DataLoaderUniquePtr – std::unique_ptr<torch::data::DataLoader> - DataLoader type
Public Functions
-
inline Int8Calibrator(DataLoaderUniquePtr dataloader, const std::string &cache_file_path, bool use_cache)
Construct a new Int8Calibrator object.
Using the provided DataLoader, construct a calibrator that can be used for PTQ with Torch-TensorRT
- Parameters
dataloader – std::unqiue_ptr<torch::data::DataLoader> - A unique pointer to the DataLoader, should be what is returned from the make_data_loader factory
cache_file_path – const std::string& - A path to store / find the calibration cache
use_cache – : bool - Whether to use the cache (if it exists)
-
inline int getBatchSize() const noexcept override
Get the Batch Size for the next batch (always 1 due to issues with TRT and explicit batch)
- Returns
int
-
inline bool getBatch(void *bindings[], const char *names[], int nbBindings) noexcept override
Get the next Batch.
- Parameters
bindings – void*[] - An array of binding pointers (fed in from TensorRT calibrator), these buffers should be filed with batch data for each input
names – const char*[] - Names of bindings
nbBindings – int - Number of bindings
- Returns
true - There is a new batch for the calibrator to consume
- Returns
false - There is not a new batch for the calibrator to consume
-
inline const void *readCalibrationCache(size_t &length) noexcept override
Read calibration cache.
How to read from the calibration cache, only enabled if use_cache is set
- Parameters
length –
- Returns
const void* - Pointer to cache data