torch.use_deterministic_algorithms¶

torch.
use_deterministic_algorithms
(d)[source]¶ Sets whether PyTorch operations must use “deterministic” algorithms. That is, algorithms which, given the same input, and when run on the same software and hardware, always produce the same output. When True, operations will use deterministic algorithms when available, and if only nondeterministic algorithms are available they will throw a :class:RuntimeError when called.
Warning
This feature is in beta, and its design and implementation may change in the future.
The following normallynondeterministic operations will act deterministically when d=True:
torch.nn.Conv1d
when called on CUDA tensortorch.nn.Conv2d
when called on CUDA tensortorch.nn.Conv3d
when called on CUDA tensortorch.nn.ConvTranspose1d
when called on CUDA tensortorch.nn.ConvTranspose2d
when called on CUDA tensortorch.nn.ConvTranspose3d
when called on CUDA tensortorch.bmm()
when called on sparsedense CUDA tensorstorch.__getitem__()
backward when self is a CPU tensor andindices
is a list of tensorstorch.index_put()
withaccumulate=True
when called on a CPU tensor
The following normallynondeterministic operations will throw a
RuntimeError
when d=True:torch.nn.AvgPool3d
when called on a CUDA tensor that requires gradtorch.nn.AdaptiveAvgPool2d
when called on a CUDA tensor that requires gradtorch.nn.AdaptiveAvgPool3d
when called on a CUDA tensor that requires gradtorch.nn.MaxPool3d
when called on a CUDA tensor that requires gradtorch.nn.AdaptiveMaxPool2d
when called on a CUDA tensor that requires gradtorch.nn.FractionalMaxPool2d
when called on a CUDA tensor that requires gradtorch.nn.FractionalMaxPool3d
when called on a CUDA tensor that requires gradtorch.nn.functional.interpolate()
when called on a CUDA tensor that requires grad and one of the following modes is used:linear
bilinear
bicubic
trilinear
torch.nn.ReflectionPad1d
when called on a CUDA tensor that requires gradtorch.nn.ReflectionPad2d
when called on a CUDA tensor that requires gradtorch.nn.ReplicationPad1d
when called on a CUDA tensor that requires gradtorch.nn.ReplicationPad2d
when called on a CUDA tensor that requires gradtorch.nn.ReplicationPad3d
when called on a CUDA tensor that requires gradtorch.nn.NLLLoss
when called on a CUDA tensor that requires gradtorch.nn.CTCLoss
when called on a CUDA tensor that requires gradtorch.nn.EmbeddingBag
when called on a CUDA tensor that requires gradtorch.scatter_add_()
when called on a CUDA tensortorch.index_add_()
when called on a CUDA tensortorch.index_copy()
torch.index_select()
when called on a CUDA tensor that requires gradtorch.repeat_interleave()
when called on a CUDA tensor that requires gradtorch.histc()
when called on a CUDA tensortorch.bincount()
when called on a CUDA tensortorch.kthvalue()
with called on a CUDA tensortorch.median()
with indices output when called on a CUDA tensor
A handful of CUDA operations are nondeterministic if the CUDA version is 10.2 or greater, unless the environment variable CUBLAS_WORKSPACE_CONFIG=:4096:8 or CUBLAS_WORKSPACE_CONFIG=:16:8 is set. See the CUDA documentation for more details: https://docs.nvidia.com/cuda/cublas/index.html#cublasApi_reproducibility If one of these environment variable configurations is not set, a
RuntimeError
will be raised from these operations when called with CUDA tensors:Note that deterministic operations tend to have worse performance than nondeterministic operations.
 Parameters
d (
bool
) – If True, force operations to be deterministic. If False, allow nondeterministic operations.