torch.use_deterministic_algorithms(mode, *, warn_only=False)[source]

Sets whether PyTorch operations must use “deterministic” algorithms. That is, algorithms which, given the same input, and when run on the same software and hardware, always produce the same output. When enabled, operations will use deterministic algorithms when available, and if only nondeterministic algorithms are available they will throw a RuntimeError when called.


This setting alone is not always enough to make an application reproducible. Refer to Reproducibility for more information.


torch.set_deterministic_debug_mode() offers an alternative interface for this feature.

The following normally-nondeterministic operations will act deterministically when mode=True:

The following normally-nondeterministic operations will throw a RuntimeError when mode=True:

In addition, several operations fill uninitialized memory when this setting is turned on and when torch.utils.deterministic.fill_uninitialized_memory is turned on. See the documentation for that attribute for more information.

A handful of CUDA operations are nondeterministic if the CUDA version is 10.2 or greater, unless the environment variable CUBLAS_WORKSPACE_CONFIG=:4096:8 or CUBLAS_WORKSPACE_CONFIG=:16:8 is set. See the CUDA documentation for more details: If one of these environment variable configurations is not set, a RuntimeError will be raised from these operations when called with CUDA tensors:

Note that deterministic operations tend to have worse performance than nondeterministic operations.


This flag does not detect or prevent nondeterministic behavior caused by calling an inplace operation on a tensor with an internal memory overlap or by giving such a tensor as the out argument for an operation. In these cases, multiple writes of different data may target a single memory location, and the order of writes is not guaranteed.


mode (bool) – If True, makes potentially nondeterministic operations switch to a deterministic algorithm or throw a runtime error. If False, allows nondeterministic operations.

Keyword Arguments

warn_only (bool, optional) – If True, operations that do not have a deterministic implementation will throw a warning instead of an error. Default: False


>>> torch.use_deterministic_algorithms(True)

# Forward mode nondeterministic error
>>> torch.randn(10, device='cuda').kthvalue(1)
RuntimeError: kthvalue CUDA does not have a deterministic implementation...

# Backward mode nondeterministic error
>>> torch.nn.AvgPool3d(1)(torch.randn(3, 4, 5, 6, requires_grad=True).cuda()).sum().backward()
RuntimeError: avg_pool3d_backward_cuda does not have a deterministic implementation...


Access comprehensive developer documentation for PyTorch

View Docs


Get in-depth tutorials for beginners and advanced developers

View Tutorials


Find development resources and get your questions answered

View Resources