torchtnt.utils.distributed.sync_bool

torchtnt.utils.distributed.sync_bool(val: bool, pg: Optional[ProcessGroup] = None, coherence_mode: Union[Literal['any', 'all', 'rank_zero'], int, float] = 'any') → bool

Utility to synchronize a boolean value across members of a provided process group.

In the case torch.distributed is not available or initialized, the input val is returned.

Parameters:

• val (bool) – boolean value to synchronize
• pg – process group to use for synchronization. If not specified, the default process group is used.
• Union[str (coherence_mode) – the manner in which the boolean value should be synchronized. 5 options are currently supported: 1. any (default): If any rank provides a True value, all ranks should receive True. 2. all: Only if all ranks provide a True value should all ranks receive True. 3. rank_zero: Makes rank 0 process’s value the source of truth and broadcasts the result to all other processes. 4. If an integer N is provided, return True only if at least N processes provide a True value. 5. If a float F is provided, return True only if at least this ratio of processes provide a True value. The ratio provided should be in the range [0, 1].
• int – the manner in which the boolean value should be synchronized. 5 options are currently supported: 1. any (default): If any rank provides a True value, all ranks should receive True. 2. all: Only if all ranks provide a True value should all ranks receive True. 3. rank_zero: Makes rank 0 process’s value the source of truth and broadcasts the result to all other processes. 4. If an integer N is provided, return True only if at least N processes provide a True value. 5. If a float F is provided, return True only if at least this ratio of processes provide a True value. The ratio provided should be in the range [0, 1].
• float] – the manner in which the boolean value should be synchronized. 5 options are currently supported: 1. any (default): If any rank provides a True value, all ranks should receive True. 2. all: Only if all ranks provide a True value should all ranks receive True. 3. rank_zero: Makes rank 0 process’s value the source of truth and broadcasts the result to all other processes. 4. If an integer N is provided, return True only if at least N processes provide a True value. 5. If a float F is provided, return True only if at least this ratio of processes provide a True value. The ratio provided should be in the range [0, 1].

Returns:

The synchronized boolean value.

Example:

>>> val = True
>>> # synced_val is True iff all ranks provide a True value to the function
>>> synced_val = sync_bool(val, coherence_mode="all")
>>> if synced_val:
>>>     print("success")