.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "prototype/maskedtensor_advanced_semantics.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_prototype_maskedtensor_advanced_semantics.py: (Prototype) MaskedTensor Advanced Semantics =========================================== .. GENERATED FROM PYTHON SOURCE LINES 9-21 Before working on this tutorial, please make sure to review our `MaskedTensor Overview tutorial `. The purpose of this tutorial is to help users understand how some of the advanced semantics work and how they came to be. We will focus on two particular ones: *. Differences between MaskedTensor and `NumPy's MaskedArray `__ *. Reduction semantics Preparation ----------- .. GENERATED FROM PYTHON SOURCE LINES 22-31 .. code-block:: default # Disable prototype warnings and such .. GENERATED FROM PYTHON SOURCE LINES 32-45 MaskedTensor vs NumPy's MaskedArray ----------------------------------- NumPy's ``MaskedArray`` has a few fundamental semantics differences from MaskedTensor. *. Their factory function and basic definition inverts the mask (similar to ``torch.nn.MHA``); that is, MaskedTensor uses ``True`` to denote "specified" and ``False`` to denote "unspecified", or "valid"/"invalid", whereas NumPy does the opposite. We believe that our mask definition is not only more intuitive, but it also aligns more with the existing semantics in PyTorch as a whole. *. Intersection semantics. In NumPy, if one of two elements are masked out, the resulting element will be masked out as well -- in practice, they `apply the logical_or operator `__. .. GENERATED FROM PYTHON SOURCE LINES 56-59 Meanwhile, MaskedTensor does not support addition or binary operators with masks that don't match -- to understand why, please find the :ref:`section on reductions `. .. GENERATED FROM PYTHON SOURCE LINES 71-75 However, if this behavior is desired, MaskedTensor does support these semantics by giving access to the data and masks and conveniently converting a MaskedTensor to a Tensor with masked values filled in using :func:`to_tensor`. For example: .. GENERATED FROM PYTHON SOURCE LINES 85-100 Note that the mask is `mt0.get_mask() & mt1.get_mask()` since :class:`MaskedTensor`'s mask is the inverse of NumPy's. .. _reduction-semantics: Reduction Semantics ------------------- Recall in `MaskedTensor's Overview tutorial `__ we discussed "Implementing missing torch.nan* ops". Those are examples of reductions -- operators that remove one (or more) dimensions from a Tensor and then aggregate the result. In this section, we will use reduction semantics to motivate our strict requirements around matching masks from above. Fundamentally, :class:`MaskedTensor`s perform the same reduction operation while ignoring the masked out (unspecified) values. By way of example: .. GENERATED FROM PYTHON SOURCE LINES 110-112 Now, the different reductions (all on dim=1): .. GENERATED FROM PYTHON SOURCE LINES 120-127 Of note, the value under a masked out element is not guaranteed to have any specific value, especially if the row or column is entirely masked out (the same is true for normalizations). For more details on masked semantics, you can find this `RFC `__. Now, we can revisit the question: why do we enforce the invariant that masks must match for binary operators? In other words, why don't we use the same semantics as ``np.ma.masked_array``? Consider the following example: .. GENERATED FROM PYTHON SOURCE LINES 139-141 Now, let's try addition: .. GENERATED FROM PYTHON SOURCE LINES 146-154 Sum and addition should clearly be associative, but with NumPy's semantics, they are not, which can certainly be confusing for the user. :class:`MaskedTensor`, on the other hand, will simply not allow this operation since `mask0 != mask1`. That being said, if the user wishes, there are ways around this (for example, filling in the MaskedTensor's undefined elements with 0 values using :func:`to_tensor` like shown below), but the user must now be more explicit with their intentions. .. GENERATED FROM PYTHON SOURCE LINES 161-171 Conclusion ---------- In this tutorial, we have learned about the different design decisions behind MaskedTensor and NumPy's MaskedArray, as well as reduction semantics. In general, MaskedTensor is designed to avoid ambiguity and confusing semantics (for example, we try to preserve the associative property amongst binary operations), which in turn can necessitate the user to be more intentional with their code at times, but we believe this to be the better move. If you have any thoughts on this, please `let us know `__! .. GENERATED FROM PYTHON SOURCE LINES 171-172 .. code-block:: default # %%%%%%RUNNABLE_CODE_REMOVED%%%%%% .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 0.001 seconds) .. _sphx_glr_download_prototype_maskedtensor_advanced_semantics.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: maskedtensor_advanced_semantics.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: maskedtensor_advanced_semantics.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_