.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "beginner/former_torchies/autograd_tutorial_old.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        Click :ref:`here <sphx_glr_download_beginner_former_torchies_autograd_tutorial_old.py>`
        to download the full example code

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_beginner_former_torchies_autograd_tutorial_old.py:


Autograd
========

Autograd is now a core torch package for automatic differentiation.
It uses a tape based system for automatic differentiation.

In the forward phase, the autograd tape will remember all the operations
it executed, and in the backward phase, it will replay the operations.

Tensors that track history
--------------------------

In autograd, if any input ``Tensor`` of an operation has ``requires_grad=True``,
the computation will be tracked. After computing the backward pass, a gradient
w.r.t. this tensor is accumulated into ``.grad`` attribute.

There’s one more class which is very important for autograd
implementation - a ``Function``. ``Tensor`` and ``Function`` are
interconnected and build up an acyclic graph, that encodes a complete
history of computation. Each variable has a ``.grad_fn`` attribute that
references a function that has created a function (except for Tensors
created by the user - these have ``None`` as ``.grad_fn``).

If you want to compute the derivatives, you can call ``.backward()`` on
a ``Tensor``. If ``Tensor`` is a scalar (i.e. it holds a one element
tensor), you don’t need to specify any arguments to ``backward()``,
however if it has more elements, you need to specify a ``grad_output``
argument that is a tensor of matching shape.

.. GENERATED FROM PYTHON SOURCE LINES 32-35

.. code-block:: default


    import torch


.. GENERATED FROM PYTHON SOURCE LINES 36-37

Create a tensor and set requires_grad=True to track computation with it

.. GENERATED FROM PYTHON SOURCE LINES 37-40

.. code-block:: default

    x = torch.ones(2, 2, requires_grad=True)
    print(x)


.. GENERATED FROM PYTHON SOURCE LINES 42-44

.. code-block:: default

    print(x.data)


.. GENERATED FROM PYTHON SOURCE LINES 46-48

.. code-block:: default

    print(x.grad)


.. GENERATED FROM PYTHON SOURCE LINES 50-53

.. code-block:: default


    print(x.grad_fn)  # we've created x ourselves


.. GENERATED FROM PYTHON SOURCE LINES 54-55

Do an operation of x:

.. GENERATED FROM PYTHON SOURCE LINES 55-59

.. code-block:: default


    y = x + 2
    print(y)


.. GENERATED FROM PYTHON SOURCE LINES 60-62

y was created as a result of an operation,
so it has a grad_fn

.. GENERATED FROM PYTHON SOURCE LINES 62-64

.. code-block:: default

    print(y.grad_fn)


.. GENERATED FROM PYTHON SOURCE LINES 65-66

More operations on y:

.. GENERATED FROM PYTHON SOURCE LINES 66-72

.. code-block:: default


    z = y * y * 3
    out = z.mean()

    print(z, out)


.. GENERATED FROM PYTHON SOURCE LINES 73-75

``.requires_grad_( ... )`` changes an existing Tensor's ``requires_grad``
flag in-place. The input flag defaults to ``True`` if not given.

.. GENERATED FROM PYTHON SOURCE LINES 75-83

.. code-block:: default

    a = torch.randn(2, 2)
    a = ((a * 3) / (a - 1))
    print(a.requires_grad)
    a.requires_grad_(True)
    print(a.requires_grad)
    b = (a * a).sum()
    print(b.grad_fn)


.. GENERATED FROM PYTHON SOURCE LINES 84-88

Gradients
---------

let's backprop now and print gradients d(out)/dx

.. GENERATED FROM PYTHON SOURCE LINES 88-93

.. code-block:: default


    out.backward()
    print(x.grad)



.. GENERATED FROM PYTHON SOURCE LINES 94-98

By default, gradient computation flushes all the internal buffers
contained in the graph, so if you even want to do the backward on some
part of the graph twice, you need to pass in ``retain_variables = True``
during the first pass.

.. GENERATED FROM PYTHON SOURCE LINES 98-105

.. code-block:: default


    x = torch.ones(2, 2, requires_grad=True)
    y = x + 2
    y.backward(torch.ones(2, 2), retain_graph=True)
    # the retain_variables flag will prevent the internal buffers from being freed
    print(x.grad)


.. GENERATED FROM PYTHON SOURCE LINES 107-110

.. code-block:: default

    z = y * y
    print(z)


.. GENERATED FROM PYTHON SOURCE LINES 111-112

just backprop random gradients

.. GENERATED FROM PYTHON SOURCE LINES 113-122

.. code-block:: default


    gradient = torch.randn(2, 2)

    # this would fail if we didn't specify
    # that we want to retain variables
    y.backward(gradient)

    print(x.grad)


.. GENERATED FROM PYTHON SOURCE LINES 123-126

You can also stop autograd from tracking history on Tensors
with requires_grad=True by wrapping the code block in
``with torch.no_grad():``

.. GENERATED FROM PYTHON SOURCE LINES 126-131

.. code-block:: default

    print(x.requires_grad)
    print((x ** 2).requires_grad)

    with torch.no_grad():
    	print((x ** 2).requires_grad)


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** ( 0 minutes  0.000 seconds)


.. _sphx_glr_download_beginner_former_torchies_autograd_tutorial_old.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example


    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: autograd_tutorial_old.py <autograd_tutorial_old.py>`

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: autograd_tutorial_old.ipynb <autograd_tutorial_old.ipynb>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_