Shortcuts

GLU

class torch.nn.GLU(dim=-1)[source]

Applies the gated linear unit function GLU(a,b)=aσ(b){GLU}(a, b)= a \otimes \sigma(b) where aa is the first half of the input matrices and bb is the second half.

Parameters

dim (int) – the dimension on which to split the input. Default: -1

Shape:
  • Input: (1,N,2)(\ast_1, N, \ast_2) where * means, any number of additional dimensions

  • Output: (1,M,2)(\ast_1, M, \ast_2) where M=N/2M=N/2

Examples:

>>> m = nn.GLU()
>>> input = torch.randn(4, 2)
>>> output = m(input)

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources