Shortcuts

raft_large

torchvision.models.optical_flow.raft_large(*, weights: Optional[torchvision.models.optical_flow.raft.Raft_Large_Weights] = None, progress=True, **kwargs)torchvision.models.optical_flow.raft.RAFT[source]

RAFT model from RAFT: Recurrent All Pairs Field Transforms for Optical Flow.

Please see the example below for a tutorial on how to use this model.

Parameters
  • weights (Raft_Large_Weights, optional) – The pretrained weights to use. See Raft_Large_Weights below for more details, and possible values. By default, no pre-trained weights are used.

  • progress (bool) – If True, displays a progress bar of the download to stderr. Default is True.

  • **kwargs – parameters passed to the torchvision.models.optical_flow.RAFT base class. Please refer to the source code for more details about this class.

class torchvision.models.optical_flow.Raft_Large_Weights(value)[source]

The model builder above accepts the following values as the weights parameter. Raft_Large_Weights.DEFAULT is equivalent to Raft_Large_Weights.C_T_SKHT_V2. You can also use strings, e.g. weights='DEFAULT' or weights='C_T_V1'.

The metrics reported here are as follows. epe is the “end-point-error” and indicates how far (in pixels) the predicted flow is from its true value. This is averaged over all pixels of all images. per_image_epe is similar, but the average is different: the epe is first computed on each image independently, and then averaged over all images. This corresponds to “Fl-epe” (sometimes written “F1-epe”) in the original paper, and it’s only used on Kitti. fl-all is also a Kitti-specific metric, defined by the author of the dataset and used for the Kitti leaderboard. It corresponds to the average of pixels whose epe is either <3px, or <5% of flow’s 2-norm.

Raft_Large_Weights.C_T_V1:

These weights were ported from the original paper. They are trained on FlyingChairs + FlyingThings3D.

epe (on Sintel-Train-Cleanpass)

1.4411

epe (on Sintel-Train-Finalpass)

2.7894

per_image_epe (on Kitti-Train)

5.0172

fl_all (on Kitti-Train)

17.4506

min_size

height=128, width=128

num_params

5257536

recipe

link

The inference transforms are available at Raft_Large_Weights.C_T_V1.transforms and perform the following preprocessing operations: Accepts PIL.Image, batched (B, C, H, W) and single (C, H, W) image torch.Tensor objects. The images are rescaled to [-1.0, 1.0].

Raft_Large_Weights.C_T_V2:

These weights were trained from scratch on FlyingChairs + FlyingThings3D.

epe (on Sintel-Train-Cleanpass)

1.3822

epe (on Sintel-Train-Finalpass)

2.7161

per_image_epe (on Kitti-Train)

4.5118

fl_all (on Kitti-Train)

16.0679

min_size

height=128, width=128

num_params

5257536

recipe

link

The inference transforms are available at Raft_Large_Weights.C_T_V2.transforms and perform the following preprocessing operations: Accepts PIL.Image, batched (B, C, H, W) and single (C, H, W) image torch.Tensor objects. The images are rescaled to [-1.0, 1.0].

Raft_Large_Weights.C_T_SKHT_V1:

These weights were ported from the original paper. They are trained on FlyingChairs + FlyingThings3D and fine-tuned on Sintel. The Sintel fine-tuning step is a combination of Sintel, KittiFlow, HD1K, and FlyingThings3D (clean pass).

epe (on Sintel-Test-Cleanpass)

1.94

epe (on Sintel-Test-Finalpass)

3.18

min_size

height=128, width=128

num_params

5257536

recipe

link

The inference transforms are available at Raft_Large_Weights.C_T_SKHT_V1.transforms and perform the following preprocessing operations: Accepts PIL.Image, batched (B, C, H, W) and single (C, H, W) image torch.Tensor objects. The images are rescaled to [-1.0, 1.0].

Raft_Large_Weights.C_T_SKHT_V2:

These weights were trained from scratch. They are pre-trained on FlyingChairs + FlyingThings3D and then fine-tuned on Sintel. The Sintel fine-tuning step is a combination of Sintel, KittiFlow, HD1K, and FlyingThings3D (clean pass). Also available as Raft_Large_Weights.DEFAULT.

epe (on Sintel-Test-Cleanpass)

1.819

epe (on Sintel-Test-Finalpass)

3.067

min_size

height=128, width=128

num_params

5257536

recipe

link

The inference transforms are available at Raft_Large_Weights.C_T_SKHT_V2.transforms and perform the following preprocessing operations: Accepts PIL.Image, batched (B, C, H, W) and single (C, H, W) image torch.Tensor objects. The images are rescaled to [-1.0, 1.0].

Raft_Large_Weights.C_T_SKHT_K_V1:

These weights were ported from the original paper. They are pre-trained on FlyingChairs + FlyingThings3D, fine-tuned on Sintel, and then fine-tuned on KittiFlow. The Sintel fine-tuning step was described above.

fl_all (on Kitti-Test)

5.1

min_size

height=128, width=128

num_params

5257536

recipe

link

The inference transforms are available at Raft_Large_Weights.C_T_SKHT_K_V1.transforms and perform the following preprocessing operations: Accepts PIL.Image, batched (B, C, H, W) and single (C, H, W) image torch.Tensor objects. The images are rescaled to [-1.0, 1.0].

Raft_Large_Weights.C_T_SKHT_K_V2:

These weights were trained from scratch. They are pre-trained on FlyingChairs + FlyingThings3D, fine-tuned on Sintel, and then fine-tuned on KittiFlow. The Sintel fine-tuning step was described above.

fl_all (on Kitti-Test)

5.19

min_size

height=128, width=128

num_params

5257536

recipe

link

The inference transforms are available at Raft_Large_Weights.C_T_SKHT_K_V2.transforms and perform the following preprocessing operations: Accepts PIL.Image, batched (B, C, H, W) and single (C, H, W) image torch.Tensor objects. The images are rescaled to [-1.0, 1.0].

Examples using raft_large:

Optical Flow: Predicting movement with the RAFT model

Optical Flow: Predicting movement with the RAFT model

Optical Flow: Predicting movement with the RAFT model

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources