raft_large¶
- torchvision.models.optical_flow.raft_large(*, weights: Optional[Raft_Large_Weights] = None, progress=True, **kwargs) RAFT [source]¶
RAFT model from RAFT: Recurrent All Pairs Field Transforms for Optical Flow.
Please see the example below for a tutorial on how to use this model.
- Parameters:
weights (
Raft_Large_Weights
, optional) – The pretrained weights to use. SeeRaft_Large_Weights
below for more details, and possible values. By default, no pre-trained weights are used.progress (bool) – If True, displays a progress bar of the download to stderr. Default is True.
**kwargs – parameters passed to the
torchvision.models.optical_flow.RAFT
base class. Please refer to the source code for more details about this class.
- class torchvision.models.optical_flow.Raft_Large_Weights(value)[source]¶
The model builder above accepts the following values as the
weights
parameter.Raft_Large_Weights.DEFAULT
is equivalent toRaft_Large_Weights.C_T_SKHT_V2
. You can also use strings, e.g.weights='DEFAULT'
orweights='C_T_V1'
.The metrics reported here are as follows.
epe
is the “end-point-error” and indicates how far (in pixels) the predicted flow is from its true value. This is averaged over all pixels of all images.per_image_epe
is similar, but the average is different: the epe is first computed on each image independently, and then averaged over all images. This corresponds to “Fl-epe” (sometimes written “F1-epe”) in the original paper, and it’s only used on Kitti.fl-all
is also a Kitti-specific metric, defined by the author of the dataset and used for the Kitti leaderboard. It corresponds to the average of pixels whose epe is either <3px, or <5% of flow’s 2-norm.Raft_Large_Weights.C_T_V1:
These weights were ported from the original paper. They are trained on
FlyingChairs
+FlyingThings3D
.epe (on Sintel-Train-Cleanpass)
1.4411
epe (on Sintel-Train-Finalpass)
2.7894
per_image_epe (on Kitti-Train)
5.0172
fl_all (on Kitti-Train)
17.4506
min_size
height=128, width=128
num_params
5257536
recipe
GFLOPS
211.01
File size
20.1 MB
The inference transforms are available at
Raft_Large_Weights.C_T_V1.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[-1.0, 1.0]
.Raft_Large_Weights.C_T_V2:
These weights were trained from scratch on
FlyingChairs
+FlyingThings3D
.epe (on Sintel-Train-Cleanpass)
1.3822
epe (on Sintel-Train-Finalpass)
2.7161
per_image_epe (on Kitti-Train)
4.5118
fl_all (on Kitti-Train)
16.0679
min_size
height=128, width=128
num_params
5257536
recipe
GFLOPS
211.01
File size
20.1 MB
The inference transforms are available at
Raft_Large_Weights.C_T_V2.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[-1.0, 1.0]
.Raft_Large_Weights.C_T_SKHT_V1:
These weights were ported from the original paper. They are trained on
FlyingChairs
+FlyingThings3D
and fine-tuned on Sintel. The Sintel fine-tuning step is a combination ofSintel
,KittiFlow
,HD1K
, andFlyingThings3D
(clean pass).epe (on Sintel-Test-Cleanpass)
1.94
epe (on Sintel-Test-Finalpass)
3.18
min_size
height=128, width=128
num_params
5257536
recipe
GFLOPS
211.01
File size
20.1 MB
The inference transforms are available at
Raft_Large_Weights.C_T_SKHT_V1.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[-1.0, 1.0]
.Raft_Large_Weights.C_T_SKHT_V2:
These weights were trained from scratch. They are pre-trained on
FlyingChairs
+FlyingThings3D
and then fine-tuned on Sintel. The Sintel fine-tuning step is a combination ofSintel
,KittiFlow
,HD1K
, andFlyingThings3D
(clean pass). Also available asRaft_Large_Weights.DEFAULT
.epe (on Sintel-Test-Cleanpass)
1.819
epe (on Sintel-Test-Finalpass)
3.067
min_size
height=128, width=128
num_params
5257536
recipe
GFLOPS
211.01
File size
20.1 MB
The inference transforms are available at
Raft_Large_Weights.C_T_SKHT_V2.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[-1.0, 1.0]
.Raft_Large_Weights.C_T_SKHT_K_V1:
These weights were ported from the original paper. They are pre-trained on
FlyingChairs
+FlyingThings3D
, fine-tuned on Sintel, and then fine-tuned onKittiFlow
. The Sintel fine-tuning step was described above.fl_all (on Kitti-Test)
5.1
min_size
height=128, width=128
num_params
5257536
recipe
GFLOPS
211.01
File size
20.1 MB
The inference transforms are available at
Raft_Large_Weights.C_T_SKHT_K_V1.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[-1.0, 1.0]
.Raft_Large_Weights.C_T_SKHT_K_V2:
These weights were trained from scratch. They are pre-trained on
FlyingChairs
+FlyingThings3D
, fine-tuned on Sintel, and then fine-tuned onKittiFlow
. The Sintel fine-tuning step was described above.fl_all (on Kitti-Test)
5.19
min_size
height=128, width=128
num_params
5257536
recipe
GFLOPS
211.01
File size
20.1 MB
The inference transforms are available at
Raft_Large_Weights.C_T_SKHT_K_V2.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[-1.0, 1.0]
.
Examples using
raft_large
:Optical Flow: Predicting movement with the RAFT model
Optical Flow: Predicting movement with the RAFT model