raft_large¶

torchvision.models.optical_flow.
raft_large
(*, weights: Optional[torchvision.models.optical_flow.raft.Raft_Large_Weights] = None, progress=True, **kwargs) → torchvision.models.optical_flow.raft.RAFT[source]¶ RAFT model from RAFT: Recurrent All Pairs Field Transforms for Optical Flow.
Please see the example below for a tutorial on how to use this model.
 Parameters
weights (
Raft_Large_Weights
, optional) – The pretrained weights to use. SeeRaft_Large_Weights
below for more details, and possible values. By default, no pretrained weights are used.progress (bool) – If True, displays a progress bar of the download to stderr. Default is True.
**kwargs – parameters passed to the
torchvision.models.optical_flow.RAFT
base class. Please refer to the source code for more details about this class.

class
torchvision.models.optical_flow.
Raft_Large_Weights
(value)[source]¶ The model builder above accepts the following values as the
weights
parameter.Raft_Large_Weights.DEFAULT
is equivalent toRaft_Large_Weights.C_T_SKHT_V2
. You can also use strings, e.g.weights='DEFAULT'
orweights='C_T_V1'
.The metrics reported here are as follows.
epe
is the “endpointerror” and indicates how far (in pixels) the predicted flow is from its true value. This is averaged over all pixels of all images.per_image_epe
is similar, but the average is different: the epe is first computed on each image independently, and then averaged over all images. This corresponds to “Flepe” (sometimes written “F1epe”) in the original paper, and it’s only used on Kitti.flall
is also a Kittispecific metric, defined by the author of the dataset and used for the Kitti leaderboard. It corresponds to the average of pixels whose epe is either <3px, or <5% of flow’s 2norm.Raft_Large_Weights.C_T_V1:
These weights were ported from the original paper. They are trained on
FlyingChairs
+FlyingThings3D
.epe (on SintelTrainCleanpass)
1.4411
epe (on SintelTrainFinalpass)
2.7894
per_image_epe (on KittiTrain)
5.0172
fl_all (on KittiTrain)
17.4506
min_size
height=128, width=128
num_params
5257536
recipe
The inference transforms are available at
Raft_Large_Weights.C_T_V1.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[1.0, 1.0]
.Raft_Large_Weights.C_T_V2:
These weights were trained from scratch on
FlyingChairs
+FlyingThings3D
.epe (on SintelTrainCleanpass)
1.3822
epe (on SintelTrainFinalpass)
2.7161
per_image_epe (on KittiTrain)
4.5118
fl_all (on KittiTrain)
16.0679
min_size
height=128, width=128
num_params
5257536
recipe
The inference transforms are available at
Raft_Large_Weights.C_T_V2.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[1.0, 1.0]
.Raft_Large_Weights.C_T_SKHT_V1:
These weights were ported from the original paper. They are trained on
FlyingChairs
+FlyingThings3D
and finetuned on Sintel. The Sintel finetuning step is a combination ofSintel
,KittiFlow
,HD1K
, andFlyingThings3D
(clean pass).epe (on SintelTestCleanpass)
1.94
epe (on SintelTestFinalpass)
3.18
min_size
height=128, width=128
num_params
5257536
recipe
The inference transforms are available at
Raft_Large_Weights.C_T_SKHT_V1.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[1.0, 1.0]
.Raft_Large_Weights.C_T_SKHT_V2:
These weights were trained from scratch. They are pretrained on
FlyingChairs
+FlyingThings3D
and then finetuned on Sintel. The Sintel finetuning step is a combination ofSintel
,KittiFlow
,HD1K
, andFlyingThings3D
(clean pass). Also available asRaft_Large_Weights.DEFAULT
.epe (on SintelTestCleanpass)
1.819
epe (on SintelTestFinalpass)
3.067
min_size
height=128, width=128
num_params
5257536
recipe
The inference transforms are available at
Raft_Large_Weights.C_T_SKHT_V2.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[1.0, 1.0]
.Raft_Large_Weights.C_T_SKHT_K_V1:
These weights were ported from the original paper. They are pretrained on
FlyingChairs
+FlyingThings3D
, finetuned on Sintel, and then finetuned onKittiFlow
. The Sintel finetuning step was described above.fl_all (on KittiTest)
5.1
min_size
height=128, width=128
num_params
5257536
recipe
The inference transforms are available at
Raft_Large_Weights.C_T_SKHT_K_V1.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[1.0, 1.0]
.Raft_Large_Weights.C_T_SKHT_K_V2:
These weights were trained from scratch. They are pretrained on
FlyingChairs
+FlyingThings3D
, finetuned on Sintel, and then finetuned onKittiFlow
. The Sintel finetuning step was described above.fl_all (on KittiTest)
5.19
min_size
height=128, width=128
num_params
5257536
recipe
The inference transforms are available at
Raft_Large_Weights.C_T_SKHT_K_V2.transforms
and perform the following preprocessing operations: AcceptsPIL.Image
, batched(B, C, H, W)
and single(C, H, W)
imagetorch.Tensor
objects. The images are rescaled to[1.0, 1.0]
.
Examples using
raft_large
:Optical Flow: Predicting movement with the RAFT model
Optical Flow: Predicting movement with the RAFT model