Perceiving Systems, Computer Vision

A naturalistic open source movie for optical flow evaluation

2012

Conference Paper

ps


Ground truth optical flow is difficult to measure in real scenes with natural motion. As a result, optical flow data sets are restricted in terms of size, complexity, and diversity, making optical flow algorithms difficult to train and test on realistic data. We introduce a new optical flow data set derived from the open source 3D animated short film Sintel. This data set has important features not present in the popular Middlebury flow evaluation: long sequences, large motions, specular reflections, motion blur, defocus blur, and atmospheric effects. Because the graphics data that generated the movie is open source, we are able to render scenes under conditions of varying complexity to evaluate where existing flow algorithms fail. We evaluate several recent optical flow algorithms and find that current highly-ranked methods on the Middlebury evaluation have difficulty with this more complex data set suggesting further research on optical flow estimation is needed. To validate the use of synthetic data, we compare the image- and flow-statistics of Sintel to those of real films and videos and show that they are similar. The data set, metrics, and evaluation website are publicly available.

Award: (Koenderink Prize at ECCV 2022)
Author(s): Butler, D. J. and Wulff, J. and Stanley, G. B. and Black, M. J.
Book Title: European Conf. on Computer Vision (ECCV)
Pages: 611--625
Year: 2012
Month: October

Series: Part IV, LNCS 7577
Editors: {A. Fitzgibbon et al. (Eds.)}
Publisher: Springer-Verlag

Department(s): Perceiving Systems
Research Project(s): Optical flow in the LGN
Learning from Synthetic Data
MPI Sintel Flow
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Award Paper: Koenderink Prize at ECCV 2022

Links: pdf
dataset
youtube
talk
supplemental material
Video:

BibTex

@inproceedings{Butler:ECCV:2012,
  title = {A naturalistic open source movie for optical flow evaluation},
  author = {Butler, D. J. and Wulff, J. and Stanley, G. B. and Black, M. J.},
  booktitle = {European Conf. on Computer Vision (ECCV)},
  pages = {611--625},
  series = {Part IV, LNCS 7577},
  editors = {{A. Fitzgibbon et al. (Eds.)}},
  publisher = {Springer-Verlag},
  month = oct,
  year = {2012},
  doi = {},
  month_numeric = {10}
}