Perceiving Systems, Computer Vision

Optical Flow Estimation using a Spatial Pyramid Network

2017

Conference Paper

ps


We learn to compute optical flow by combining a classical spatial-pyramid formulation with deep learning. This estimates large motions in a coarse-to-fine approach by warping one image of a pair at each pyramid level by the current flow estimate and computing an update to the flow. Instead of the standard minimization of an objective function at each pyramid level, we train one deep network per level to compute the flow update. Unlike the recent FlowNet approach, the networks do not need to deal with large motions; these are dealt with by the pyramid. This has several advantages. First, our Spatial Pyramid Network (SPyNet) is much simpler and 96% smaller than FlowNet in terms of model parameters. This makes it more efficient and appropriate for embedded applications. Second, since the flow at each pyramid level is small (< 1 pixel), a convolutional approach applied to pairs of warped images is appropriate. Third, unlike FlowNet, the learned convolution filters appear similar to classical spatio-temporal filters, giving insight into the method and how to improve it. Our results are more accurate than FlowNet on most standard benchmarks, suggesting a new direction of combining classical flow methods with deep learning.

Author(s): Anurag Ranjan and Michael Black
Book Title: Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017
Pages: 2720-2729
Year: 2017
Month: July
Day: 21-26
Publisher: IEEE

Department(s): Perceiving Systems
Research Project(s):
Bibtex Type: Conference Paper (inproceedings)
Paper Type: Conference

Event Name: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Event Place: Honolulu, HI, USA

Address: Piscataway, NJ, USA
ISBN: 978-1-5386-0457-1
ISSN: 1063-6919

Links: pdf
SupMat
project/code

BibTex

@inproceedings{spynet,
  title = {Optical Flow Estimation using a Spatial Pyramid Network},
  author = {Ranjan, Anurag and Black, Michael},
  booktitle = {Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017},
  pages = {2720-2729},
  publisher = {IEEE},
  address = {Piscataway, NJ, USA},
  month = jul,
  year = {2017},
  doi = {},
  month_numeric = {7}
}