Temporal Interpolation as an Unsupervised Pretraining Task for Optical Flow Estimation

2018

Conference Paper

ps

The difficulty of annotating training data is a major obstacle to using CNNs for low-level tasks in video. Synthetic data often does not generalize to real videos, while unsupervised methods require heuristic n losses. Proxy tasks can overcome these issues, and start by training a network for a task for which annotation is easier or which can be trained unsupervised. The trained network is then fine-tuned for the original task using small amounts of ground truth data. Here, we investigate frame interpolation as a proxy task for optical flow. Using real movies, we train a CNN unsupervised for temporal interpolation. Such a network implicitly estimates motion, but cannot handle untextured regions. By fine-tuning on small amounts of ground truth flow, the network can learn to fill in homogeneous regions and compute full optical flow fields. Using this unsupervised pre-training, our network outperforms similar architectures that were trained supervised using synthetic optical flow.

@inproceedings{Wulff:GCPR:2018,
  title = {Temporal Interpolation as an Unsupervised Pretraining Task for Optical Flow Estimation},
  author = {Wulff, Jonas and Black, Michael J.},
  booktitle = {German Conference on Pattern Recognition (GCPR)},
  volume = {LNCS 11269},
  pages = {567--582},
  publisher = {Springer, Cham},
  month = oct,
  year = {2018},
  doi = {https://doi.org/10.1007/978-3-030-12939-2_39},
  month_numeric = {10}
}