Three Guidelines of Online Learning for Large-Scale Visual Recognition

Yoshitaka Ushiku, Masatoshi Hidaka, Tatsuya Harada

Paper, Supplemental material

Download the code (version 1.0.0)

Comparison using ILSVRC 2010 1.2M dataset with SIFT+FV. The darker bar for each algorithm shows the accuracy with averaging. The brighter shows the accuracy without averaging for easy reference.

Three guidelines for online learning for large-scale visual recognition

  1. Perceptron can compete against the latest methods.
    • Provided that the second guideline is observed.
  2. Averaging is necessary for any algorithm.
    • First-order algorithms w/o averaging cannot compete against second-order algorithms.
    • When averaging is used, the accuracies of all algorithms become very close to each other.
    • Averaging accelerates not only first-order algorithms but also second-order algorithms.
  3. Investigate multiclass learning first.
    • Both one-versus-the-rest learning and multiclass learning achieve similar accuracy.
    • However, one-versus-the-rest takes much longer CPU time to converge than multiclass does.

Contact: ushiku (at)