BFGS with Update Skipping and Varying Memory

Abstract

We give conditions under which limited-memory quasi-Newton methods with exact line searches will terminate in n steps when minimizing n-dimensional quadratic functions. We show that although all Broyden family methods terminate in n steps in their full-memory versions, only BFGS does so with limited-memory. Additionally, we show that full-memory Broyden family methods with exact line searches terminate in at most n + p steps when p matrix updates are skipped. We introduce new limited-memory BFGS variants and test them on nonquadratic minimization problems.

@article{KoOlNa98,  
author = {Tamara G. Kolda and Dianne P. O'Leary and Larry Nazareth}, 
title = {{BFGS} with Update Skipping and Varying Memory}, 
journal = {SIAM Journal on Optimization}, 
volume = {8}, 
number = {4}, 
pages = {1060--1083}, 
month = {November}, 
year = {1998},
doi = {10.1137/S1052623496306450},
}