Updating the inverse of a matrix
Perhaps you are looking for the Sherman-Morrison-Woodbury update, which allows to perform a rank k update directly to the inverse of a matrix rather than inverting a rank k update to the matrix itself.
It is used heavily in statistics and optimization literature.
The key insight is that one way to compute the SSCP matrix is as a sum of outer products of the rows of X.
Therefore if x`) The formula shows how to compute the inverse of the updated SSCP by using a matrix-vector multiplication and an outer product.
Imagine doing linear regression, but being given the datapoints one-by-one.
The following program creates a new design matrix (Z) that excludes the row, forms the new SSCP matrix, and finds its inverse: similar computations.