Find an alternative representation of matrix inversion using subspace projection without explicitly constructing the complementary orthonormal matrix | Step-by-Step Solution

What We're Solving:

We need to compute the matrix H = D(D^TGD)^{-1}D^T without explicitly constructing the complementary orthonormal matrix D. This is a beautiful problem about subspace projections and matrix identities!

The Approach:

The key insight is to use the relationship between C and D through projection matrices. Since C and D are complementary orthonormal matrices, they span orthogonal subspaces that together form the entire space. We'll leverage the fact that I = CC^T + DD^T (the identity can be decomposed into projections onto these complementary subspaces).

Step-by-Step Solution:

Step 1: Understand the relationship between C and D Since C and D are complementary orthonormal matrices:

• CC^T projects onto the subspace spanned by C
• DD^T projects onto the subspace spanned by D
• CC^T + DD^T = I (they form a complete orthogonal decomposition)

Step 2: Express DD^T in terms of C From the identity above: DD^T = I - CC^T

Step 3: Recognize what H represents The matrix H = D(D^TGD)^{-1}D^T has a special structure. Notice that:

• D^TGD is a "reduced" version of G in the subspace spanned by D
• (D^TGD)^{-1} is its inverse in that subspace
• The full expression H represents the pseudoinverse of G restricted to the subspace complementary to C

Step 4: Use the Sherman-Morrison-Woodbury identity For matrices of the form A(B^TAB)^{-1}A^T where A has orthonormal columns, there's a beautiful relationship. In our case, we can show that:

H = (G + CC^T)^{-1} - (G + CC^T)^{-1}C[C^T(G + CC^T)^{-1}C]^{-1}C^T(G + CC^T)^{-1}

Step 5: The elegant solution Since G is symmetric, we can use the fact that H is the orthogonal projection of G^{-1} onto the subspace spanned by D. This gives us:

H = (I - CC^T)G^{-1}(I - CC^T) when G is invertible on the complement of C's subspace.

The Answer:

H = (I - CC^T)G†(I - CC^T)

Where G† represents the appropriate generalized inverse of G. If G is positive definite, this simplifies beautifully. The key insight is that we've transformed a problem requiring explicit construction of D into one using only the known matrix C!

Memory Tip:

Remember: "Complement without construction!" When you have complementary subspaces, you can always write DD^T = I - CC^T. This trick appears frequently in optimization, statistics (especially in regression with constraints), and numerical linear algebra. The pattern "I minus the known projection equals the unknown projection" is your friend!

This problem showcases how elegant linear algebra can be - sometimes the indirect path (avoiding explicit construction) leads to more beautiful and computationally efficient solutions!