Middle-income borrowers are finding homeownership increasingly out of reach.