But with mortgage rates and home prices remaining elevated over the past several years, that’s blocked many would-be homeowners from breaking into the housing market. The number of first-time home buyers in the U.S. is abysmal: in 2004, the number of first-time homebuyers was nearly 3.2 million, according to NAR data shared with Fortune on Tuesday. By 2024, that number had plummeted to just 1.14 million.

The roadblocks of buying a home are essentially a chicken-and-egg situation. Older generations who bought homes decades ago—and who would typically be ready to downsize by now—aren’t budging out of fear of relatively high mortgage rates. Mortgage rates were sub-3% during the pandemic, peaked at 8% in October 2023, and currently hover near 7%. And since that supply isn’t on the market, home prices are higher, preventing younger generations from being able to buy a home. 

Plus, many Americans own their home outright—meaning they don’t have a mortgage payment. That’s good news for them, considering it’s unlikely mortgage rates will drop anytime soon, but dismal news for people trying to break into the housing market. 

The share of homeowners who don’t have a mortgage payment rose to 40% in 2023, up from 33% in 2010, which reflects a trend toward outright homeownership and conservative borrowing, according to a Goldman Sachs note published Tuesday. 

Meanwhile, the housing market is particularly vulnerable for homeowners who just recently bought. 

“We’re seeing early signs of risk building within specific markets and within specific borrower populations, like borrowers with limited equity or who are behind on student loans,” Tim Bowler, president of ICE Mortgage Technology, said in a statement.

People aren’t borrowing against their homes because rates are too high—and therefore risky

Meanwhile, U.S. mortgage borrowers have $11.5 trillion of tappable home equity in their properties, according to ICE Mortgage Technology. But their preference to tap into their equity has been more muted than between 2001 and 2008 because it appears borrower demand for expensive and riskier debt has shifted following the Global Financial Crisis (GFC), according to Goldman Sachs. 

“Rather, borrowers have focused on paying down their mortgages and owning their homes outright,” wrote Goldman Sachs analyst Arun Manohar. That’s because borrowers are more averse to riskier debt products (those with a higher interest rate) like home equity lines of credit (HELOC)

Manohar did not respond to Fortune’s request for additional comment. 

ICE reported borrowers used just 0.41% of available tappable equity in the first quarter of 2025, which was less than half of the typical withdrawal rate observed from 2009 to 2021. Still, about 25% of homeowners said they are considering a home equity loan or HELOC in the next year, according to ICE’s June 2025 Mortgage Monitor report.

“Equity levels remain historically high, and now we’re seeing the cost of borrowing against that equity drop meaningfully,” Andy Walden, head of mortgage and housing market research at ICE, said in a statement. “If the Fed moves forward with anticipated rate cuts, borrowing against home equity could become even more attractive in the second half of the year.”

Although the bottom line is more people are buying their homes outright or don’t have a mortgage payment out of fear of high home prices and mortgage rates, the silver lining is people are building more wealth and less debt by avoiding taking out home equity.

“The continued slow pace of home equity extraction is likely due to factors such as higher mortgage rates, stricter underwriting standards, lower levels of mortgage lending by banks, and more conservative borrowing behavior,” Manohar wrote.