If you are in the market for a house, you know that it is rough out there. 

To recap: Mortgage rates are the highest they’ve been since 2000, at 7.5% for a 30-year fixed, according to Freddie Mac. Homeowners are locked in to much lower mortgage rates, so they don’t want to sell. And prices for homes that do go on the market remain high.

Building new housing could help things, and the U.S. Census Bureau reported Wednesday that the number of completed homes rose last month. But that likely won’t last, because further up the housing pipeline, homebuilders are starting fewer projects right now.

In a lot of parts of the country, the first thing a homebuilder has to do to build housing is get a permit. And the number of permits issued last month fell over 7% compared to a year ago because the math doesn’t really work for builders right now, said Jessica Lautz, deputy chief economist at the National Association of Realtors.

“It’s hard to find a buyer who can afford a new build, but also it’s expensive to build a home,” Lautz said.

It’s expensive for a few familiar reasons: Construction costs are high, it’s hard to find skilled workers, but most of all, “the increase in long-term interest rates,” said Robert Dietz, chief economist at the National Association of Home Builders.

Just like homebuyers taking out mortgages, homebuilders take out loans too.

“As interest rates increase, the cost of financing for acquiring land, developing building lots, and the construction loans for builders, all of those increase,” Dietz said.

Donny Wills sees that in Austin, Texas, where he owns River City Construction and Design. He said he used to take out loans to buy land pretty regularly. He’d build a house on spec, and then sell it for a profit. These days, not so much.

“The amount of interest you’d end up paying on that loan, you would never be able to sell that house for a good profit. You know, we would end up probably taking a loss on spec homes,” Wills said.

So, he’s building fewer houses. And that doesn’t bode well for a market that is seriously lacking in supply, said Aaron Jodka at the investment firm Colliers. He said ultimately, it may take more government support, like zoning changes and grants or loans, to meet demand.

“That’s probably a combination of municipal policy as well as tax relief, state government, etc., coming together to try to solve those problems. It’s certainly not easy,” Jodka said.

Meanwhile, Donny Wills in Austin said as long as interest rates are high, he can make money another way.

“I get more calls right now for remodels than I do for custom homes,” Wills said.

And he said, when rates eventually come down to 5% or 6%, builders like him will start investing in new homes again.