Mortgage interest rates hit a new low this week, again. According to Freddie Mac, the average rate on a 30-year, fixed-rate mortgage is now 3.71%. That is, for those keeping track, the 14th record low set this year.

That’s great, if you’re a potential homebuyer staring down ever-rising prices on the few homes available for sale. But if you’re looking to refinance an existing mortgage to save a little money, a new fee kicked in this week that might make you think twice.

It’s called an “adverse market fee,” and Fannie Mae and Freddie Mac started collecting it for refinance loans they buy from lenders. Those lenders are expected to pass at least some of the 0.5% charge on to borrowers. The idea, said Marina Walsh, vice president of industry analysis with the Mortgage Bankers Association, is to protect the companies against losses from unpaid mortgage bills.

“The current projection is that with COVID losses, it could be upwards of $6 billion for Fannie and Freddie,” Walsh said.

But there’s something else going on here. Fannie and Freddie have been in conservatorship — basically, government protection and control — since they almost went under during the financial crisis of 2008. Their top regulator, Federal Housing Finance Agency Director Mark Calabria, wants the companies to raise enough capital to return to shareholder control.

David Stevens, commissioner of the Federal Housing Administration during the Obama administration, sees the fee as Calabria’s way of speeding up that process.

“If there was higher risk to these loans, that would be one thing,” Stevens said. But Fannie and Freddie already own or guarantee most of the loans people would be refinancing, he said, “and if you can refinance them to a lower rate, and therefore lower payment, you’re actually putting the borrower in a better position.”

Adding to the cost of refinancing undermines the Federal Reserve’s efforts to stimulate the economy by keeping interest rates low, Stevens said. And if you’re wondering whether the incoming Biden administration will nix the fee, the FHFA is an independent agency, Stevens pointed out, and Calabria has more than three years left in his term.

But Eric Kaplan, with the Milken Institute’s Center for Financial Markets, said the fee likely won’t have a big impact on demand. If an extra $15 or $20 a month is enough to make or break a decision to refinance, he said, “that, to me, is a loan that probably shouldn’t get made.”