Once Americans have to start repaying their federal student loans, it’ll be a struggle for many of them to make payments, a report out this week from the Federal Reserve Bank of New York predicts.

Payments were paused more than two years ago at the start of the pandemic as part of the CARES Act; they are scheduled to start again on May 1, although there are hints it may be extended further.

“This is a monumental task to restart payments for 37 million borrowers,” said Daniel Mangrum, an economist at the New York Fed and one of the authors of the new report. “Millions of them have never made a payment because they graduated during the pandemic and have never had to make a payment before.”

Easing people into that will be hard, but if you look at this as a game hot potato — the potato being the debt — it’s the government that is losing.

“Eighty-five percent of all outstanding balances are owned by the federal government. So essentially, this is kind of increasing the cost of the federal student loan program,” Mangrum said.

The Fed estimates the government held off on collecting $195 billion over the course of the pandemic and also missed out on interest payments.

It’s time for the government to declare victory, take its potato and go home, said Marc Goldwein, senior vice president and senior policy director with the fiscally conservative Committee for a Responsible Federal Budget.

“Right now, the employment situation for college graduates is actually quite strong,” he said. “And the idea that we should still be giving them emergency relief doesn’t really make much sense.”

Others, like Dalié Jiménez, who directs the Student Loan Law Initiative at the University of California, Irvine, say the loans should be erased altogether or not needed in the first place. She believes college should cost less.

“The answer is to allow people to continue their education and to do it without having to bankrupt themselves,” Jiménez said, especially if more jobs require college degrees.