A foreclosed property in the Los Angeles area. (AP Photo/Damian Dovarganes)   The recent settlement of mortgage fraud claims between the federal government, forty-nine states and five large banks unquestionably falls short of what’s needed— either to hold Wall Street accountable for its role in the financial crisis or to provide adequate restitution for the millions of families still living with the aftermath. With 11 million homeowners holding nearly $800 billion in negative equity, the $25 billion settlement simply doesn’t solve the problem. The Department of Housing and Urban Development, a key party to the deal, says so directly on its website: “This agreement does not—and is not intended to—solve or resolve all the issues and abuses related to the housing crisis.”

But as the settlement is actually executed, and as a related federal investigation into Wall Street malfeasance led by New York State Attorney General Eric Schneiderman swings into action, it’s worth remembering that the deal might have been a whole lot worse. Early this year, well-sourced rumors suggested the settlement might offer banks immunity for everything from origination of shaky mortgage-backed products to recent foreclosure abuses. Senator Sherrod Brown of Ohio said that he heard the deal would be “not much more than a slap on the wrist,” and insisted that “Wall Street should not get another bailout.”

Progressive groups, acting largely under the umbrella Campaign for a Fair Settlement, pushed back quickly, organizing a large petition drive and lining up Congressional advocates to blast the deal. When the details finally came out, they were much better than expected. The settlement, while carrying a relatively low dollar amount, only released banks from prosecution over robo-signing abuses. Schneiderman, a noted advocate for homeowners seeking justice, was named co-chair of a federal task force to investigate mortgage fraud.

Progressive pressure on the administration was crucial, particularly in creating Schneiderman’s investigation. “The task force thing was really something that came about at the end because there was so much heat and so much pressure on the White House to do something bigger and broader,” said Mike Lux, a progressive strategist and former Clinton White House staffer who played a key role in the pushback.

This is an extremely valuable lesson going forward, because without continued activism the whole deal may yet fall apart. The devil is in the details—and that’s where progressives need to focus their attention. Under the terms of the settlement, for example, banks are obliged to issue mortgage modifications and stop foreclosing on homeowners unless they have the necessary paperwork and legal rights. It’s largely the job of Joseph Smith, North Carolina’s banking commissioner and the settlement’s court-appointed monitor, to make sure the banks comply. His first major task is to select a primary accounting and consulting firm to help him monitor bank compliance.

Smith’s selection is pivotal. The chosen firm must not have close connections with the banks it’s monitoring, which is exactly what happened last year when the Federal Reserve ordered banks to review their foreclosure practices after disturbing revelations about robo-signing abuse. Instead of conducting the reviews from within the Federal Reserve, the banks were allowed to select outside firms to monitor their operations. This led to predictable conflicts of interest. JPMorgan Chase, for example, selected Deloitte to review the banks’ foreclosures, which came in no small number from the acquisition of Washington Mutual. Deloitte was that bank’s auditor for many years and is currently being sued by Washington Mutual shareholders—creating a clear incentive for it to minimize problems at JPMorgan.

Advocacy at the state level is crucial for proper execution of the settlement as well. The deal offers Wall Street no immunity from individual homeowner lawsuits and gives states $3.5 billion to support legal counseling. But already, six states are planning to divert those funds to close budget gaps. It’s largely the work of Republican governors, including Wisconsin’s Scott Walker and New Jersey’s Chris Christie—but Colorado’s John Hickenlooper, a Democrat, may divert the funds as well to fund a private prison initiative.

Even if the settlement is executed properly, it’s not enough to save the 11 million distressed homeowners, which is why progressive groups are pushing for a second deal. Schneiderman’s task force is absolutely crucial to that effort—the idea is to quickly issue enough subpoenas and depositions to scare Wall Street executives into another settlement, this time for hundreds of billions of dollars. Along the way, at least some high-ranking executives should face criminal prosecution. This is what Schneiderman promised not long after his appointment, saying, “You can’t have equal justice under law and too big to fail.”

But there’s alarming evidence that the task force may be stalling out. The Justice Department has yet to assign the fifty-five promised staff members, and even if it does, that’s not nearly enough to probe adequately this level of widespread abuse. By comparison, the investigation into the savings and loan scandal of the ’80s had more than 1,000 staffers—for a much smaller scale of fraud.

On their side, the banks have hundreds of lawyers who can bury a small federal task force under mountains of paperwork, and there are dangerous statutes of limitations on some of the fraud allegations. Some violations have a prosecution limit of five years, meaning now is the last shot to prosecute some abuses that occurred in 2007. CREDO Action has launched a massive campaign to amp up task force staffing to at least 1,000 people.

Finally, to truly fix the massive economic drag of underwater homeowners, Fannie Mae and Freddie Mac must write down the mortgages they hold to fair market value. There has been significant progressive pressure on Edward DeMarco, the regulator of Fannie and Freddie, to do so—and now that pressure is even coming from the administration, in particular from HUD Secretary Shaun Donovan. DeMarco announced in April that he would at least consider a pilot principal reduction plan, though one that would address only a sliver of homeowners. This may represent a true change in thinking by DeMarco or simply a red herring, destined to fail and designed to decrease some of the progressive pressure—but either way, activists must keep pushing.

Much of this seems like an inside game—selecting the right bank monitor, staffing a task force and so on. But outside pressure is what got us here in the first place, and it’s what we need to get the best deal for homeowners and the economy at large. This isn’t a typical game of political theater, in which advocates build public support and fend off Republican attacks. Polls show massive majorities favor getting tough on Wall Street, and there has been deafening silence from the GOP about the settlement and the federal task force. It’s a political winner—but the administration, sadly but surely, must be persuaded to take on the powerful financial sector head-on.