When Thurgood Marshall won a case, he would throw wild celebrations—and when he won Brown v. Board of Education before the Supreme Court, the champagne flowed like waterfalls. Sixty years ago in May, Marshall joined his NAACP colleagues to toast their success and to drink well into the night. And yet, as the evening wore on, Marshall grew increasingly sober. “I don’t want any of you to fool yourselves,” he warned the jubilant crowd. “The fight has just begun.”

Marshall, as it turns out, was too optimistic.

As Harvard’s Michael Klarman has documented, five years after the Court’s decision, just forty of North Carolina’s 300,000 African-American students attended integrated schools. A year later, only forty-two of Nashville’s 12,000 black students studied alongside white children. By 1964, ten years after Marshall’s victory before the Supreme Court, just one in eighty-five Southern black children attended an integrated school.

The Court bears much of the blame for this. The justices of the mid-1950s came of age at a time when judges routinely struck down federal child labor laws and other progressive legislation, citing dubious theories of the Constitution. By the time they became justices, many of them were deeply skeptical of judicial action of any kind.

Accordingly, the justices settled on a deeply compromised approach to integration. The Court held, a year after Brown, that integration need not be rushed, and must proceed only “with all deliberate speed.” Worse, the Court left local federal district judges, many of whom had been selected by segregationist senators, to supervise the implementation of Brown.

Though Southern resistance was inevitable, this timid order emboldened segregationists. Close to a hundred members of Congress signed a “Southern Manifesto” decrying the “explosive and dangerous condition created by [Brown] and inflamed by outside meddlers.” Several Virginia schools shut down entirely rather than permit black students to be educated alongside white ones, and of course there was Governor Orval Faubus’s infamous stand in Arkansas.

White racists also had another tactic to keep children separate: terrorism. The only way to force compliance was a court order, and that required a black family willing to sue. With the threat of Klan violence looming over anyone who dared to join a lawsuit, no one filed a case seeking to integrate a Mississippi grade school until 1963.

Brown was not a total washout. In border states like Maryland, where segregation was less rooted in the state’s culture, 90 percent of school districts complied with Brown by 1964. But in the Deep South, Brown barely laid a glove on Jim Crow. Ten years after the justices declared school segregation unconstitutional, Justice Hugo Black wrote for a frustrated Court that “there has been entirely too much deliberation and not enough speed in enforcing” Brown.

What finally broke the back of segregation wasn’t the Supreme Court. And while the mass civil rights movement rallied countless Americans to the cause, it couldn’t have achieved its goals alone. It took Congress to breathe real life into Brown. Though the Civil Rights Act of 1964 is best remembered today for its provisions targeting private discrimination, it also contained two provisions enabling the federal government to desegregate public schools.

In the decade following Brown, few Southern lawyers were willing to bring desegregation cases—even if they could find a plaintiff willing to become their client. The Civil Rights Act solved this problem by empowering the Justice Department to file suits in the name of the US government against schools that refused to integrate. Southern districts that were able to dodge lawsuits brought by the NAACP suddenly had all the resources of the government to contend with.

In addition, the act enabled federal officials to withhold funding from segregated schools. This provision allowed the federal government to adapt more quickly to Southern attempts to thwart integration than the courts could ever have accomplished.

By the end of 1966, the administration had terminated federal funds for thirty-two school districts. By the time President Johnson left office, the government had begun proceedings against 634 school districts and terminated funding for more than 120 of them. Alabama Governor George Wallace, surveying the wreckage administration officials were making of public school segregation, declared that if he were president, he would “bring all these briefcase-toting bureaucrats in the Department of Health, Education and Welfare to Washington and throw their briefcases in the Potomac River.”

Though the Nixon administration drastically rolled back enforcement of the Civil Rights Act, it was forced by a court order to resume it. And just two years after the act took effect, the percentage of Southern black children attending integrated schools had increased more than fivefold. Within five years, nearly a third of Southern black children attended integrated schools. By 1973, as many as 90 percent did.

Half a century after this act became law, however, the civil rights movement faces a far graver danger than Richard Nixon. Not long after George W. Bush’s nominees remade the Supreme Court, they struck down two race-conscious plans seeking to integrate public schools—with Chief Justice John Roberts simplistically claiming that “the way to stop discrimination on the basis of race is to stop discriminating on the basis of race.” Six years later, Roberts tore the heart out of the Voting Rights Act.

A robust federal government was the only thing strong enough to break the back of Jim Crow. Especially given the radical bent of the Roberts Court, it may once again be the only thing that can stop the latest iteration of these old prejudices.