Let’s start with the obvious. Behind every great fortune is a great crime and, with the exception of the 364,122 owners of the Green Bay Packers, there’s no owner in pro sports who does not have some blood under their buffed fingernails. This is a club of billionaires, and that kind of money doesn’t come from scratching Lotto tickets.

Let’s also state the obvious and say that Donald Sterling is hardly the sole racist who calls the owner’s box his home. If you think he is, I have some Lady Gaga tickets at the Verizon Center to sell you. One person who works amongst the owners in major North American sport e-mailed me just to say, “I can tell you off the record about at least three of these guys who talk in a way straight from the Donald Sterling handbook.”

But even though many owners are far from cuddly creatures and have taken part in “wealth building” exercises that make Sterling’s archipelago of slums look like a side project, we should not be cynical about what just took place in the NBA. In fact, we should embrace it.

History has been made with the banning of Donald Sterling. Dating back to the uprooting of Chavez Ravine to create Dodgers Stadium, we have seen professional sports use “eminent domain” to separate people from their property to build their sports cathedrals. Here we have Adam Silver and the NBA owners lining up to seize the property of Donald Sterling not only because he is an embarrassment and a bigot but, most critically, as NBA Commissioner Adam Silver said, because of a “pattern of behavior.” (Sterling will of course be compensated to a far greater degree than those kicked out of their homes to build stadiums.)

That pointed reference to Sterling’s years as a slumlord immediately raises the issue of what Dallas Mavericks owner Mark Cuban fearfully called the “slippery slope.” As Cuban said, “If it’s about racism and we’re ready to kick people out of the league, OK? Then what about homophobia?… What about somebody who’s anti-Semitic? What about a xenophobe? In this country, people are allowed to be morons.” I can’t speak for Cuban, but given the billions in public dollars handed to owners in public financing, tax breaks and public trust, I don’t think it’s too much to ask that the owner of a franchise not be a raving bigot. But Cuban also of course pointedly ignores that Sterling was a bigot in word and deed. Mark Cuban is not, to use his language, “a moron.” The real fear that beats in his libertarian heart is that the harm caused by Donald Sterling’s business practices could be used against other owners as well. After all, if we start judging in the public square how billionaires make their money, few would dare to even leave their house.

If Donald Sterling’s slums make him unsuitable to own an NBA team, what do we say about the repeated chemical dumping practices of Jacksonville Jaguars owner Shahid Khan? His company Flex-N-Gate has been cited for “39 violations of hazardous waste regulations over 20 years.” Workplace conditions at Khan’s plants have routinely sent his employees to the hospital. Chemical dumping also has a long history of racism, as the neighborhoods with the least power get the dumping and the ensuing ill-health effects.

If Donald Sterling is a public embarrassment, what do you say about David Glass, owner of the Kansas City Royals and former CEO of Wal-Mart. In 1992, Dateline NBC broadcast damning scenes of Bangladeshi children slaving away at Wal-Mart products. When asked to defend this practice, Glass said simply, “You and I might, perhaps, define children differently.” He stomped out of the interview before answering why signs in Wal-Mart that said “Made in America” were hanging over goods made by Bangladeshi children. Yet David Glass remains.

If the NBA aims to welcome Jason Collins and have a league where homophobia is a thing of the past, how can it still have Dick DeVos as the owner of the Orlando Magic? DeVos has for four decades been the fundraiser in chief for the right-wing edge of the Republican Party. His nonprofit, the DeVos Foundation, has pumped millions into groups that support radical “reparative gay therapy” and other “traditional” family values, as other members of the DeVos clan have also underwritten “the culture war” for the radical right. Ironically, all their free-marketeering and railing against “creeping socialism” didn’t stop them from taking hundreds of millions of dollars in public money for the new Orlando Magic arena. (DeVos is not the only free marketeer to use public financing to underwrite political pet causes, including the keeping of LGBT people in the closet. Ladies and Gentlemen, your Oklahoma City Thunder!)

And then there is Dan Snyder, owner of the Washington football team. I’ve written about Snyder enough. Either we accept racism in sports or we oppose it. In the post-Sterling world, we have had politicians from Harry Reid to John McCain who have said that it is time for the name to change. Not only does Snyder clutch onto the team, but also does so like a cornered bobcat, lashing out at anyone who dares encroach on his precious brand. If Sterling’s racism has no place in sports, then the question of how the Washington football team can continue to be named after a slur becomes a crackling, livewire discussion.

Maybe Mark Cuban is correct. Maybe it is a slippery slope. And maybe, after so many years of billionaires living above any semblance of the law, this Donald Sterling moment is even bigger than we think. Maybe, this slope should be greased up and we should not be afraid to get behind the once untouchable owners in professional sports and give a little push.