Joe Frazier (AP Photo/File) 

The first African-American man to address the South Carolina state legislature after the Civil War wasn’t Booker T. Washington or W.E.B. Du Bois or Dr. Martin Luther King. It was heavyweight boxing champion “Smokin’ ” Joe Frazier, who died this week at the age of 67. Frazier had just emerged victorious from his epic 1971 encounter against Muhammad Ali, in a fight that was cast as a culture war between the “draft dodger” Ali and the “establishment hero” Joe Frazier. If you were against the war in Vietnam, you rooted yourself hoarse for Ali. If you wanted the hippies, freaks and Black Power disciples humbled, you wanted Smokin’ Joe.

In the wake of Frazier’s death, many have written that he didn’t deserve this tag: that he was labeled unfairly as a “sellout” by Ali and suffered for it. It is certainly true that Ali and Frazier were friends before their conflicts consumed Frazier with fury. It’s also true that when Ali was forced into exile for resisting the draft, it was Joe Frazier who gave Ali money when many others turned their backs on “The Greatest.”

Ali said to Frazier, “You just keep whupping those guys in the ring, and I’ll keep fighting Uncle Sam and one day we’ll make a lot of money together.”

But by 1971, both men were playing their roles. Ali taunted Frazier for being an Uncle Tom. Frazier also, which is less remembered, taunted Ali for being against the war.  He said that because he loved America, he’d proudly fight in Vietnam. He also repeatedly insulted Ali by calling him by his birthname, “Clay”

And then, after he whipped Ali in the “fight of the century,” Joe Frazier accepted that invitation to speak at the South Carolina legislature: a conquering hero.

One of thirteen children and born in abject poverty in Beaufort, South Carolina, it’s certainly understandable why he would accept the historic invitation. But that doesn’t make it any less of a full embrace of his role as the “good one” in the Ali-Frazier melodrama.

Speaking in a room with a Confederate flag backdrop in front of a chamber with only three black representatives among its 170 elected officials, Frazier’s message was gentle. He told jokes to great laughter about growing up in Beaufort and saying, “Yes, bawse” and “No, bawse” no matter the question.

He also said earnestly, “We must save our people, and when I say ‘our people’ I mean white and black. We need to quit thinking about who drives the fanciest car or who is my little daughter going to play with, who is she going to sit next to in school. We don’t have time for that.” Then his own 10-year-old daughter, to great cheers, stood and said, “Float like a butterfly, sting like a bee. My daddy is the one who whipped Muhammad Ali.”

While the chamber and national media swooned, Ali seethed. The beaten champ said Frazier was “consorting with the enemy.” He had, in Ali’s eyes, become a hero to the very people who as a young man in South Carolina wouldn’t have even spit in his direction.

As the 1970s labored on, and the movements that thrived at the decade’s inception began to wither, Ali’s taunts of Frazier became less political and more indefensible. When their epic 1975 fight in Manila loomed, Ali repeatedly called Frazier “a gorilla.” He spoke verses on how “black and ugly” Frazier was. For Ali, it was part of the show. For Frazier, it was more scarring than any punch in Ali’s arsenal.

Years later, Ali commented, “I said a lot of things in the heat of the moment that I shouldn’t have said. Called him names I shouldn’t have called him. I apologize for that. I’m sorry. It was all meant to promote the fight.”

Joe Frazier didn’t want to hear apologies. In retirement he would express joy at any role he may have played in Ali’s Parkinson’s disease. When Ali famously lit the torch at the 1996 Olympics, Frazier expressed grief that he couldn’t be there to shove Ali into the fires.

The roots of his anger were deeper than just anything uttered by Muhammad Ali. Joe Frazier was the 1964 Olympic Gold Medalist. He never dodged a draft. He never boasted of throwing his medal in the Ohio River. He never said “God damn America.” Yet there was Ali lighting the torch while he was stuck at home. The establishment had chosen the anti-hero, and Joe Frazier was cast merely as the foil and the fool.

It boggled Frazier’s mind when his adopted home of Philadelphia put up a statue of a boxer, and chose the very fictional—and very white—Rocky Balboa as their favorite fighting son. He did things “the right way” and Philly gave him the back of their hand like they were just another “bawse” in Beaufort.

This shouldn’t have been Joe Frazier’s fate: the convenient hero of everyone who wanted to see Ali punished for his politics. This shouldn’t have been Joe Frazier’s fate: internalizing and nursing every barb from “Gaseous Cassius” instead of letting it roll off his back. This shouldn’t have been Joe Frazier’s fate: rejected by the same establishment so quick to embrace him when it suited their needs. Smokin’ Joe deserved so much better.