EDITOR’S NOTE: This article originally appeared at TomDispatch.com. To stay on top of important articles like these, sign up to receive the latest updates from TomDispatch.com.

Almost 80, I’ve been stunned and bewildered by the ever-expanding list of sexually predatory males, from movie mogul Harvey Weinstein to former New Republic editor Leon Wieseltier to comedian Louis C.K. And it’s triggered a list of questions for me: Who raised these louts? How did they give themselves permission to harass and assault women the way they did? Why did they think they could get away with it? And above all, who enabled them to advance along this vicious spectrum from creepy remarks to groping to rape?

Slowly, I’ve come to a realization I probably should have had long ago. It’s men like me, the bystanders, who enabled them. However righteous we may feel as they’re exposed and punished, the truth is we’re the problem, too.

But we’re at least part of the solution as well. One lesson from Donald Trump’s boasts to a sycophantic TV broadcaster, revealed during his run for president, about grabbing “’em by the pussy”: Sexual harassment—or even claiming to have done it—is just another way of preening for the pack. Trump obviously saw the female objects of his faux-macho lust as props. He might as well have put his hands on Billy Bush or me for that matter (as Kevin Spacey evidently did to young men on his movie sets). Alpha dogs like them have always been able to do more or less what they wanted—until, that is, people started listening to the women speaking up. And that reality reinforces who the cowards were: we bystander bros.

I learned that 70 years ago in my elementary-school playground in Queens, New York. At recess, Crazy Ronnie pinned girls against the chain-link fence and cackled as he felt them up. We boys, maybe 9, 10, 11 years old, were afraid of Ronnie. No one of us could “take” him, so we just watched. Of course, three or four of us could have pulled him off and stopped it all. Even at that age, what were we thinking? Didn’t we read books and see movies about heroic male saviors of women and children? Could we have been getting our own secondhand thrills from his acts?

Eventually, a teacher would notice and drag him away, ending the show. Nothing would be said and life would go on, except that the young girl probably wouldn’t forget the assault (and, as it turns out, neither would I).

When I started working at The New York Times in 1957 as a 19-year-old copyboy, there were few more approachable older guys in the newsroom than the motion-picture editor and third-string movie critic known as Doc. The culture departments were next to sports, where I worked, and Doc was friendly, loud, and inclusive, especially when he lurched back from lunch, waving to the guys and squeezing the women.

Even then, though everyone seemed officially amused, it was discomfiting to watch. When I first arrived at the paper, he was 50-ish, short, with curly dark hair and a push-broom moustache. He played cute, acted puppyish as he stood on his toes to kiss cheeks or encircle a woman from behind. Mostly, he pawed young news assistants and secretaries. The few female editors and reporters at the Times in those days wouldn’t have been likely to tolerate his advances. He wasn’t that important.

It was a year or two before l worked up the nerve to ask any of the women why they suffered him that way. “Oh, that’s the way he is,” came the typical, slightly embarrassed answer, or “He’s really nice, he means no harm.” It took a lot longer for a few of them to trust me enough to confide that, while they thought he was a jerk, he still had the juice to drop a negative remark that could stall a promotion or hurt a future career. He could have done that to me, too, then, had I intervened.

But once I was a reporter, presumably beyond his reach when it came to doing damage, why didn’t I step up? Did I feel that such a move would also mean stepping beyond the brotherhood, marking myself as a grandstander, an outsider? Did I prefer to identify with the perp instead of the victim? Had I forgotten all the chivalric novels I’d read growing up and decided it just wasn’t my fight?

And then, one day, it was.

I began dating Marjorie, an ambitious secretary in the paper’s music department. One necessity of her job was friendly relations with the culture editors, since they controlled the assignments for the sorts of feature stories and reviews that would advance her career. In those years, that meant she’d grin and bear a little rubbing, which offended me—territorially.

So I made a show of sitting on the edge of her desk and glaring Doc away. In what I assumed was a gesture of macho courtesy, he then left her alone, even threw her some ideas for feature stories. Soon enough, some of those pieces got her a “promotion” to subeditor in the women’s department, hardly what she wanted.

That was a systemic aspect of the routine sexism at the Times in those days, one that only added to the vulnerability of the women it employed. It was so much harder to crack the “men’s departments” (that is, the rest of the paper). Marjorie soon left the Times and we got married. Doc was effusive in his congratulations.

It Starts With Jock Culture

For a while I assumed that Crazy Ronnie and Doc were discreet episodes, a boyhood psychopath and a jerk. But as a sportswriter for the next 50-odd years, I began to see both of them somewhat differently: as damaged victims of what I’ve come to think of as jock culture, that web of male attitudes and behavior that’s most commonly absorbed through sports participation, but can affect computer geeks and drama-club nerds just as surely as it does varsity athletes.

The same jock-culture codes that we bumped up against in school—the ones that insisted that real men are tough and aggressive, take risks, and trust no one who isn’t on their team, especially women (by definition on the other team)—were waiting for us in the military, business, medicine, the law, and beyond.

Journalism, which expends a great deal of energy examining and carping about all forms of popular culture, rarely criticizes jock culture in any fundamental way. It may finally be ready to kick any celebrity caught with his pants down, but not even now does it expose the system that made him feel entitled to do whatever he wanted to do to whomever he could reach and abuse. Not surprisingly, journalism itself has been exposed as a cesspool of abusive personal power, thanks to Roger Ailes, Bill O’Reilly, and most recently Mark Halperin.

In the 1980s, I found myself on air at the CBS Sunday Morning show, where the executive producer indulged in some of the most vile and provocative language I’d ever heard directed at women. I never saw him actually touch anyone, but female producers winced and sometimes cried after he verbally assaulted and insulted them. At a meeting for my first assignment for his show, the executive producer, known as Shad, described my producer, who was standing right next to me, as a “crazed slut” who had to be watched every minute while we were on the road lest she embarrass us. I was flabbergasted. Once outside, I asked if she was all right. “Oh, he’s like that,” she told me. “Forget it. It’s like dues for working here and getting the good stories.”

I felt that Shad was not only reminding her but also showing me, as a newcomer, who was the boss. Since my dreams had never included a TV career (and I wasn’t a woman), I didn’t feel trapped by him the way my producer undoubtedly did. Unlike Crazy Ronnie, Shad, though taller than me, couldn’t “take” me. Unlike Doc he couldn’t damage my future, one I always assumed was in print, not on screen.

Nevertheless, as much as I could, I stayed away from him, feeling both resentful and diminished in his presence. Somehow, the fact that he was acclaimed as a brilliant television journalist only made matters worse. If that was the case, why did he need to flaunt his power in such a petty and repulsive way?

Several years later, after Shad had briefly left the show for a network assignment, he returned to discover that I was doing a piece on Ernest Hemingway with his temporary replacement, a male senior producer. He called me into his office, slammed the door, and started screaming. How dare I do that behind his back! That was his territory! What did I know about manhood or the sensibilities that he and Papa Hemingway shared? I had flashbacks to other bullies in my life and for the first time began yelling back. I told him that he was a sick man. I used unprintably worse names and even raised my hands to threaten him, which shocked us both. All of a sudden, we fell silent. My hands dropped to my side. He opened the door and, as I marched out, he sneered, “You think that’s going to make you taller?” It was the best he could do. I felt triumphant… briefly.

A few days later, the senior producer told me that our Hemingway story was dead, no further explanation offered. I had no more problems with Shad, but my brief burst of pride in standing up to him faded fast. I hadn’t won, because nothing had changed. He still cursed at and made suggestive remarks about women, just not in my presence. Those who loved the show and needed the job had little choice but to take it. Others left.

I told the story to a male producer, my best friend on the show, a big former college athlete and ex-military officer, who admitted that he felt emasculated by Shad. He had long dreamed of punching him. Like me, however, he ended up just working around him. We never reported the hostile atmosphere he had created or confronted it in any public way. Looking back, I now understand that such behavior was another way of enabling him. Think of us as the bystanders, a situation too common for the majority of men who don’t consider abusing women reasonable behavior. But how many of us could be judged complicit in the post-Harvey flood of accusations against the likes of filmmaker James Toback, Senate candidate Roy Moore, Mad Men creator Matthew Weiner, not to mention a directory of male Silicon Valley types?

Rallying the Bystanders

Such “bystanding” attacks the heart of manhood, or at least what I grew up in the 1940s and 1950s thinking manhood entailed: taking responsibility for protecting those, women in particular, who couldn’t protect themselves from bullies. I understand now that none of this is so simple. Not all bullies cut and run when challenged, and it’s increasingly obvious that women are anything but the “weaker sex” or necessarily want to be protected by men, especially if that protection represents just another example of male chauvinism. The male savior (still relentlessly promoted in pop books and films) is generally just using women as props in his rivalry with other men—a subtler version of what the sexual bullies are doing.

The real job, the hard job, for all of us male bystanders, isn’t to rescue women but to rescue other men from their own worst behavior and so prevent abuse in the first place, be it by a heroic and possibly dangerous personal intervention or the more difficult political mission of, say, passing an Equal Rights Amendment. One of the crises of contemporary manhood and contemporary bystanderdom, perhaps the very reason for our passivity, our cowardice, is the realization that most of the time we can’t even protect ourselves from the men who intimidate us, who take pleasure in treating us like “girls.”

No wonder we, too, derive some pleasure from the precipitous falls of Harvey Weinstein, Leon Wieseltier, Roger Ailes, Bill O’Reilly, Mark Halperin, and the other celebs, politicians, and male eminences going down at the present moment. We sense with satisfaction that they’re finally paying for whatever they did or tried to do to women (or young men) sexually, for reducing them to powerless objects, just as they’ve always done, even if far less directly, to men like us, like me.

Women are finally rising up, spreading the net, pulling it tight. Maybe, at last, it’s our turn as well. So applaud #metoo, then raise #ustoo, where men can begin to be as courageous as the women who brought down the pigs. We have a lot of catching up to do.

We need to keep expanding the “Harvey effect” into all the corrupt and abusive pockets of politics, entertainment, and business, as perilous as that can sometimes be. It’s time, too, to begin bucking the trend of rewarding aggressive men.

And count on it, there will be a bro-lash. There are too many underdogs in the pack who think their fates are entwined with those of the alpha dogs, including those generals like White House Chief of Staff John Kelly in Washington, too many producers and actors in Hollywood, too many hedge-fund vice presidents in New York, too many engineers in Silicon Valley. But there are also millions of us dudes ready to become ex-bystanders, including old guys like me, who need to redeem ourselves from our histories with the Crazy Ronnies, Docs, and Shads, and young guys who need to be woken to the thrilling power of standing up to such bullies.

Someday soon—or so I dream—we’ll march, blow whistles, kick butt, but at least for today we’ll start small. We’ll just say something if we see something.

We’ll just say, “Don’t do that, brother. Be a man.”