In big cities like Chicago, New York, or Los Angeles, a farmers’ market might not be a center of economic and social life. But in Bloomington, Indiana—with a population of 80,000 when Indiana University is in session—the farmers’ market has run for 45 consecutive years, and it’s a big deal.

That puts in perspective the City’s decision to cancel the August 3 and August 10 markets amid rising tension over the presence of a vendor, Sarah Dye, an active member of the American Identity Movement (formerly Identity Evropa). AmIM is a Southern Poverty Law Center–designated white supremacist organization; Dye also has friends in white supremacist circles, such as Brien James, a prominent skinhead who allegedly nearly stomped someone to death for refusing to “Sieg heil!”

Dye denies being a white supremacist and refers to herself as “an identitarian.” According to the SPLC, identitarians want nations that are “culturally and ethnically homogenous within their borders,” and the identitarian “label is simply a cover for rampant racism and antisemitism.” The goal of the term, according to the SPLC, is to “smuggle white nationalist views into mainstream politics.”

How does an event centered around heirloom tomatoes, improvised folk music, and kids gorging on kettle corn devolve into a source of potential violent conflict? The answer speaks to a nationwide problem local governments and law enforcement have with confronting white supremacy directly, as they do readily with other issues they identify as threats. Instead, they claim impotence, citing a misguided belief that the First Amendment ties their hands, until the situation escalates to a crisis point.

The City of Bloomington and Mayor John Hamilton sounded the right notes in their July 29 announcement canceling the market. They condemned Indiana’s permissive gun laws and President Donald Trump’s racist rhetoric for creating a dangerous environment. Yet the City, police, and market organizers had been aware since at least 2017 that Dye and co-owner Douglas Mackey were alleged white supremacists who had generated at least three reports of harassment and one for a “physical altercation” with a shopper.

As activists and residents raised concerns, officials responded with hand-waving. Kyle Billman of Goldleaf Hydroponics, a longtime market vendor, said the City should at least have “immediately made a public statement and taken a firm stance that no hate group’s presence will be tolerated at the market or in the city of Bloomington.” Instead, as Bloomingtonian Teal Lynn said, “the City responded via the local paper that they could do nothing because of the First Amendment.” This flawed understanding of the First Amendment apparently negated officials’ responsibility to take seriously reports of threats and harassment.

The online presence of Mackey and Dye is an extensive record of their involvement in violent extremist groups. Even after the FBI interviewed the pair in relation to a fellow Identity Evropa member’s vandalizing an Indianapolis synagogue (which received national news coverage), Bloomington officials declined to take stronger steps.

Matters came to a head during July of this year. The tenor of the Trump presidency and the increasing boldness of white nationalist violence increased activists’ attempts to get the city and market organizers to do something. Inaction bred frustration, and on July 19 black-clad counterprotesters confronted the vendor in a silent, nonviolent protest during the market. In response, xenophobic groups including AmIM and the Three Percenters appeared in force on July 26, flashing white supremacist signs and circulating threats of violence. Knowing that these groups are heavily armed and given the laxity of Indiana’s gun laws, city officials canceled the event, citing “credible threats.”

Notably, Bloomington police did finally make an arrest on July 27; they arrested Cara Caddoo, an Indiana University professor, for holding a sign near the white supremacists’ booth. This selective deference to First Amendment rights versus the need to make arrests did not go unnoticed. Abby Ang, the activist behind No Space for Hate Bloomington, said, “What happened in El Paso proves that white supremacists don’t care about what is lawful. By quibbling about where to carry a sign or where to hand out flyers, it’s very easy to avoid the more difficult question of what to do with the presence of white nationalists, and how to keep marginalized peoples safe.”

Officials seem to recognize the gravity of the danger without understanding how to respond now that the situation has metastasized. “The cancellation was proof that they did take the threats [from white supremacists] seriously,” Ang said, “but I am concerned about what the City will do next. White supremacy isn’t going to go away in a couple of weeks.” That sentiment was echoed by others like Mallory Rickbeil, a politically active Bloomingtonian who said event organizers “chose to cancel the market rather than take necessary steps to deal with the presence of white supremacists because they believe, falsely, that everything can and will go back to normal.”

If this sounds familiar, it is because Bloomington, Indiana, is far from the only American town where this dynamic of under-response to legitimate threats and over-response to counterprotests exists. Police and local governments routinely create and legitimize legal rationales for aggressively targeting some groups—the homeless, people of color, anti-hate activists, people in possession of recreational drugs—yet when white nationalism and racism are involved, they feign powerlessness until the situation grows out of hand. When the community organizes counterprotests, law enforcement rediscovers its confidence to act. The protesters, not the people they protest, are defined as the problem.

We see examples of this selectively vigorous approach everywhere. In El Paso, where a white supremacist just murdered 21 people in a Walmart, police arrested and then immediately released a young white man driving around a migrant center flashing a knife and loaded gun. Imagine a black male of similar age brandishing a loaded gun in public; it is inconceivable that the police would ask him a few questions and just send him on his way. Similarly, mailing a written threat to the El Paso police stating an intention to commit a mass shooting would hardly be treated as First Amendment–protected speech. Yet, rather than responding to the obvious threat, officials stick their heads in the sand and pretend that an armed man flashing weapons at a crowd has “rights” against which police are powerless.

This is where law enforcement, elected officials, the media, and much of the country get it wrong about white supremacy. The ideology is violence, and its adherents are by definition a threat. Their presence makes other people—people of color, people of targeted religions—unsafe. White supremacy is treated as a difference of political opinion instead of as terrorism. People who devote their lives to purging “their” country of nonwhites are a threat; their presence at a community event makes others unsafe. People may reasonably differ on political issues; whether or not people of color have a right to exist is not one such issue.

It is hard to criticize the City of Bloomington for canceling a community event when armed white supremacists started showing up. The problem is that the situation could have been addressed long ago, long before it got to the point where a fun event turned into a potential flashpoint for the next mass shooting. The same ways the City would prevent a street gang or a terrorist organization from participating in such a community event—on the grounds that they are dangerous groups that regularly engage in violent crimes—could and should have been leveraged against people involved in Identity Evropa.

Instead, the City did nothing. Doing nothing emboldens white supremacists, and tacitly communicates that their actions will be tolerated up to a clear line: Just don’t shoot anyone, and we’ll leave you alone. When you do shoot someone, we’ll act like we never saw it coming. Bloomington claims it did all it could, but what other group known to advocate and engage in violent crime is treated with the same hands-off approach? Police departments throughout Indiana have special “gang task force” units to aggressively police gang activity, including profiling known members. It is unclear how many white supremacist groups must commit a crime before they warrant the same treatment.

It should also be noted that white supremacists are a presence in many police forces, as the FBI and other credible investigations have routinely warned. Failure to aggressively police white supremacists looks a bit suspicious compared to law enforcement approaches to other violent groups. It is hard to imagine, for example, two enthusiastic Islamic State or MS-13 fans selling corn at the local farmers’ market without a response from police and local government.

That is where one Indiana city fell short, and where the rest of the nation frequently falls short too. We need to stop thinking of white supremacists as people with dissenting political opinions and start recognizing them as the violent terrorists-in-preparation they are. Our laws and our system embrace a panorama of ideas and viewpoints, but “I want to kill and rid the country of people who look like you” is not one of those. Groups like AmIM take great pains to describe themselves as “peaceful,” but every day we see more evidence that their ideology is anything but.

If the City of Bloomington did cancel the events in the hope that things would magically go back to normal on their own, that seems unlikely. A week after the mayor’s announcement, Ku Klux Klan “Neighborhood Watch” flyers appeared in neighborhoods. Having been given the message that terrorizing the community is their First Amendment right, it cannot be surprising that white supremacists do as they please without repercussions.

Correction: A previous version of this article incorrectly stated that Bloomington police arrested Indiana University professor Cara Caddoo on July 26. The arrest took place on July 27.