The Council on Foreign Relations is usually regarded as a peak institution of the US ruling class. Its fellows design policy and its members, drawn from Wall Street, academia, and elite journalism, hobnob with government ministers and even the occasional president. But its star has fallen with the demise of the old WASP establishment and the replacement of its bipartisan deliberative style with the crude bombast of the present.

It can still attract some marquee names though, even if the quality of the discourse has fallen off somewhat. Last Monday, former Google CEO Eric Schmidt and former Theranos board member Henry Kissinger sat down (via Zoom) to discuss artificial intelligence (AI)—the topic of a new book the two of them have written with Daniel Huttenlocher, the inaugural dean of the Schwarzman College of Computing at MIT. The conversation, like the book, was a strange amalgam of Schmidt’s techno-enthusiasm and Kissinger’s central European gloom that largely accepted the breathless claims of AI promoters as fact.

The project had its origins several years ago when Kissinger happened to hear a conference talk about a computer that had been programmed to play the immensely complex game Go. (Was this the first he’s heard of it?) Kissinger apparently began worrying about what this all meant for the future of humanity, and wrote up his concerns in a 2018 article in The Atlantic. AI, Kissinger declared, meant the end of the Enlightenment (which, to tell the truth, has been looking none too healthy for some time). “Human cognition loses its personal character. Individuals turn into data, and data become regnant.”

Big if true, as they say on the Internet. That machines can be so skilled at playing chess or Go may say more about those games than AI’s potential. Despite their complexity, the scope of such games is extremely limited—and is, for example, nothing next to the seemingly mundane complexity of driving a car.

I’ve been following progress in AI for a couple of decades and the story has always been the same: a handful of successful examples presage a vast payoff that’s always just around the corner—but never quite arrives. Claims for self-driving vehicles are particularly grandiose right now. Hardly a day passes without Elon Musk touting the autonomous driving skills of his Teslas. Reality is quite different.

“Full self-driving” is a long way away, as CNN reporter Michael Ballaban showed just a few weeks ago with his attempt to let a Tesla conduct him safely along a treacherous passage on Brooklyn’s Atlantic Avenue, a densely crowded and chaotic stretch of road whose perils I dread every time I navigate it. Only his intervention kept the car from driving him into an oncoming UPS truck, and that was only one of many near-misses. Ballaban’s misadventures come three years after a self-driving Uber killed a pedestrian in Arizona. That car’s software took several fatal seconds to figure out that what it first thought was a bicycle was actually a person. It finally decided to brake way too late. Evidently the software hasn’t made much progress since.

In their book, Kissinger, Schmidt, and Huttenlocher make a great deal out of GPT-3, the latest iteration of an AI project that can produce words that look very much like human speech. It has many dazzling capabilities, and can even write credible prose like this Guardian article. Well, not exactly. The machine spewed out eight different attempts, which editors turned into a publishable article by picking out “the best parts of each.” And GPT-3’s answers to simple questions are often silly, wrong, and even racist, but those embarrassing parts are rarely presented in public demonstrations, whether out of normal tech boosterism or the desire to impress venture capitalists.

Just for a moment, let’s cede the point that AI is something big that is changing the way we live. Schmidt and especially Kissinger worry about what this means for being human. (It’s weird when the architect of the secret bombing of Cambodia becomes the humanist on the program, but such are the politics of elite organizations.) Over the next 15 years, Schmidt claims, computers will increasingly set their own agenda, exploring paths and producing results beyond the intention or understanding of their human programmers. What will this do to our sense of ourselves, Schmidt asked, “if we’re not the top person in intelligence anymore?”

One response might be, “Well, maybe don’t let them go there?” But the authors will have none of that. “Once AI’s performance outstrips that of humans for a given task, failing to apply that AI, at least as an adjunct to human efforts, may appear increasingly as perverse or even negligent,” they declare. Will we delegate our war-making capacities to machines—not merely in guiding weapons to their targets but deciding whether to attack in the first place? Schmidt apparently thinks so, though he acknowledges that there are some complexities. “So, you’re in a war and the computer correctly calculates that to win the war you have to allow your aircraft carrier to be sunk, which would result in the deaths of 5,000 people, or what have you…. Would a human make that decision? Almost certainly not. Would the computer be willing to do it? Absolutely.”

In a review of the book, Marc Rotenberg complains that all Kissinger’s skepticism from the Atlantic article disappears into the techno-euphoria of Schmidt and Huttenlocher. That’s not entirely accurate; the book does feature the occasional irruption of Kissingerian gloom and ponderous musings on Descartes and Spengler. But it’s mostly true, and he’s right to single out that “perverse or even negligent” passage as symptomatic of the two tech authors’ “unquestioned assertion of inevitability.”

Weirdly, though, at the CFR event Rotenberg asked how AI might be designed to “strengthen democratic principles.” Democracy is supposed to be about transparent deliberation and debate, for which AI doesn’t seem like a good fit. For an answer, Schmidt first deferred to Kissinger, a man whose democratic sensibilities were nicely encapsulated by a remark he made in the run-up to the coup against democratically elected Chilean president Salvador Allende, “I don’t see why we need to stand by and watch a country go Communist due to the irresponsibility of its own people.” On the democratizing AI question, Kissinger admitted having no idea: “I think it must be studied, but I don’t yet know how to approach it.” Picking up, Schmidt acknowledged that we need to do some thinking about the topic but quickly pivoted to criticism of Europe’s penchant for regulation.

Reading the book or listening to the CFR session leaves one little wiser about either the technical realities of AI or its political implications. Kissinger and Schmidt are correct that we need to be talking about these things, but an imperial master architect and what the software engineer and AI skeptic Dwayne Monroe calls “a hype man for a super charged ad firm” are not the ones to be leading the conversation (though Schmidt has left Google he still owns about 1 percent of the company’s stock).

In the words of computer scientist Jonathan Bennett, “the real danger is not that AI will gain superhuman powers but rather that we will be convinced that it has, and we will cede to it important reasoning tasks that it is not capable of performing. As the sophistication of AI parlor tricks increases, that possibility increases along with it.”

AI projects like GPT-3, as Luciano Floridi and Massimo Chiriatti argue, should be viewed as tools that humans can use, and not something seriously resembling human intelligence: “GPT-3 is an extraordinary piece of technology, but as intelligent, conscious, smart, aware, perceptive, insightful, sensitive and sensible (etc.) as an old typewriter.” Retaining such perspective, rather than surrender to the inevitable dominance of our silicon masters, is the place to begin.

One consolation: It was satisfying to hear Kissinger the day after a socialist was elected president of Chile—in part by promising to undo the five decades of reaction launched by the coup Kissinger did a lot to plan. I wanted to ask him about that, but only CFR members are allowed to ask questions at these events. Journalists are there only as passive receptacles.