In the last weeks of 1954, Eric Hobsbawm and a small group of British historians set out on a goodwill trip to Moscow. It was a strange time to be visiting the Soviet Union, even stranger for a communist eager to see the achievements of actually existing socialism. Stalin had died the year before, and his corpse lay embalmed in a glass box in Red Square. After a vicious power struggle, Khrushchev had gained control of the government, but intrigue abounded. Beria, the longtime head of the security services, had been tried and executed in secret. Molotov and Malenkov, stalwarts of the old regime, were on their way out. Tens of thousands of prisoners, released after Stalin’s death, were returning from the gulag with horror stories of starvation and torture.

At first, nothing seemed amiss to Hobsbawm and his traveling companions. On their arrival in Moscow, they surveyed the city’s elaborate subway system, before being whisked to Leningrad in the sleek overnight cars of the Red Arrow. Returning to the capital after a matinee performance of Swan Lake, they rang in the New Year with the country’s leading scientists over canapés and champagne. But as the historians settled in, they began to suspect something was wrong. The intellectuals they met were tight-lipped and wary of private conversations. The group’s minders from the Soviet Academy of Sciences appeared almost entirely cut off from Western scholarship. When the historians wandered outside the city center, they found outer Moscow gray and patched over. On one derelict street, they spotted a group of “middle-aged women, presumably war widows, hauling stones and clearing rubble.” “As intellectual VIPs,” Hobsbawm later observed, “we almost certainly were treated to more culture…as well as an embarrassing share of products and privileges in a visibly impoverished country…. It was not a good advertisement for communism.”

On his return to England, Hobsbawm was treated to an even worse set of revelations. Early in 1956, Khrushchev gave his speech outlining Stalin’s brutal reign of terror. In the summer, Polish workers went on strike in the industrial town of Poznan, launching a wave of protests against the Soviet-imposed government, and then were gunned down by the army. In the fall, after a revolution in Hungary led the country to leave the Warsaw Pact, Khrushchev sent in the tanks. Thousands were killed, the prime minister deposed, and the country forced back into the Soviet fold.

For a growing number of Hobsbawm’s close friends—Christopher Hill, E.P. Thompson, Raphael Samuel, Rodney Hilton—enough was enough: Enraged by Soviet actions and by the British Communist Party’s refusal to denounce them, they broke with communism once and for all. Hobsbawm, however, held on. He, too, was horrified by the events in Hungary and by the brutality documented in Khrushchev’s speech, and he publicly lamented how communist parties on both sides of the Iron Curtain had “dropped what measure of democracy [they] might originally have contained.” But as more and more of his comrades jumped ship, Hobsbawm remained—and for another 40 years. Only after the Soviet Union’s collapse did he let his party membership lapse, and even then he insisted that the “dream of the October Revolution is still there somewhere inside me.” Citing one of Bertolt Brecht’s poems—“We, who wanted to prepare the ground for kindness / Could not be kind ourselves”—he explained: “Hardness was forced upon the revolutionaries.”

Hobsbawm was one of the 20th century’s most important historians. Arguably, he was its most important in the English language. His epic series on the making of the modern world—The Age of Revolution, The Age of Capital, The Age of Empire, and The Age of Extremes—introduced millions of readers to the history of the 19th and 20th centuries. His research on banditry, nationalism, and the “invention” of tradition helped inspire entire fields of British and American historiography. His prose was lean and elegant, uncluttered by the theoretical fads of the day. His historical judgment was infused with the keen insights and irony of a historian who seemed to “know everything.” Yet part of Hobsbawm’s enigma is that, while he wrote with such an acute sensitivity about the modern age’s contradictions and tragic reversals, he never seemed to fully come to terms with those that bedeviled his own convictions. “Why,” Perry Anderson once mused, “did he stay to the bitter end?”

Richard Evans’s new biography, Eric Hobsbawm: A Life in History, offers some clues to this puzzle while also making the case that this might not be the right question to ask. Compellingly narrated and meticulously researched—among other things, Evans draws from a half-century of MI5 surveillance reports—the book provides a more nuanced portrait of Hobsbawm’s political and intellectual development, revealing that Hobsbawm was a far more ambivalent communist and a far more pragmatic socialist than either his critics or his champions recognized.

Haunted by the ghosts of 1930s sectarianism, Hobsbawm campaigned for the Labour Party throughout the second half of the 20th century—and not just its radical factions. After the ordeals of 1956, he participated in an effort to democratize the British Communist Party and, when that failed, he abandoned nearly all party activities. Finding succor in the politics of Latin America and Western Europe, he spent much of his later years championing those practical socialists willing to build coalitions with liberals and organized labor. For Hobsbawm, a popular front was more than a defensive strategy; it was the very basis upon which egalitarian change could occur.

Eric Hobsbawm was born in Alexandria, Egypt, in 1917, just five months before the Bolsheviks came to power in Russia and a year and a half before World War I ended. His father came from London’s working-class East End and his mother from a well-to-do Austrian-Jewish family. The two had met in Alexandria’s colonial district, got married and honeymooned in Switzerland, and then set up a comfortable Victorian home in the city, complete with a nanny for Eric. By all accounts, their time in the city was a pleasant one. Eric’s father worked in the telegraph services, and his mother began her career as a novelist and translator. The period was also short-lived: In 1919, an anti-colonial rebellion broke out, and the Hobsbawms left for his mother’s Vienna, never to return.

There, in the twilight world of a formerly grand metropole, Eric and soon a younger sister grew up. Street fighting and threatened coups were everyday events. So was economic hardship: No one—not even his mother’s haute family—had any money, and the Hobsbawms struggled to make ends meet. Things only got worse after 1929. Returning home from a wasted day in search of money to earn or borrow, Eric’s father died of a heart attack. Soon after, his mother fell ill and died. At the ages of 14 and 10, Eric and his sister had become orphans.

From then on, the Hobsbawm siblings lived a peripatetic and threadbare life, and in a world that was about to buckle under the weight of economic devastation and an ascendant fascism. “We were on the Titanic,” Hobsbawm later recalled, “and everyone knew it was hitting the iceberg.”

After months of moving between relatives, the pair ended up in Berlin, where an uncle took them in. In Berlin, where “the world was visibly breaking down,” Hobsbawm became a communist. The choice was easy: In a city teetering between two revolutions—one fascist, the other socialist—what else would he choose? Some of his Jewish peers had found hope in a third alternative: the quest for a Jewish state. But having already passed through two countries ravaged by nationalism, and now caught in a third, Hobsbawm had concluded that the only revolution for him was that of worldwide liberation.

Largely unsupervised, Hobsbawm threw himself into a flurry of agitation. He slid leaflets under apartment doors and joined in the massive citywide marches. He dodged brownshirts on street cars and hid a banned mimeograph under his bed. He was troubled by his fellow communists’ sectarian stance toward the social democrats, later observing that it “bordered on political insanity.” But he maintained party discipline. Above all else, Hobsbawm had found a new family. “We belonged together,” he recalled.

In 1933, the Hobsbawm siblings were forced to move once again—this time to London. A British citizen and a native English speaker, Hobsbawm found the move more of a homecoming than an exile, and he immediately took to the country. An avid cyclist, he spent holidays biking through the hilly countryside. (“If physical mobility is an essential condition of freedom,” he later noted, “the bicycle has probably been the greatest single device for achieving what Marx called the full realization of the possibilities of being human…since Gutenberg.”) He grew close to his London relatives, in particular an uncle who was a local Labour councillor, and he “gulped down” as much English poetry and fiction as he could find.

Within three years, Hobsbawm won a place at King’s College, Cambridge. Overjoyed by the news, he marked the occasion in his diary with a parodic self-portrait:

Eric John Ernest Hobsbaum, a tall, angular, dangling, ugly, fair-haired fellow of eighteen and a half…. Some people find him extremely disagreeable, others likeable, yet others (the majority) just ridiculous. He wants to be a revolutionary but, so far, shows no talent for organization. He wants to be a writer, but [is] without energy…. He is vain and conceited. He is a coward. He loves nature deeply. And he forgets the German language.

He even allowed himself a little bit of optimism: “Perhaps, just maybe, I shall live a less ‘second-hand’ life?”

Enrolling in 1936, Hobsbawm found Cambridge to be at once parochial and exhilarating. It was full of cloistered quads, lawns that one couldn’t cross, and the lazy and half-literate children of England’s ruling class. To his surprise, it also turned out to have “the reddest and most radical generation in the history of the university.” With fascists threatening to overrun Europe, one had to take sides—even at a distant college campus.

Helping lead the Communist Party’s student branch, Hobsbawm joined the university’s “nursery of revolution” set up in a suite of rooms just below Ludwig Wittgenstein’s. With the bitter memories of sectarian Berlin still fresh in his mind, he worked hard to broaden the branch’s activities and forge alliances with socialists, liberals, and other left-wing factions on campus. “We had,” he recalled, “only one set of enemies—fascism and those who (like the British government) did not want to resist it.”

Hobsbawm also thrived socially and intellectually. At the center of red Cambridge, he quickly gained a reputation among students and faculty as the “freshman in King’s who knows about everything.” He discovered an affinity for history and was particularly taken by the world-spanning work of Marc Bloch and the French Annales school. By his third year, he was the editor of The Granta, then Cambridge’s leading student magazine; was elected to the Apostles, where he dined with the likes of E.M. Forster; and won a fellowship to study the history of colonial North Africa.

Hobsbawm’s swift advancement through the ranks of England’s intellectual and socialist elite, however, was cut short by World War II. The year he graduated, Germany invaded Poland, and he was conscripted into the British Army. He had hoped he might enlist his fluency in German and French in the struggle against fascism, and he lobbied for an intelligence appointment. But the British military had other plans: Putting him under surveillance for his communist activities, the army relegated him to an inland sappers’ unit and then to an education division. “I had neither a ‘good war’ nor a ‘bad war,’” Hobsbawm later observed, “but an empty war.”

His war years did, however, give him the opportunity to develop an interest in popular history. Assembling courses for the army’s mostly working-class conscripts, Hobsbawm discovered the pleasures of teaching and writing for a general audience. Impressed by his fellow soldiers’ organic “sense of class, comradeship, and mutual help,” he changed his area of specialization and embarked on a study of British labor. Passed over for a series of posts at Cambridge—likely because of his communist affiliations—he continued to develop his interest in popular working-class history after the war. In 1947, he joined the faculty of London’s Birkbeck College, a night school for working adults. He remained there for nearly the rest of his life.

Despite having found gainful employment, Hobsbawm’s postwar years were bleak. During the war, he had married a fellow communist, a young woman studying at the London School of Economics. But after his return from the army, their marriage fell apart. His first two books—one on the history of wage work, the other on the Fabians—were rejected by publishers for being too radical. The British security services did not lessen their surveillance of him after the war; they increased it. He suspected his doctoral adviser, M.M. Postan, of sabotaging his job applications with “poisoned-arrow” recommendations.

The sectarian direction of the Communist Party after the war also proved dispiriting. No longer galvanized by wartime emergencies, the party’s leadership abandoned its popular-front stance and began to wage an active campaign against Labour to split votes. “What do we expect to get out of the contest,” Hobsbawm wrote in an angry letter to the Daily Worker, “except the chance of making individual attacks on the leader of the Labour Party?”

Hobsbawm did discover new circles of friends, though, which helped lift his spirits: Falling in with a group of dissenting communist historians—Christopher Hill, E.P. Thompson, Dorothy Thompson, Rodney Hilton, Raphael Samuel, Dona Torr—he founded the Communist Party Historians Group and the journal Past & Present with the aim of helping popularize working-class history. Spending time in London’s West End jazz clubs, he also found solace and comradery with its “quasi-underground international freemasonry” of avant-garde musicians, and he began writing a series of pseudonymous music columns for The New Statesman.

Hobsbawm’s time with these two groups also inspired his first two published books, which, given his meteoric rise among England’s intellectual elite before the war, came at the rather late age of 42. In 1959, he published The Jazz Scene, a wide-ranging social and cultural history of the musical form. The same year, he also published Primitive Rebels, a considerable work of sociological and historical inquiry that examined those forms of working-class resistance—Andalusian anarchism, Italian banditry, British Luddism—long ignored by historians on the left.

Neither book made Hobsbawm a household name, nor did either bring him large sums of money. But both did help him hit his stride as a historian and an intellectual, opening him up to new worlds of working-class and radical life. Together with the other books produced by the Historians Group—Christopher Hill’s The World Turned Upside Down, E.P. Thompson’s The Making of the English Working Class, Dorothy Thompson’s The Chartists—Primitive Rebels also helped to revolutionize how English history was studied on both sides of the Atlantic.

Upon its initial publication, Primitive Rebels must have seemed like an odd book for a Cambridge PhD to write, even one with admittedly radical commitments. The study of history at that time was divided mostly along two lines: political histories that mapped the rise and fall of nations and empires through the elites—kings, prime ministers, generals—who led them; and economic histories that examined the institutions and competing interest groups that defined a particular nation’s or empire’s commercial system. Marxist historians, as well as the liberal ones, divided along these lines: The few Marxists writing at the time were studying either the revolutionary vanguards who sought to transform society by seizing the state, or the larger economic forces that helped lay the foundations of modern capitalism.

Primitive Rebels attempted something altogether different: As in The Jazz Scene, its protagonists were people on the margins, displaced by modernity. They were primitive not because they lacked sophistication or coordination—Hobsbawm was careful to show the opposite—but because they did not conform with how most socialists and liberals understood modern politics. His rebels did not want to seize the state; they mostly wanted to sustain their ways of life below it. They were the Spanish farmers forming agricultural collectives, the English journeymen smashing spinning frames, the Sicilian peasants taking up arms in self-defense.

The politics of Primitive Rebels may have been hard to decipher at first, given its departure from Marxist as well as liberal historiography. But as with the rest of the Historians Group’s work, it stemmed from a desire to recover those sites of working-class agency that took place below the titanic clash between elites and competing economic structures. No longer convinced that a disciplined party was the only way forward, he and his peers wanted to retrieve from the annals of history those traditions of resistance that fell outside the purview of the socialist left, whether in its revolutionary and Bolshevik variety or its reformist and Fabian one.

1956 was at the center of this project. The growing bureaucratization of communist and social-democratic states after World War II had driven Hobsbawm and the Historians Group into the archives in search of more democratic forms of radicalism, and the postwar collapse of the popular front spurred them even further. But it was the traumas of 1956—Khrushchev’s speech, Poznan, Hungary, Britain’s involvement in the Suez Crisis—that catalyzed their sense of mission: Neither North Atlantic social democracy nor Soviet-bloc communism appeared faithful to the left’s ideals. Out of a usable past, they hoped, might come a more egalitarian present, hence the name of their journal: Past & Present.

A key strength of Evans’s biography is that he shows how much this was the case for Hobsbawm as well as for those who left the party. After Khrushchev’s speech, Hobsbawm co-authored an angry letter denouncing the British Communist Party for its “uncritical endorsement of all Soviet policies and views.” In the wake of the Hungarian uprising’s violent suppression, he signed a public statement decrying the slavish “support given by the Executive Committee of the Communist Party to Soviet action in Hungary.” Refusing to be disciplined, he helped launch a campaign to reform the British party from within, insisting that “the test of inner-party democracy is, whether policy and leadership can be modified from below.” When that failed, he tried to have the party’s leadership ousted and then abandoned nearly all formal party activities. “Hungary proved the last straw for E[ric],” one Historians Group member reported in a letter intercepted by British security services. Hobsbawm had become an “opportunist” and a “dangerous character,” complained another. Even MI5 began to take note of “hobsbawm’s own shaky party affiliations.”

A strange game of chicken ensued after 1956. At times, Hobsbawm wanted the party to throw him out. At others, the party wanted him to throw himself out. Over the years, Hobsbawm gave different answers for why he remained: loyalty to a cause he’d been dedicated to since he was a teenager; solidarity with those rank-and-file communists who had sacrificed so much; revulsion at the ex-party members who had become Cold War hawks. He also confessed a more personal reason—pride. “It would have been easy to slip out quietly,” he later admitted. “But I could prove myself to myself by succeeding as a known communist—whatever ‘success’ meant…. I do not defend this form of egoism, but neither can I deny its force.”

No matter the reasons for his public resoluteness, Hobsbawm’s experiences as a communist in the first half of the 20th century dramatically reshaped his political and intellectual activities in the second half. In a blistering essay published several years before both men died, Tony Judt insisted that this was for the worse: Hobsbawm’s near-lifelong affiliation with communism had “provincialized” him as an intellectual and narrowed the scope of his insights as a historian. While Hobsbawm “can acknowledge his mistakes readily enough…,” Judt argued, “he doesn’t seem to understand why he made them.” Paying closer attention to Hobsbawm’s political and intellectual development after 1956, Evans’s biography makes a compelling case that the opposite was true: Hobsbawm’s bitter years as a communist in the first half of his life only helped him become a better historian and a more practical socialist in the second half.

Beginning with Primitive Rebels, Hobsbawm’s social histories—Labouring Men, Bandits, Captain Swing, Worlds of Labour—were driven by a newfound purpose: He wanted to save those forms of unconventional radicalism not only from what E.P. Thompson called the “enormous condescension of posterity,” but from the enormous condescension of party apparatchiks. After 1956, he also found his Marxism swiftly outpacing his communism. Discovering Gramsci and taking to heart Marx’s dictum that “men make their own history, but they…do not make it in the circumstances of their choosing,” he began to see the struggle for human emancipation as far more multifarious than Lenin and the Bolsheviks had allowed. Instead of a zero-sum game between communism and capitalism, egalitarian politics rested on a continuum: Everything from America’s welfare state to Western Europe’s social democracies to Yugoslavia’s third camp could fit into the broad sweep of human progress. “Were we right in believing that there was only one way, that there was one railroad that alone led forward?” he asked in a 1978 interview. “The answer is no. There were all sorts of other things happening that we should have taken note of.”

Hobsbawm’s epic trilogy on the “long nineteenth century”—The Age of Revolution, The Age of Capital, and The Age of Empire—magnified this insight across the entirety of a century. The profound failures of the October Revolution and the Western communist parties were only echoes of a larger pattern of progress and reaction that shaped all of modern history. The “dual revolutions”—French and industrial—that opened the long century overturned the stifling hierarchies of the ancien régime, but they then imposed their own forms of domination. The formation of constitutional democracies in the 1860s and ’70s liberated millions from the grip of absolutism, but self-government in Europe was then subsidized by the brutal colonization of much of the rest of the world. Every right turn appeared to be followed by a wrong one.

This is what made Hobsbawm’s narrative so monumental: His Age of series was his War and Peace. Narrated from on high and synthesizing vast tracts of research, each volume was infused with a novelist’s eye for the paradoxical and disorderly nature of human progress. The century that began with the bright light of emancipation had concluded in the darkness of empire and world war. The era’s industrialists and imperialists were to blame, but so too, Hobsbawm insisted, were the revolutionaries who opened the age with a fury of sectarian violence and the liberals and social democrats who helped close it by banging the drums of continental war.

This insight also changed Hobsbawm as a political actor, opening him to a far wider array of egalitarian movements and causes. In the early 1960s, he took part in the Trafalgar Square sit-downs against nuclear arms and in the late 1960s, he found himself caught up in the Paris protests, which, after some initial skepticism, he heralded for their “revolutionary potentialities.” He only visited the Soviet Union once more as an official guest after his first trip, but he traveled frequently to Latin America, establishing close ties with socialist intellectuals and politicians throughout the region, including, among others, future Brazilian President Lula da Silva. When, in 1970, Allende’s Popular Unity government came to power in Chile on a rising tide of left-wing collaboration, Hobsbawm celebrated it as “a thrilling prospect.”

Where local communist parties proved willing to open up, Hobsbawm lent them his name and energy, meeting frequently with a new generation of Italian communists seeking to break from their party’s sectarianism. In those countries where socialists and social democrats were making headway, Hobsbawm also proved ecumenical, finding in the coalitions of France’s François Mitterrand and Spain’s Felipe González the possible beginnings of a new popular front.

At home, too, Hobsbawm championed a popular-front politics, finding common cause with Labour’s Michael Foot and then, to the frustration of many of his socialist comrades, with Neil Kinnock, who was moving Labour to the right. “Unity of all progressive and democratic forces was needed,” he had come to believe, “if Thatcher’s conservative revolution was to be stopped.”

Hobsbawm still retained some of the blinkered limitations of an early 20th-century Marxist: He was frustratingly indifferent to the feminist activism that burst onto the scene in the 1960s and ’70s, and, in his later historical work—especially on the “short twentieth century”—he had a tendency to focus more on the tectonic collisions happening at the top of history at the expense of those everyday struggles happening below. There also was the ever-present danger that his popular-front politics might cast too wide of a net in its pursuit of power, allying with those on the left who were more than willing to abandon their party’s traditional egalitarian programs.

But the bitter lessons gleaned from Hobsbawm’s years as a militant had nonetheless left a profound mark: For Hobsbawm, the left would only succeed if it found a way to transcend its ideological differences and build large multitendency movements. “The popular front strategy,” he explained in an article from 1985, “was more than a temporary defensive tactic…. It was also a carefully considered strategy of advancing to socialism.”

Writing about Keynes, Hobsbawm once observed that the economist had found himself forced to radicalize his liberalism in the wake of capitalism’s early-20th-century failures. The same could be said of Hobsbawm: Faced with the failures of early-20th-century communism, Hobsbawm found himself forced to liberalize his socialism. This was, he insisted, what Marx would also have done—“to recognize the novel situation in which we find ourselves…and to formulate not only what we would want to do, but what can be done.”

Like the dialectical swings between progress and reaction that inaugurated the modern world and that continued to transform it in the centuries that followed, Hobsbawm’s own biography can be divided into two separate and opposing movements: From 1917 to 1957, he lived an itinerant, scattered, and often sectarian life; from 1957 until his death in 2012, his politics and life both began to gain a centripetal force. After years of not publishing, he brought out a new book almost every other year until he died—more than 30 in all. His Age of series was translated into dozens of languages, and his social and economic histories were central to the transformation of English-language historiography. He married again, this time happily. He had children, purchased a house, and summered in Wales. He became a near celebrity in many parts of Latin America, South Asia, and Western Europe. Having played a marginal role as a communist militant in the first half of the 20th century, he reinvented himself as a globe-trotting intellectual and “guerrilla historian” in the second half, happily lending his support to those on the left building broad-based movements and coalitions.

Hobsbawm tended to characterize his own “short twentieth century” as a catastrophic age divided between the extremes of a socialism that had gone terribly awry and a capitalism that seemed permanently entrenched. “Never did the pattern of progress or continuous change appear less plausible,” he asserted in the concluding pages of The Age of Empire. But one of the considerable achievements of Evans’s new biography is that it helps us tell a different story—both about Hobsbawm’s life and about the century he lived in. Rather than an era defined only by wasted ideals and sectarian extremes, the Age of Hobsbawm was also shaped by a surprising number of moral and political advances. Whether in the North Atlantic or the decolonized world, the long middle third of the century saw new popular fronts arise in pursuit of more democratic and egalitarian societies. Women, embattled minorities, colonized peoples, immigrants, and disenfranchised workers all over the globe won new freedoms for themselves, and many of their achievements still stand today.

Hobsbawm often lamented that the agitations of his primitive rebels did not leave behind any lasting institutions for the present. But the unconventional radicals of the 20th century—Hobsbawm included—did leave a mark, giving the left an image of a wide-ranging egalitarianism with which to challenge the reigning inequalities and injustices of its day. The popular fronts that it inspires will certainly not look like those of the past. They will face new challenges, and they will be forced to make history in their own way. But then again, none of us gets to act in the circumstances of our choosing.