I was shocked, shocked, to learn that the United States is spying on the leaders of allied nations -- just about as shocked as the French police captain in Casablanca when he saw gambling going on in Rick's café.

Seriously, though, I really was shocked to hear the U.S. Director of National Intelligence, James Clapper, admit the espionage publicly. Our allies spy on our leaders just as we spy on theirs, he explained to the House Intelligence Committee: "It's one of the first things I learned in intel school in 1963, that this is a fundamental given in the intelligence business is leadership intentions, no matter what level you're talking about."

Racking up more points for honesty, Clapper pointed out the hypocrisy of the legislators, who know perfectly well what goes on yet are treating this as some scandalous new revelation. Like me, he couldn't resist the obvious film cliché: “Some of this reminds me of the classic movie Casablanca -- ‘My God, there’s gambling going on here.’”

Of course U.S. intelligence agencies want to know everything that's done and said, everywhere, even on German Chancellor Angela Merkel's cell phone. That's their job, as they see it. Why else would they get those uncounted (literally uncounted, hidden in a blacked out budget) billions of our tax dollars every year? What's more, if the members of the House Intelligence Committee (or the Senate Intelligence Committee) don't know what's going on, they obviously aren't doing their own jobs competently.

Yet here is Senator Dianne Feinstein, chair of the Senate Intelligence Committee, who generally defends all kinds of government surveillance, expressing her own shock: She does “not believe the United States should be collecting phone calls or emails of friendly presidents and prime ministers.” 

It seems like everyone inside the Beltway is shocked about something or other.

What's really going on is a tangle of conflicting narratives, all evoking strong passions because they hold such deep mythic meanings.

For Clapper and the humongous intelligence bureaucracy he supposedly runs (though it's far too big for him to know even a fraction of what's really going on), the story is simple: Since September 11, 2001, we have been at war. And in wartime spying is a totally acceptable, indeed indispensable, weapon.

It's a story at least as old as the days of ancient Israel, when biblical writers didn't hesitate to say that their ancestors had used spies to conquer the land they now lived in (see Joshua 2:1-7). The Roman empire had its spies, and it's safe to assume that every later empire did the same.

The fledgling United States of America relied on spies to help win its first war, against Great Britain. Nathan Hale is the most famous of those spies. But when the Military Intelligence Corps Association set up an award to "recognize individuals who have contributed significantly to the promotion of Army Military Intelligence in ways that stand out," it didn't name the award for Hale. It created the Knowlton Award, commemorating Hale's commander, Thomas Knowlton. He was picked by George Washington in 1776 to create America's first spy unit, commonly known as "Knowlton's Rangers." Espionage in wartime has been de rigueur for the U.S. military ever since.

But why spy on allies? The simple answer is the obvious imperative to gather as much information as possible, from every source possible. Ideally, every military commander would like to be omniscient, because in wartime, especially, knowledge is power.

But things got more complicated during the cold war. Americans learned to expect an endless war as the new normal; the old line between wartime and peacetime disappeared. The battle was waged by economic, diplomatic, and cultural as well as traditional military means, so the line between civilian and military was blurred, too. Hence intelligence gathering became a constant responsibility shared across that line.

Yet another line was blurred by the cold war, the most important of all: There was no longer any conventional front line between friend and foe. "The commies" could be anywhere -- even behind the desk at your local library, Senator Joseph McCarthy said. And certainly they might have infiltrated the highest levels of allied government offices. So it only made sense to spy everywhere.  

The U.S. still spies on allies all over the world, as the New York Times points out, and not just on government leaders but on "their top aides and the heads of opposing parties" too. "It is all part of a comprehensive effort to gain an advantage over other nations, both friend and foe," the Times bluntly concludes.

That's all the explanation needed for foreign policy elites and their journalistic scribes, who live within the narrative of political "realism." That view had already grown ascendant in Washington before World War II, and it was firmly entrenched by 1962, when James Clapper learned it: In the "great game," every major power is jockeying for advantage. So everyone spies on everyone, as they always have and always will, during times of cold as well as hot war.

Why, then, is so much criticism leveled inside the Beltway at Clapper, the whole establishment he heads, and its ultimate boss, the president. Why would members of Congress, or anyone else, deny what seems so obvious to intelligence professionals and "realists" everywhere?

Much of the answer, I think, comes from the cold war's unique contribution American political mythology. By 1962 the distinctively American mythology of homeland insecurity had become institutionalized as the dominant narrative of the nation. It demanded that official voices in government and media express deep ambivalence toward "realism," embracing it while also rejecting it.

The mythology of homeland insecurity assumes that America is always the innocent nation, trying only to make the world better. So it must reject the idea that America should do what everyone else does. America, it insists, is more moral than everyone else. We have a higher set of national values. We are the standard-bearers of virtue and civilization in a world always threatened by savage evil.

That virtue alone gives us the right to fight evil wherever it appears, by any means necessary. That's the only reason we can use "realist" tactics -- because our goals are definitely not those of the "realist." We want to build up the nation's moral standing, not its brute power.

At least that is our prevailing public narrative. And anything that undermines our public appearance of unique virtue -- like snooping in allies' offices and tapping their phones -- must be denounced, at least in public.

There's another side of the myth involved, too. If our homeland is constantly insecure, we'd better have a leader who is powerful enough to defend us against all the unpredictable threats that may pop up anytime, anywhere -- even inside his own executive branch. 

That's one reason the question "What did the president know and when did he know it?" is so urgent. A president, who is the nation's highest military commander, is like a god. He cannot be omnipotent unless he is omniscient.

To be sure, there's yet another time-honored American myth at play here: The narrative of a government consisting of three coequal branches, each jealously guarding its own powers. Congress must have something to criticize the president about, if only to assert itself.

That's especially true at a time when so many headlines have been trumpeting, in one way or another, the story, "President defeats House." As pundit David Gergen points out, "this is a important turn in the Obama administration's position within American politics. They were really riding high coming out, because the Republicans were on the defensive, you know, and the extremism over the government shutdown. And that narrative has now been replaced by narrative of what did the president know and when did he know it."

Gergen was talking about the question of what the president knew about the weaknesses in the Obamacare software. But his words shed just as much light on the controversy about spying on allies.

As members of Congress know, the mass media are eager right now to magnify any controversy between the president and anyone else. The mass media need their own story, one that will sell. And they know that any narrative of conflict revolving around the president is a guaranteed good draw. It will boost media ratings more than Congress' ratings.

If narrative is the key issue, though, the administration is currently in a stronger place than its critics on the spying-on-allies issue. If the best narrative that the critics can come up with is, "America should be more virtuous than others and the president should know everything that's going on," James Clapper can justly reply:

"I agree. Absolutely. That's what I said, too. We all know that the intelligence community, including the Senate and House committees, is a kind of old boys (and girls) club, much like the casino at Rick's café. We all share those same basic premises. We all know that espionage has always been part of the game. We all know what we can talk about publicly and what we're supposed to keep secret. We've just got a little quarrel about tactics going on here. It's merely a question of how we, and especially our president, can best safeguard our virtue in a world full of evildoers."

When the quarrel is only about tactics, the professionals in the executive branch are likely to best their critics, in Congress as well as outside of it, just about every time.

If the critics hope to gain any real traction in this debate, they'll have to take it to a deeper level and challenge the basic premises of the intelligence community's narrative. They'll have to join the ad hoc coalition of left and right who are challenging the whole idea of government spying.

That coalition has its own narrative, which also goes back to Revolutionary War times. Why did we fight that war in the first place? One big reason, they say, was to get rid of a monarchical system that could invade and control our private lives at any time, on any whim. The glory of the new system was enshrined in the Fourth Amendment to the Constitution, guaranteeing everyone the right "to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures."

Our "effects" surely now include our digital communications. And we didn't fight selfishly just for our own rights, the traditional story says. We claimed those rights in the name of all humanity. So every human, everywhere, is entitled to the same right of privacy in their digital communications, no matter what political roles they might play. 

Yet the "right to privacy" narrative itself will face an uphill struggle as long as the mythology of national insecurity dominates our public conversation. If we are an innocent nation, constantly threatened by "evildoers" who might pop up anywhere, it only makes sense that we must always be on our guard. After all, eternal vigilance is the price of liberty, right? Always has been; always will be. And if we now have digital technology to be vigilant for us, why not use it?

As long as fear of unseen "evildoers" haunts the land, that argument will be hard to refute. 

If advocates of the right to privacy want any chance to set meaningful limits on government spying, on allies or anyone else, they will have to challenge the basic premises of the national insecurity state. They will have to argue not only that privacy trumps security, but that the demands for security have been far exaggerated in an American society that has never really escaped the cold war narrative of constant danger and fear.

Some critics of spying are already making that case, to be sure. But their voices will have to grow a lot louder if anything is really going to change.