The problem of knowing

Hello friends, welcome the 42nd episode of Conflicts With Interest. I had warned you all that my frequency of writing might start to slip and lo and behold, I wasn’t lying to you. It’s been at least six weeks since my last episode, maybe more depending on when I get this out.

But absence makes the heart grow fonder I’m told and we’re here together now so let’s not dwell on the decreasing frequency of Conflicts With Interest.

In this episode, I want to grab a problem by the horns that I have had cause to think about a lot and that is the significance of intelligence in the espionage sense of the word.

Popular culture has for decades found fertile ground in the world of intelligence. Jason Bourne, James Bond, George Smiley. These characters and others have sold billions of dollars' worth of books and movie tickets.

As a consequence, most people have some preconceived notion of what it is that intelligence agencies do. And if you strip those preconceived notions bare, to their most elemental and abstract states, often you will come across the idea that, among other things, intelligence agencies are supposed to influence the future by enabling better decisions.

For example, intelligence agencies should provide forewarning of terrorist attacks so they can be prevented and people’s lives saved. Or they should get a hold of an enemy state’s invasion plans so that when the invasion does come, the agencies’ own countries have their defenses in place. Or perhaps they are supposed to steal some kind of technology on behalf of their country which is an alteration of the future insofar as it makes a country's economy grow faster than it otherwise would have.

Ultimately, these are all different forms of shaping the course of events. And the hint is right there in the name. It’s called intelligence because one hopes it will enable leaders to make smarter decisions and shape the future in a way beneficial to the holder of the intelligence.

But working against this conception of intelligence are the many examples where the intelligence was good, but it either didn’t affect the outcome, or didn’t affect the outcome in the way we would expect looking at the events after the fact.

In recent years, we have the wars in Ukraine and Gaza as examples of this. But we also have historical examples too. I have been reading a lot about the Civil War recently, and that might become an episode of its own. But for the purposes of this episode, suffice to say the Civil War has good examples where intelligence was not used as we would expect.

We will get into all of these in good time, but first I want to lay out the scope of the problem. You can simplify the issue of intelligence failure and success as it relates to decisions into a simple 2 x 2 matrix.

Along one axis, you have whether the intelligence itself is good or bad. The simple interpretation of intelligence being “good” is that it’s right. But more concretely, good intelligence is accurate, timely and well-sourced which means it comes from someone or something that has proved reliable in the past. Bad intelligence is the opposite of those things.

On the other axis is whether the downstream decisions based on intelligence are good or bad. In other words, a decision maker has the good leads, the Glengarry leads, but are they able to close? Just a little Glengarry Glen Ross reference for you there.

It’s the combination of good intelligence / bad decisions that fascinates me the most for two reasons. First, it’s the most vexing. How can people who have risen to the top of national politics or the military still make bad decisions with good information?

Second, I actually think it’s where more of the action takes place as it relates to the use of intelligence than we realize. My interpretation of history is that very few major events were truly unforeseeable. The British diplomat Eyre Crowe outlined the basic tensions between Great Britain and Germany which would lead to World War 1 seven years before the outbreak of hostilities. Time and again, Hitler made clear that Germany had expansionist goals in Western and Eastern Europe. Many such cases.

I think at some level, this relates to a distinction that I draw between soft intelligence, and hard intelligence. Soft intelligence refers to fuzzy questions like “what is my adversary thinking?”. Hard intelligence refers to questions that usually have discrete answers such as “How many troops has my adversary?” or “How far can my adversary’s missiles fly?”.

Enormous energy and resources are expended on collecting soft intelligence. But it faces many inherent challenges. Let’s imagine we are trying to determine whether an enemy leader plans to launch a military attack.

The first problem is the enemy’s mind can change. Today, they might be planning an attack. But tomorrow they may change their mind. If you receive both reports as intelligence, how do you interpret them?

The second problem is the enemy themselves may not know what they think. Imagine you have a well-placed source close to the enemy leader who is privy to a conversation in which the enemy leader says they think a military attack is required. Is that a sincerely held belief? Or is the enemy leader saying that to elicit the views of their advisors?

Consider a more ambiguous example of the same situation. Imagine that the source is in a meeting where the enemy leader is asking detailed questions of their generals about an attack. Not just how many troops does each side have but specifically how and where the generals would attack? How long would it take? What is their level of confidence? And so on.

Such behavior could easily be interpreted one way or the other by the source. Either the detailed questioning is the leader making sure all the plans are buttoned up for an attack they want to launch, or they are convincing themselves they don’t want to launch an attack because they’re unconvinced by the military options available to them.

In theory, the way this tension gets resolved is by accumulating multiple data points over time. There are multiple problems with this in practice. The first is that getting well-placed sources close to enemy leaders is very hard! For KGB and CIA officers during the Cold War, getting once such source might be the sum total of their professional working lives.

This creates a downstream problem which is statistical in nature. For multiple data points to resolve the uncertainty created by the inherent imprecision of the human mind, you need those data points to be uncorrelated with each other. In other words, you need, not just well-placed sources, but well-placed sources that are seeing different aspects of the enemy leader’s thinking, or are exposed to it in different contexts. Otherwise, you’re just getting the same insight from two different places which gives the impression of greater certainty, but is actually the result of bias in your population of intelligence sources.

Does your head hurt yet? Well grab some painkillers because now we need to deal with an entirely different class of problem.

Earlier I talked about good vs. bad intelligence with the basic distinction being that in the former case, the information is good and in the latter it’s bad. But there are two different flavors of bad intelligence.

One is what I’ll call accidentally bad intelligence, the other I’ll call deliberately bad intelligence. Accidentally bad intelligence is when your source is just wrong. Returning to the enemy invasion hypothetical above, maybe they heard the enemy leader say there would be an attack tomorrow, but the leader changes their mind that night and calls off the attack.

That’s bad intelligence, but it’s an accident. The source did the right thing but the situation changed.

But consider the inverse. What if the enemy leader tells the source they don’t plan to launch an invasion, which the source duly reports, but then an attack does take place? This could well be a case where the enemy leader, to gain the element of surprise, deliberately sends out false intelligence. And in this particular hypothetical, it doesn’t even matter whether the source has been discovered. The enemy leader might be seeding false reports in lots of places, with no real idea of who is reporting on them or not.

The potential for deliberately bad intelligence has a whole bunch of knock on effects for intelligence agencies which are worth touching on briefly for a couple of reasons. For one, they help illustrate why intelligence agencies are so secretive. For another, they have ramifications in terms of intelligence agencies’ potential to succeed in their core mission.

Because of the potential for the enemy to feed deliberately bad intelligence, intelligence agencies have created an entire discipline called counterintelligence. Counterintelligence is not just about security. It is about actively targeting and penetrating the intelligence agencies of your adversaries to find out what they know about your activities.

Going back to the earlier example of the well-placed source close to an enemy leader. If that source is being fed bad information because the leader is generally spreading around bad information, that’s one thing. But if that source is being fed bad information because the enemy knows they’re a source, that’s quite another. Arguably, it isn’t altogether a bad thing! If you know that your enemy is feeding you bad intelligence, you might be able to reverse engineer their true intentions from that bad intelligence!

But think about what is involved to determine one way or another if your source has been uncovered. When you target an adversary’s intelligence agency, that’s about the hardest target there is. You’re trying to gain access to your enemy’s intelligence professionals, the people who should be most well trained to defeat your attempts. Intelligence agencies know they’re targets for their adversaries, hence in part why they are so obsessed with secrecy down to the identity of their officers.

All of this activity incurs costs both in terms of direct material resources, but mental energy as well. You have to staff counterintelligence teams. You have to shoulder great risks because of the inherent danger that is ever present in dealing with enemy intelligence professionals. All of this detracts from the main mission of figuring out what your adversaries are going to do.

I want to take a quick sidequest that will hopefully bring the matter to life. Arguably the most famous counterintelligence officer who ever lived was a man named James Jesus Angleton. Angleton was the Chief of CIA Counterintelligence during the first half of the Cold War, you can look him up, he has a Wikipedia page. In terms of dramatizations, Angleton’s life was a significant part of the inspiration for Matt Damon’s character Edward Wilson Sr. in the film The Good Shepherd. He was also played by Stephen Kunken in the television series A Spy Among Friends. The Good Shepherd and A Spy Among Friends are both excellent and if you like this stuff, I commend them both to you.

Anyway, as I said, Angleton was the Chief of CIA Counterintelligence. To give you a sense for the kinds of things this entailed, one of them was to maintain a huge repository of trivia about the Soviet Union. And when I say trivia, I mean stuff like what color was the door to city hall in Leningrad in 1953.

The use of this trivia being that when defectors from the Soviet Union approached the CIA, it was the responsibility of Angleton and his team to establish their bona fides which is Latin for good faith. In other words, Angleton had to make sure they were real defectors and not Soviet plants bent on spreading disinformation.

One of the ways of screening our the real defectors from the false ones, called provocations back then, was and remains to check every single possible part of their story. If the person says they’re from Leningrad, can they remember details of that city that line up with the dates they claimed to live there?

If that were not confusing enough, keep in mind that Angleton and his team were not looking for people who could answer all such questions correctly. A normal person might remember the name of their second grade teacher, but not their third grade teacher for example. Memory is incomplete and it is not always clear what will stick and what won’t.

But someone who can remember all that minutiae… That’s suspicious, right? That’s certainly what Angleton thought and so it became very much a question of judgment.

As I recount this story, I’m a little worried that it might be apocryphal. That’s a wanker’s way of saying made up or not entirely based in fact. The reason I think that is I cannot recall where I read about this which means it might have been a novel? Well, even if it’s made up, in this particular case it’s fine because it does a good job of highlighting the basic set of challenges and issues someone like Angleton and his staff would have dealt with.

Anyway, in the course of his espionage career Angleton became closely associated with and a good friend of Kim Philby. Kim Philby was an officer of MI6, or more technically the British Secret Intelligence Service and the UK’s equivalent of the CIA. In 1963, Philby was conclusively unmasked as a Soviet mole, and he defected to the Soviet Union. Notably, at the time of his defection, Philby was responsible for MI6’s counterintelligence operations against the Soviet Union. Needless to say he was and remains Britain’s worst ever traitor and a dozen people if not more were murdered by the KGB as a consequence of his actions.

My unoriginal reading of Angleton is that the realization he had been so thoroughly deceived by someone so close to him is that mentally, it broke him. He started to see moles everywhere in the CIA and his paranoia eventually reached a point where he was more of a hindrance to the agency than a help.

This is what I mean when I say the counterintelligence mission is an enormous drain on mental energy and resourcing. Angleton had a huge staff. He had huge influence in the CIA. And many historians believe he was ultimately a detriment to the CIA’s ability to function by the end of his career.

Man. Where were we? Where are we? Wilderness of mirrors indeed. Haha nah I remember. I was walking you through the conceptual problems with intelligence gathering and especially the two x two of good / bad intelligence and good / bad decisions. We’ve talked a lot about the intelligence axis, but now I want to turn to the decisions axis. And we’re going to talk about it in the context of a few real life cases.

Ukraine

Let’s start with Ukraine. Recall that Russia invaded in February 2022. In the lead up ,there was absolutely no doubt in anyone's mind that a Russian invasion was a real possibility. In January, the Center for Strategic and International Studies, a well-regarded Washington DC thinktank, released a report entitled "Russia's Possible Invasion of Ukraine". Everyone from the Ukrainians to the Biden Administration knew this might happen.

And it did happen. Good intelligence! How about the decisions, though?

If you go back to those early days of the invasion, it's not like Russia's military collided with extensive fortifications the Ukrainians had built to counter an attack. I found this report from NPR in which the deputy head of the Kyiv city council describes the security of the city as "so-so". Not very inspiring!

I don't have a great answer for why Ukraine was not more dug in ahead of the invasion. Russia's troops were massed on the border, they were receiving lots of intelligence pointing to the possibility of invasion, and yet instead of fighting from multiple layers of trenches behind landmines and tank obstacles, Ukraine had to mount a pretty disorganized defense.

If I had to guess it was probably some combination of the Ukrainian government dealing with lots of issues and an uncertain prospect of invasion only rising so high on their priority list. And on top of that, just the general psychological resistance of the populace to the idea of being invaded. In today's modern world, where we are mostly comfortable with enough food to eat and air conditioning, most people want to talk about Netflix, not the prospect of enemy tanks rolling down the street.

For the sake of argument, let's just assume that basic interpretation is correct. Here's a situation where arguably it wasn't even good intelligence / bad decision but good intelligence / no decision. What do you do with that? What does that say about the significance or potential impact of intelligence?

To me, it is a pretty good argument for seeing its limits.

There's another example from the start of the invasion, however, that perhaps inverts things. In the days prior to the invasion, US intelligence officials and the military assessed that Kyiv could fall within days of an invasion. And this is clearly what Putin thought too. His conception of the invasion was always as a short, sharp "special military operation" that would topple the Ukrainian government and install a puppet regime.

As we know, Kyiv never fell and scenes of Russian armored vehicles stuck in a traffic jam and getting steadily picked off by Javelin anti-tank missiles will remain one of the defining scenes of the war.

This raises a lot of questions. Whose intelligence was bad in this situation? I would say certainly the US had bad intelligence. It clearly misread the success of Russia's military modernization program and while it was able to count all the new stuff Russia bought, it was not able to assess the quality of the underlying force.

But shouldn't Russia have had perfect intelligence about their own capabilities? Like, shouldn't Putin have known exactly what his own military could and could not do? Evidently not! Let's go all the way back to the problem of good intelligence / bad decisions we discussed earlier. If the US had had a well-placed source next to Putin who heard him say "our military can take Kyiv within days of an invasion", what is that? Is it intelligence because Putin said it and he should know what his military is capable of? Or is it something else. Man, I dunno.

Obviously, part of the solution to this conundrum is that Putin didn't know what his military was capable of. His own intelligence about his own armed forces was faulty. Bad intelligence, bad decision. But part of the solution is also that Putin had bad intelligence about Ukrainian capabilities relative to Russian ones. The Russian armed forces were worse than he thought, the Ukrainian ones were better.

And so the genuine question this raises for me is, could the US get valuable intelligence from Putin even if it had a source in his inner circle? Or would anything that source receives be tainted as bullshit by virtue of it having come from Putin who clearly had no idea what was going on?

Let's turn to the US. Now, not being a strongman dictatorship, the various internal political pressures that led to Putin being badly misinformed were not present in the US. And yet, the US was calamitously wrong about the quality of Russian armed forces and the potential stubbornness of Ukrainian resistance.

If the US had thought Ukraine had a chance, I think it would have approached the situation differently. I suspect it would have put more pressure on Ukraine to dig in and mobilize, it would have started training the Ukrainian military at greater scale earlier on, and it would have provided more weapons. Basically, it would have put Ukraine in a better place to defeat the Russian invasion.

But instead, the US resigned itself to a sudden collapse in Ukraine so it didn't reinforce the country in the way it otherwise might have.

What's notable to me about the Ukraine situation is it's a case where good intelligence about one part of the problem was rendered useless by bad intelligence about another part of the problem. The US knew there was a high likelihood Russia would invade Ukraine. So the very simple response here is to help Ukraine resist the invasion? A stronger defense is better than a weaker defense?

And yet, because the US expected a quick defeat, my read is that prior to the invasion, its support was not as urgent as it could have been.

Importantly, I am not making a critique of the US here. For a start I don't have all the facts. Maybe the US was doing a lot I don't know about. Second, the US government has a lot to worry about and trying to make Ukraine's defeat less bad perhaps just didn't make "the list". You have to prioritize.

Nevertheless, I think it is all told a great example of the limits of intelligence. Just because you know something is pretty likely to happen, doesn't mean you end up doing something about it because your perception of what comes after the initial something is cloudy.

It's almost as if there's a breakpoint in the value of knowing something. Because when you know a little bit, that opens up a whole bunch of new questions you think you need to answer before you can decide on a course of action. It's almost like it's a problem. A problem of knowing… Hehe, see what I did there?

Ok yes, yes, very good by me. Let's move on from Ukraine to another case, the Hamas attack on Israel on October 7 of last year.

Hamas and Israel

We have been over many times the events of October 7, please listen to almost any one of the last six episodes for that.

What I want to focus on is what Israel knew ahead of the attack. This article from Haaretz, a major Israeli newspaper, alleges that the Israeli intelligence services knew just about everything they could reasonably be expected to know ahead of the attack down to the number of hostages to be taken and instructions for their treatment while in captivity. If the above link doesn't work because of an ad blocker you might be running, try this one.

I really encourage you to read the whole thing because it's quite remarkable. Israel's military intelligence 8200 Unit compiled a comprehensive report on the attack which was based on a variety of sources, including direct observation of training exercises in preparation for the raid.

Despite the Israeli military having forewarning of an attack, October 7 was wildly successful from Hamas' perspective by killing over 1,200 people, mostly Jews, and taking several hundred more hostage. Here is your classic case of good intelligence, bad decision. Bad, bad, bad.

How can we explain this? The simple answer is that this is a case of politics trumping the intelligence. Israeli Prime Minister Benjamin Netanyahu's strategy for dealing with Palestine has been to divide the Palestinian cause so that he can credibly claim he has nobody to talk to on the Palestinian side with respect to the establishment of a Palestinian state.

In 2007, Hamas took control of Gaza, while the Palestinian Authority retained control of the West Bank. Perfect for Netanyahu's strategy of keeping Palestinians divided and pushing off the need for him to negotiate on a Palestinian state. Because in this situation, he really could claim it was unclear who Israel should be negotiating with.

But obviously, for this to work, Netanyahu would need Hamas to remain in power in Gaza. To this end, he approved billions of dollars in cash investment into the group from Qatar. Here's an article written in 2021 where an Israeli professor gloats at the strategic genius of this move. This particular article is as remarkable as the one from Haaretz. It specifically talks with glee about Israel's strategic deception of letting Qatar fund the construction of the "Gaza Metro", a network of tunnels used by Hamas, and then subsequently destroying them in 2021.

Fast forward to today and we know that whatever Israel did to the tunnel network in 2021, it wasn't much. Today the IDF is constantly reminding the world at large how hard it is to fight Hamas given that it has embedded itself so deeply in a vast tunnel network.

As freezing cold takes go, this one is a true absolute zero. So bad.

Anyway, Hamas got money and control of the Gaza Strip, Netanyahu got his divided Palestinian movement. Both sides were happy with this trade. But Netanyahu was relying on Hamas to keep its violence against Israel limited in scope. He thought Hamas' leadership would be happy getting rich off Qatari money and exploiting Gazans. This in turn allowed him to focus on the West Bank where he could work to advance the project of his far right political coalition partners who want to build more settlements there as a way of making any future negotiations over a Palestinian state more difficult.

Into the middle of this happy arrangement, for Netanyahu, comes intelligence reports that actually it's Hamas who has gotten the drop on Israel, not the other way around, and Hamas is planning a brutal attack against the country. Very inconvenient I would say for Netanyahu's strategy and his political survival!

Keep in mind that prior to October 7, Netanyahu was under immense political pressure. First, he was facing trial for corruption charges. Second, he and his far right colleagues were trying to curtail the power of the judiciary so that they could pass more laws favoring the goals of ultraorthodox settlers. These judicial changes had brought tens of thousands of Israelis out in the streets to protest Netanyahu and his government.

Imagine if, into the middle of this situation, came the revelation that by sowing dragon's teeth in Gaza, Netanyahu had ended up growing an actual dragon? Again, inconvenient!

And so, ultimately, the reports of an impending Hamas assault were ignored. We currently do not know who knew what and when they knew it, partly because Netanyahu has fiercely resisted any attempts at investigations into decisions made leading up to October 7.

But that aside, at a minimum, we can say that Netanyahu's prioritization of the West Bank meant that the IDF's focus was not on Gaza which left it in a worse position to defend against the October 7 attacks than it might otherwise have been.

So here's classic good intelligence / bad decision. And the cause of the bad decision appears to have been politics. At least at some level. So, again, what do you do with this? What does this say about intelligence and its value?

My instincts tell me that politics will generally trump intelligence. And in part it will do so because of all the reasons we talked about in the introductory section. Intelligence professionals themselves are very quick to qualify their assessments, speaking in terms of probability and degrees of certainty. And while thinking probabilistically is the right way to approach these matters, it does leave the door open for politicians to exploit that intellectual rigor.

To point out what I mean, there's a scene in Zero Dark Thirty where the main character, Maya, is in a room with CIA Director George Tenet and a bunch of other senior CIA officers. Tenet asks for their confidence that bin Laden is in the compound they've identified in Abbottabad because he will have to go to the president and get his greenlight for Neptune Spear, the operation which would ultimately kill bin Laden.

The senior officers give confidence intervals ranging from 60-80% and when it gets to Maya she says, without hesitation, that it's 100%. One of the points the scene is making is that Maya is a maverick for going with 100% confidence. That's just not the done thing in intelligence agencies.

Anyway, back to Netanyahu, the inherent uncertainty in intelligence no doubt gave him or his representatives enough mental room to maneuver around what was clearly intelligence that was problematic for their political strategy. In the end, the political strategy won out with disastrous results.

Another very brief example of good intelligence / bad decision where politics was the trump card comes from World War II. I can't remember if we've discussed this guy before but very briefly, the greatest spy who ever lived might well have been a man named Richard Sorge who spied for the Soviet Union. A full six weeks before the German invasion of the Soviet Union, Sorge got his hands on the date for the invasion and duly reported it back to Moscow.

But Stalin had only recently approved the Molotov-Ribbentrop Pact of cooperation between the Third Reich and the Soviet Union. The two countries had just happily carved up Poland together. Surely the great Stalin hadn't been completely duped by Hitler? Stalin ultimately did nothing with the information and German divisions would reach the outskirts of Moscow before falling just short of bringing the entire Soviet Union under German control. It perhaps would not have been such a close run thing had Stalin done more to prepare Russian defenses or move Russian troops from the Far East to the western border with Germany by taking advantage of the intelligence provided to him by Sorge.

I think you get the point. Ok, so far we've talked mostly at the strategic level when it relates to intelligence failures. But they happen at the level of specific operations too and I want to talk about one such example from the Civil War.

Civil War

Alright, history. Yessssss. This is the best bit. The history parts are always the best. And I'm excited for this. I've been reading about the Civil War a lot, like I said. There should be a word for the phenomenon where you read something fascinating, and then you find an excuse to raise it in conversation. If that phenomenon were a disease, I'd for sure be terminal. Particularly for blokes this is a constant crisis. And the funny part is, I reckon often everyone wants to achieve the same outcome.

Like you'll be standing in the playground watching your kids to make sure they don't get knocked out by some twirling piece of steel and talking to another dad about how what they need to buy at Home Depot or whatever and really the whole time you just want to blurt out something like "do you know how many rivets it took to make the Titanic?". But you don't because you don't want to seem weird. Half the time though, I reckon the other dad is standing there mentally drifting off from the conversation to wonder how many rivets it took to make the Titanic. For the record, I have no idea.

Anyway, back onto the main path. Ok so for those who are unclear on the background, the US Civil War was fought primarily over the issue of slavery. In particular, the animating events for the outbreak of the war related to whether states added to the Union would be slave states or free states. The existing slave states, mostly located in the South, wanted slavery to expand into new states, while northern and western states opposed further expansion. There's more detail to it than that, but I think that's a pretty fair summary of the core issue over which the war was fought.

In terms of the events I want to discuss here, the year was 1862 which was about the halfway mark of the war. The Union, which fought against slavery, was economically dominant party in the war but it had suffered a series of military defeats. The Confederacy, which fought to preserve slavery, was an economically backward society based on the slave plantation economy. But the Confederacy was fortunate enough to quickly install its best general at the head of its army in the eastern theater of conflict, Robert E. Lee. Lee in turn brought along similarly gifted generals, especially Stonewall Jackson and James Longstreet.

Together, they inflicted a string of military defeats on the Union in the opening years of the war. And by September 1862, Lee was attempting to invade the Union at the head of the Confederacy's Army of Northern Virginia. In charge of defending the Union capital, Washington DC, was Major General George B. McClellan who led the Union's Army of the Potomac.

McClellan had the superior force. Not only was his force roughly twice the size of Lee's, it was better fed and better equipped too. But for reasons that are both debated by historians and not altogether relevant for our discussion here, McClellan was notoriously hesitant to give battle to Lee and his forces. At the time, the justifications McClellan gave came down to wild overestimations of the size of Lee's forces, and a general tardiness and insistence on bringing up all of his supplies before moving his army forward.

McClellan's hesitancy to fight is a core part of his legacy today. There are some positive aspects to his legacy, but for the most part, historians of the Civil War focus on the fact that the guy just wasn't aggressive enough.

So Lee, the brilliant Confederate general is attempting to invade the north, and McClellan, the hesitant and paranoid Union general is maneuvering out to meet him. Given the disparate strength of their armies, Lee has to separate parts of his army to achieve his military objectives.

During this period in warfare, having more numbers was not simply a case of being able to last longer in the fight. It made it more likely that you would be able to flank your enemy, and roll up their entire line of fire.

So this is an incredibly risk move on Lee's part. Despite facing a numerically superior force, he is going to break his army into smaller pieces to enable him to achieve multiple objectives simultaneously. If McClellan were to catch on, he could move quickly to separate these components of Lee's army and destroy them "in detail" which is military jargon for one-by-one.

Into the midst of all of this comes Special Order 191. The short story here is that Union troops found Lee's precise instructions to his generals, including Jackson and Longstreet, as it related to the separation of their armies, where they should go, and the routes they should take. A similarly critical point here is that the detail of the orders, which you can read here, should've given McClellan some sense for the size of Lee's total force. So he ought to have known not just that Lee was going to separate his army in the face of a larger enemy, a huge risk at this time in warfare, McClellan should have concluded that even when reconstituted, he would have the superior force.

In the world of intelligence, it just doesn't get more clear cut than this. The information was correct, it was specific, it was impeccably sourced, and it was timely. This is the proverbial gold dust that George Smiley, played by Gary Oldman, refers to in the movie adaptation of the novel Tinker Tailor Soldier Spy.

I think you can guess what happened next, though. McClellan, despite the written orders from Lee that he himself held in his hands, continued to believe that Lee had a superior force. For that reason, he did not move his forces forward until a full 18 hours after Special Order 191 came into his possession. This was enough time for a Confederate sympathizer, who had observed McClellan's jubilant reaction to getting his hands on Special Order 191 to inform Confederate cavalry. The Confederate cavalry in turn was able to inform Lee.

Lee, obviously moved to reconstitute his force, but he also had the time to place some troops in front of McClellan's army and their delaying action would give Lee an extra day to bring all of his forces together. As a consequence of these various events, the Civil War got the Battle of Antietam. Approximately 22,000 men were killed in the ensuing battle and it remains the bloodiest single day in American warfare. To give you a sense for comparison, total Allied casualties on D-Day were about 10,000 men.

And so, while McClellan ultimately warded off Lee's invasion of the North, he missed a golden opportunity to destroy the Army of Northern Virginia in detail, and potentially end the war much sooner than it otherwise would have.

Good intelligence, bad decisions! And not just bad decisions, but the same repeated bad decisions again and again. In this case, the cause of the bad decisions seems to have been plain timidity and paranoia. One might be tempted to label McClellan as simply incompetent but that's not really fair. Briefly, McClellan was responsible for organizing and training his army, the Army of the Potomac. And by all accounts he performed brilliantly in this task. It's just that when it came to fighting, he overestimated his enemies and moved too slowly as a consequence.

I really think Special Order 191 drives home the limits of intelligence itself. I think you can make a case that intelligence, in addition to all of the other qualities we talked about at the top, also has to be actionable. Someone, somewhere, has to have a clear use case for fact that you have obtained otherwise I think you just have information.

But if you agree with that contention, how do you define actionability? McClellan was a major general, he was the senior commander in the field. If anyone should have found Special Order 191 actionable, it was him. And yet, he did not find it actionable enough. So does that make Special Order 191 intelligence, or just information?

I entitled this episode the problem of knowing. But perhaps a better title might have been the problem of acting. Because on reflection, I think what we've really picked through here is that for all of the hype that Hollywood gives to intelligence, and the work of intelligence agencies, ultimately the value in intelligence comes from its successful exploitation by leaders. And the willingness of leaders to exploit the knowledge available to them time and again proves limited whether it's because of their own biases, political constraints, or limitations in their own ability.

I wish I had some kind of penetrating insight to finish on. I don't know how penetrating this is but something this whole series of thoughts has led me to conclude is that in any field, it can often be helpful to ask how gathering information will shape a decision one way or another. Because if that cannot be articulated, I think it might be helpful to discuss whether it's worth gathering the information in the first place.

To use a more every day example, someone might be asking to see company sales broken down by ZIP or postal code. That might be a helpful piece of information for a retail store which is trying to figure out where to build new stores. But does it matter for a SaaS company? Probably not. And asking for information or further "studying" an issue is a way I think a lot of people in positions of authority defer having to make a decision and thus putting their careers and reputations at risk.

I guess this is just another way of saying something I learned at McKinsey which is that it is efficient and productive to be hypothesis-led. Start with what you think is the right course of action, then gather the information you need to prove or disprove that contention.

Well, I think that is as good a place as any to leave it. As always thanks for listening and hope to speak with you again soon.