Read any government security document, any of the national security strategies produced by a now large number of states and you will get a feel for the proliferation in the number of threats they feel they face. The preamble will normally contain a paragraph explaining that after the Cold War or after 9/11 everything got a little more complex, a little less explicable.

Heightened complexity in the international system appears to have coincided (and is only partially causally linked) to the increased levels of activity/ improvements in technology, social media etc. The rate at which information can be collected has increased, even if the sort of information being collected is broadly the same.

The problem of accounting for events like the Algerian gas-plant siege a few weeks ago (or the development of the insurgency in Syria, or in the hijacking of the state in Mali) for state-based security organisations is that their resources allocated in such a way that it logical for them to be looking the wrong way when this happens. It would be unlikely – although we can’t be sure, obviously – that there’s a bod in every security community across Europe pondering the safety of gas-plants in the ME and Maghreb. So, when this happens the information required to rapidly come down the pipe needs to be hastily scoped and drawn in. And this got me thinking about Robert Steele’s ‘open source everything’ manifesto (I declare the interest that Robert has written a chapter for the Routledge Handbook on Intelligence that I, Mike Goodman and Claudia Hillebrand have compiled and which will be in a good bookshops from August, and that he and I have corresponded at length about these issues), and how it could be used or applied in these circumstances. I have my own take on this, and I’ve provided the link above to the source: Robert also has a good search on his name I think so I’d guess he’ll correct me in comments too! But my wonder is more in the aggregation of huge quantities of information.

If we assumed that insurgents or terrorists leave an electronic detritus of chatter (be it closed loop phone or some other form), movement data, financial data, and the chatter of their associates, family etc and local media reports etc etc, then the actual ‘intelligence’ required to identify, contain and roll-back a threat or ‘black-swan’ event should be there, right? No-one – it seems – can totally avoid leaving the sort of trail that could be used in anticipating an event, so the issue is in collecting the data in a way that makes sense, and making predictions on it (lenses through which we understand the world). And that made me wonder about how one could translate this kind of regional or localised intelligence into a western European perspective: does it need expert ciphers to do so? Or can it be done with generalists? This fits into one of my side projects, which is thinking through how to make better use of scholarship in the ‘real world’. Would a more open source arrangement provide the sort of information to be better resilient to these black-swan events? This is not to say that the current arrangements are ‘bad’ or ‘failed’, but just like in defence it seems that there’s a constant circle to be squared of ‘more’, ‘more diverse’ and with relatively static methods or money.

Relatedly, some years ago Milja Kurki (who might be the smartest person I’ve ever met) and I wrote a paper about the intersection between intelligence studies and IR theory. We never published this paper and for their pains my final year students get to read it as a tiny part of the reading list for my final year option. Whilst we felt that the field was under-theorised, the paper we wrote never really connected up to the reality of intelligence work adequately enough. The two communities or endeavours seemed immune from each other – indeed to try and overlay one on the other seemed to produce an immune system response. But with the benefit of some years to cogitate on it, and having thought about the work of people like Steele, Fuller, Cairney (as above), I think the missing element from this paper was the realities of complexity:

• Things in the international system tend away from equilibrium and not towards it as most IR theory suggests
• The international system tends to chaos and not order (and the level of chaos might be reduced to the level of individuals, making generalizable lessons problematic)
• Man-made uncertainty shocks, or ‘black-swans’ to use other language, are mostly resistant to accurate advanced prediction. Thus one main function of theory – to predict – is always likely to fail. Using a different approach we could learn lessons quickly enough to be able to deal with a problem or series of problems close enough to the source that in effect it looked like prediction and pre-emption.

The issues all fall-down to how to best use (in terms of creating the right institutional frameworks and having the right cultures) the information available, and in that we need more thinking work into whether alternatives genuinely stack-up.