The “new” Open Source Intelligence and Silobreaker

The Tsunami that struck South Asia on December 26th 2004 was a wakeup call for the world in many aspects. Hurricane Katrina that struck New Orleans 8 months later was another. The crisis in Darfur is here and now.

The heartbreaking stories about human losses and torn-apart families will eventually fade. The insights about what was really working in information matters will live on in the ways we need to interact with information, knowledge and intelligence. Governments slow decision cycles, below-par information management and in some cases extremely hierarchical fears make them slow to respond. Media are a little bit faster but the proliferation of information and the global scope makes the validation of what people need to find, know and understand extremely complex .

But this is the world we all live in. Instabilities such as war, terrorism, environment, crime, disease, human rights issues, political upheaval, and natural disasters are global and affect us all. There is a lot we do not know and cannott foresee, but many times the writing is on the wall somewhere in the world, if we just learn how and where to look for it

Information volumes

The data volumes in all areas such as the above is growing exponentially each year due to the advances in information technology and the sociological and cultural changes. In a study by the School of Information Management and Systems at the University of California at Berkeley, it was estimated that in 2002, print, film, magnetic, and optical storage media produced data equivalent to half a million new libraries the size of the Library of Congress print collections. Ninety-two percent of the new data was stored on magnetic media, mostly on hard disks. Data that flows through telephone, radio, TV, and the Internet was three and a half times the volume held in storage media. Ninety eight percent of this total is data sent and received through telephone calls – including both voice and data and on both fixed lines and wireless. These statistics were based on data from 2002 and the exponential growth hasn?t exactly stopped since then. The challenge we all face is to reduce these incomprehensible volumes in order to act or react. When it comes to volume less is definitely more.

We need to understand what data to ask for and why we need it.

What about Intelligence?

But we have Military Intelligence, Competitive Intelligence, Political Intelligence, Market Intelligence and thousands of other buzzwords to help us… Or?

The consumers of the present practice of intelligence are kept in the dark. Most intelligence activities are just information or data hoarding and at best secondary research smartly upgraded by buzzword wizards. Maybe this is a bold statement but our experiences and encounters with decision makers and their support teams within both governments and business the last 20 years all confirms this observation. ´

One man´s data is another man?s information and what is information for one organization can be key intelligence for another. During the Cold War, and still the case, governmental intelligence organizations existed with a culture of “just in case” which meant that they tried to collect everything. Today, with the overwhelming access to open sources information and changing requirements it is the mode of “just in time” that the corporate world must build their intelligence operations on.

But still in 2007, decision makers down to mid-level managers are all drowning in the amount of information they receive each and every day. They have all the data and information they need, they sometimes even coin it intelligence, but they do not use it because it is not targeted, relevant or timely. And to further complicate this depressing picture, whole organizations experience avalanches of data with the main achievement of which is to generate guilty consciences. Decisions will not be made, actions will not take place and goals will not be achieved. Caught in the avalanche of buzzwords and “must haves” there are far too many examples of how management buy their consciences free with the latest IT solution that many times glorify their deeds for a couple of months but rarely in the strategic perspective. The quick wins become long term losses.

The same is true for the Government sector, but now at the taxpayers expense. This makes the situation much worse. Actually, many of the recent failures in anticipating and preventing conflicts and corporate crises or responding to emergencies can be attributed to the culture of intelligence gathering itself.

As practiced today in government and private industry, the intelligence and knowledge processes are still centralized, top-down, elitist, parochial, secretive, and culturally-biased. Most corporations and government agencies tend to reward silo builders, not silo breakers. The result is a culture in which information is not shared and insights are rarely communicated across organizational boundaries. As the world becomes more global and decentralized as well as more multi-cultural and multi-lingual, the need for a service that reflects a new paradigm is paramount.

Intelligence gathering and sharing should be more collaborative, more bottom-up in its thinking – a real silo breaker in the true sense of the word. With billions of dollars being spent on new intelligence technologies, precious little is being invested in global data capture, discovery, and understanding. There is no shortage of information today. In fact, it is the overload that is crippling us and leading us down a proverbial blind alley. What is missing from most intelligence tools is the ability to see the whole in context.

Even in small countries like Sweden, there is a wealth of money spent on intelligence, but for what purposes? The insight and benefits we gain as a nation from the products that the different agencies produce are very small at best. The legacy of secrecy is today not only protecting the few secrets and sources that might be worth protecting, but it is also protecting the fact that the lion share of what these agencies provide could be provided by anyone, at a fraction of the cost and much more timely. Small networked and flexible companies and organization have the information advantage today.

Open Source Intelligence

OSINT tools & sources provide 95 percent of the world´s intelligence; however, the processes commonly used to detect threats, assess risks, and analyze instabilities cast a rather narrow net.

In the world of secret services, OSINT means useful information gleaned from public sources, such as scientific articles, newspapers, phone books and price lists. We use the term differently.

The “new” Open Source Intelligence is about the translation of disparate data and information to actionable intelligence via focused real time contextual analysis. It is about content, methods and tools.

It is also the application of collaborative principles when it comes to the gathering and analysis of information. These principles include: peer review, reputation- rather than sanctions-based authority, sharing of products, and flexible levels of involvement and responsibility.

It might sound like a familiar theme but the time has come to re-define the concept of intelligence for business and organizations. The time has come for put together bits and pieces into a whole, both with regards to content and technology,

Silobreaker and Open Source Intelligence

By looking for meaning, not words, and focusing on the relatedness of people, events, and ideas, Silobreaker is designed to help anyone anticipate, prevent, and solve problems before they become full-blown crises. This is fully in line with the original objective for Silobreaker to create a high-awareness, answer-driven, analytic search application as good as, if not better than, those used within the intelligence community and make it commercially available to everyday user such as executives, journalists, consultants, marketers, planners, technologists, investors, scientists, and scholars. Silobreaker addresses gathering of content, analytical methods and information tools, in one integrated platform. In the Silobreaker customized and ASP- solutions it also addresses the collaborative objectives.

In other words Silobreaker is a virtual Open Source Intelligence (OSINT) center that focuses on the public information found in newspapers, magazines, newsletters, video, web sites, blogs, and more.

Unlike large content aggregators and Internet search engines which are data-rich and information-poor and present their results in a linear perspective, Silobreaker is an insightful discovery tool that extracts meaning from unstructured news flow, clusters and visualizes it for deeper analysis, and Silobreaker provides the contextual intelligence necessary for making decisions and taking action.

* * *

We believe that this new school of integrated thinking will soon bring down the old school of intelligence that still rule on the corporate and government agendas. Information is no longer a competitive advantage. Everyone, at least in theory have access to almost everything. Awareness and use of the power of integrated real-time analytics is a key competitive advantage. Insight in seconds is a must to stay ahead of the curve. Insight as a service is what Silobreaker and the new OSINT paradigm is all about.

By Mats Bjore

Director Silobreaker, and former Founder and Head of the Open Source Intelligence Section at the Swedish Headquarters Military Intelligence & Security Service.

Source: Silobreaker


One response

  1. […] the site is a demo for enterprise intelligence gathering. But that’s not the only place this approach has cut its teeth — variously known as open source intelligence, public intelligence, and naked intelligence, […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: