Making the Most of Open Private Sector Knowledge
Despite the fact that U.S. taxpayers have been paying more than $30 billion a year for a national intelligence and counterintelligence community to protect it from both traditional state-based threats and unconventional non-state actors, the events of 9-11 demonstrated our inability to detect and prevent bold asymmetric attacks that used our own airliners as precision missiles. Armed with new concepts, money, and suicidal pilots, Osama bin Laden has cost us at least $20 billion in damages.
The problem with spies is they only know secrets
Unfortunately, our spies and our satellites have lost touch with reality, for they collect less than 10% of the relevant information that we must digest to understand the complex multi-cultural world that is now capable of producing very wealthy and suicidal terrorists. We need a “new craft of intelligence” that can access and digest the broad historical, cultural, and current events knowledge that is available openly in over twenty-nine languages — by exploiting these open sources we can create open source intelligence, or OSINT, suitable for informing our public as well as our state and local authorities and our international partners, as to the threats to our nation.
What are open sources? Open sources go well beyond the Internet (3 billion pages of substance and rising) and premium online services (ten times what is on the Internet, with value-added) to include “gray literature” (limited edition publications including dissertations and local directories from around the world); specialized market research, private investigations, and other information broker services; and geo-spatial information services including commercial imagery and Russian military maps for all countries of the world (the U.S. does not have military maps for 90% of the world.) Open sources include experts on any subject, in any language. Shocking as it may seem, our intelligence community does not routinely strive to identify the top people in the world (not just Americans) on the various topics of concern — from terrorism to the environment to human trafficking to corruption to disease and public health — with the result that our analysis tends to be shallow and incestuous, relying on the same consultants again and again.
Where’s the action?
Why is this not obvious, and, more importantly, why is it not being acted upon? Although the bipartisan Aspin-Brown Commission on intelligence reform (reporting in March 1996) found that our intelligence community is “severely deficient” in its access to open sources of information, and also found that the various departments and agencies of government have failed to fulfill their responsibilities for collecting, processing, and analyzing open source information relevant to their missions, nothing has been done to implement the Commission’s recommendations for reform. The Commission specifically stated that OSINT should be a top priority for funding within our $30 billion a year intelligence budget, and that it should be a top priority for the attention of the Director of Central Intelligence.
– Digital History Project ($5M) to digitize and translate key Islamic, Chinese, and other foreign language historical, political, economic, cultural, social, and technical materials.
– Non-Governmental Organization Data Warehouse ($10M) to provide free storage and network access to the various international organizations whose “local knowledge” is vital to U.S. understanding.
– Global Coverage Distance Learning and Expert Forum Network ($10M) that will establish open ethical boards of review for all countries and topics, including distance learning and expert forums.
– Generic Open Source Training Initiative ($10M) to create both distance learning modules accessible by our state and local, armed forces and diplomatic personnel and our public.
– Public Information Sharing and Collaboration Toolkit ($10M) comprised of a generic set of industry standards and related tools for desktop level exploitation and analysis of digital foreign information.
– Regional Open Source Information Networks for Africa, Asia, Europe, and Latin America ($40M) , each with an open source collection and processing center in partnership with local governments who will provide regional language skills and access to gray literature and local experts.
– International Trade and Chamber of Commerce Network ($5M) to establish a web-based network maximizing access by U.S. businesses to foreign economic, regulatory and taxation information.
– Digital Marshall Plan ($20M) to provide direct assistance and subsidies to extend the Internet to every corner of the world (including rural areas in America) via wireless delivery means.
– University of the Republic & Global Outreach Program ($15M) that will bring together and educate “cohorts” of mid-career subject-matter experts from state and local governments, the federal government, and the business, academic and media communities, as well as foreign professionals.