Category Archives: Software

The W/E BLOSINT Brief

This Week BLOSINT Brief (WBB) contains the following information:

1. The Americans are smart enough to go forward with enhancing OSINT as Open Intelligence to the Academic field.

“U.S. Sen. Kay Bailey Hutchison (R-Texas), a member of the Senate Appropriations Committee, said Thursday that the committee has approved $1 million for open source intelligence and intelligence analysis at Sam Houston State University.
The bill is now ready to be considered by the full Senate.
“America’s military and intelligence services are constantly on guard for potential threats against our homeland,” Hutchison said.
SHSU will use the funding to expand open source intelligence collection and analysis efforts and training for Foreign Military Studies Office elements at the El Paso Intelligence Center.” Source: The Huntsville Item

2. MySpace is going on with Qizmt [kiz-mit]
“MySpace on Tuesday will release as open source a technology called Qizmtthat it developed in-house to mine and crunch massive amounts of data and generate friend recommendations in its social-networking site.
Qizmt is a distributed computation framework based on the MapReduce programming model for processing large data sets in processor clusters.” Source: PCWorld

3. The U.S. Intelligence Community and Foreign Policy: Getting It Right (a Brookings Institution report that makes a lot of good points, especially about the IC reducing its reliance on Classified sources) via IntelFusion

4. Robert Steele, the longtime proponent of a robust open source intelligence program, has a new web site which notably includes an archiveof intelligence-policy related documents, several of which I had missed.  The collection is accompanied by his own occasionally tart commentary. via Secrecy News

5. Intelligence Event: “Intelligence for the Twenty First Century. Intelligence for Everyone.” Madrid 22nd-23rd September 2009, Interligare 1st International Forum, via Public Intelligence Blog

Intellipedia goes on

Intellipedia, the intelligence community’s version of Wikipedia, hummed in the aftermath of the Iranian presidential election in June, with personnel at myriad government agencies updating a page dedicated to tracking the disputed results. Continue reading →

Open Source Web Based Geospatial Processing with OMAR

“There is one thing stronger than all the armies in the world, and that is an idea whose time has come.” – Victor Hugo

The availability of geospatial data sets is exploding. New satellites, aerial platforms, video feeds, global positioning system (GPS) tagged digital photos, and traditional geographic information system (GIS) information are dramatically increasing across the globe. These raw materials need to be dynamically processed, combined and correlated to generate value added information products to answer a wide range of questions. Continue reading →

Seven Predictions for Open Source in 2009

2008 was an eventful, breakthrough year for many open source companies, and 2009 will be even more so, especially in terms of business purchasing patterns, software business model shifts, and enterprise software stack evolution. The current economic conditions will certainly prompt businesses to look more closely at alternative IT solutions — and open source technology will be one of the big winners next year. Continue reading →

Data Visualization

First – Sources and Methods

John Stasko and the computer scientists at the Information Interfaces Lab at Georgia Tech may not have found the Holy Grail of visual analysis but they have come pretty darn close with their Jigsaw product.

This extraordinary visualization tool automatically extracts entities (names, places, dates, etc.) fromplain text documents. Then, it automatically creates a visualization of the relationships between those entities and the documents containing them. The screenshots below do not do it justice (I hope to have a video of the product in action within a couple of days, though).


The program is fully customizable so you can add or delete data, designate entities or create relationships to modify what the automatic entity extractors come up with.

The real power of the tool comes into play after the data is in the program. You can play with it in a variety of powerful and interesting ways all accessible through a drop dead easy user interface.

The software is continuously improving. On the horizon is the ability to use web input and there is a long analyst generated to-do list that the grad students at GA Tech are cranking through one at a time.

The software runs on a desktop and was developed with a DHS grant so government and academics should reach out to John for a test copy. GA Tech is also home of the Visual Analytics Digital Libraryand well worth checking out.

From a visual analysis conference sponsored by the National Visualization And Analytics Center

Second – Sources and Methods

Yesterday, I posted my initial reaction to an extraordinary piece of software called Jigsaw I saw demo-ed during my visit to GA Tech for the Workshop on Visual Analytics Education sponsored by NVAC. Today, John Stasko and his team uploaded a video of the software in action that is a must-see.

I understand the video is a little old so some of the functions I saw are not in this video. The good thing about John and his band of software wizards, though, is that they are constantly improving the product. 

DHS discovers the challenge of creating a collaborative social network

The GAO (Government Accountability Office) has held up the deployment of the DHS (Department of Homeland Security) critical (non-classified) information sharing system.  The unlucky group that must deal with the herculean task of launching a collaborative social network of law enforcement, state, local federal and tribal agency members must answer some tough questions before continuing.  That group is the Homeland Security Information Network Advisory Committee (HSINAC). 

Some of the things these poor guys must respond to include:  

-Conduct an immediate manpower survey for an outreach effort which is basically the sales and marketing effort of the Next Generation social network they are attempting to build
-Create a fully articulated business strategy with management controls (their words not mine.)

Large scale information sharing is what social networking is all about.   Creating a critical mass and going viral is one of the phases of any successful launch.  Having a “well articulated business plan” and “management controls” in place *before* launch is not going to help.

Look at MySpace, FaceBook, Slashdot, Craigslist, Digg, Reddit,Twitter, or even Tipd which is in the process of trying to create social site for finance.   All were launched with significantly less than the $150 million that DHS is going to spend on their critical information portal.   

Here is what DHS should do to create an effective information sharing portal. 
1.    Use open standards of course. No proprietary software that has to be deployed to each desktop or department.  
2.    Immediately take the information resources they have available now and publish them.  Lot’s of great tools for that. 
3.    Create a social bookmarking capability such as Digg or Reddit so that the important information can float to the top and discussions can occur.  (Reddit’s code is free and open source)
4.    Faciliate the social aspect. Create a Twitter like functionality so first responders can follow each other and link to each other’s resources.  
5.    Provide a feed reader capability (or just let everyone use Google Reader) so the end users can follow the stuff they determine is important. 
You know that this could be deployed in a matter of days, maybe weeks since it is a government project.  Certainly within the $3 million budget that DHS has for maintaining the failed system already deployed through next September.  The only thing that may take significant investment is the credentialing system needed to keep the bad guys off the network.  

No social network has ever succeeded through central planning, outreach efforts, and management controls. Every successful new information sharing network has just happened, usually explosively.  Unless the DHS can come to grips with that the Homeland Security Information Network-Next Generation is doomed to fail.

Source: http://www.networkworld.com/community/node/34307

Open software tools

The list of Free software tools for designing productive community sites is showing us the future of the intelligence realm: how to become rapid and collaborative in an open world.

Open GEOINT: OpenStreetMap

“It is always so easy to ask why someone doesn’t do something until you realise that you are that someone. ”

Go read the article of Gary Richmond on FSM …

In a recent article on free software and the Large Hadron Collider I mentioned that here in the United Kingdom The Guardian, a national British newspaper, had founded a campaign called “free our data”. They objected to the fact that the Ordnance Survey (and others), funded by the British taxpayer, was charging business and individuals for its cartographic data thus effectively making people pay for it twice. Their campaign is great but until such times as it succeeds an alternative is needed. A free software alternative. Enter OpenStreetMaps.

Google Earth and Google Maps are too well know to require iteration here, but the spectre of proprietary software haunts them. They are not free software. If you want to incorporate any of them into you budding business project and run your software under a relatively permissive licence for others to take up your ideas and improve them you will have to find something else.

Just like Wikipedia, on which it is loosely modelled, OpenStreetMaps is resolutely free software. It is an attempt, by community participation, to map the Earth.