Total information awareness
lives! From the CS Monitor
The US government is developing a massive computer system that can collect huge amounts of data and, by linking far-flung information from blogs and e-mail to government records and intelligence reports, search for patterns of terrorist activity.
The core of this effort is a little-known system called Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement (ADVISE). Only a few public documents mention it. ADVISE is a research and development program within the Department of Homeland Security (DHS), part of its three-year-old "Threat and Vulnerability, Testing and Assessment" portfolio. The TVTA received nearly $50 million in federal funding this year.
What sets ADVISE apart is its scope. It would collect a vast array of corporate and public online information - from financial records to CNN news stories - and cross-reference it against US intelligence and law-enforcement records. The system would then store it as "entities" - linked data about people, places, things, organizations, and events, according to a report summarizing a 2004 DHS conference in Alexandria, Va. The storage requirements alone are huge - enough to retain information about 1 quadrillion entities, the report estimated. If each entity were a penny, they would collectively form a cube a half-mile high - roughly double the height of the Empire State Building.
I’m leery of this on a couple different levels. First, it seems like it would be easy to screw up. Last night on Olbermann’s Countdown
the “Worst Person in the World
” was a toss up between a bank and DHS because they froze the bank accounts of some nuns under provisions of the patriot act. Why? Because of they didn’t have the right info on file. And the track record of the “no-fly
” list doesn’t give me a warm fuzzy about this type of program either. What happens if ADVISE and the no fly list are linked? Sorry sir, you bought a copy of the wrong book from Amazon so you can’t board the plane.
Of course, I can see the intelligence utility of such a program, but I can also see the civil liberties pitfalls. In our increasingly connected world, our society needs to think about these things. How will our government exercise independent oversight of ADVISE? Or will it be like NSA programs and be so highly classified that anyone outside the intelligence community won’t be able to see what’s going on? Perhaps we should start electing people to congress who already have Top Secret clearances.
Finally, I wonder if this is not yet another example of our country’s over-reliance on technology to try to solve its intelligence problems. UBL probably isn’t clicking around on Amazon. The bad guys are smart enough to adapt to the environment in which they live. They know
when our satellites
are passing over. They know that we monitor their communications and work to counter that. They’ll counter this too. I’m not saying that something like this won’t produce useful intelligence. I’m sure it will, but we’ll still be left with gaps.
The key to success in the counter-terror fight is Human Intelligence and cultural adaptability. ADVISE will be mining tons of data, but how many of its operators will speak Arabic? Not many based on our thru-put for language training. ADVISE may be able to tip or cue the intelligence community that something might be happening, but without a robust HUMINT follow through, you’re just jousting electrons.
Udate: DefenseTech has a great post
on this. There is a great comment on the post as well:
I wonder how well the data mining stuff can work.
I once read the Earth Liberation Front's "How to Commit Arson" document
(I worry about the Dirt-First types and cyberattacks).
They focused a lot on how to avoid traceability through electronic
records (purchases, etc). If a bunch of shaggy, granola eating, firebombing,
hummer-burning treehuggers can figure it out, couldn't Osama & co?