As the volume of intelligence data grows, agencies must be able to store and move massive amounts of information. By Peter Buxbaum GIF Correspondent Geospatial intelligence practitioners have an insatiable appetite for data. They want to slice, dice, crunch and analyze that data, and they want to keep it for long periods of time. Meanwhile, geospatial data sets are growing exponentially, especially with the increased utilization of imagery, video and LiDAR. The National Geospatial-Intelligence Agency expects to be collecting four petabytes of data annually in coming years. That’s 4 million gigabytes. These conditions challenge users of geospatial data on two levels. The first is the question of how and where to store the massive volumes of data. Another is how to move the data around for analysis and exploitation without degrading the performance of networks and systems. There are a number of approaches being taken today to handle the problems associated with “big data.” Storage systems are being developed that attempt to make the most-used data available on a priority basis. There are systems that break up 4 | GIF 10.3
and distribute massive files so that they can be processed with available computational resources and then reassembled when needed. Various management techniques are being brought to bear, and software is being deployed to handle the storage and processing of large geospatial intelligence data files. Storage-as-a-service is a leadingedge approach to handling the problems associated with big data. “Examining the world requires large amounts of data, no matter if we are looking at large regions with broad areas of coverage, or smaller areas with high levels of detail,” said Jason Dalton, senior director for applied technology at GeoEye Analytics. “In that sense, storage has always been an issue for geospatial systems. Geospatial intelligence requires us to have a thorough picture of the ground truth, which means imagery, terrain, demographics, infrastructure, events and all of their attributes come into play. With these data requirements come large storage requirements.”
The number of satellites gathering geospatial data has increased in the last few years, but more importantly, the quality of the imagery has improved dramatically. “It is not just the volume of data, but the intensity of the data that is increasing,” said Kevin Haar, chief executive officer of Appistry. “This sensor data may include visual images or video, LiDAR, infrared, audio or other types,” added Pratish Shah, director of marketing at Quantum3D. “Video captured at 1080p resolution as a UAV is flying over an area over many hours captures gigabytes and gigabytes of data.” “The problem is being compounded by how much data is coming from advanced sensor platforms like Blue Devil and Gorgon Stare,” said Patrick Humm, president of Hie Electronics. “Analysts want to review the data over a multi-year time span. So it is one thing to gather a petabyte of data. It is another thing to access critical data nine months or two years later so that you can compare that with the www.GIF-kmi.com