An average organization makes decisions based on only 12% of its available data because it’s “too messy” to deal with. Experts predict that the amount of data will increase by 800% in the next five years, and most of it will be multi-structured. (Source: DZone Guide To Big Data 2016)
“Dark” Data Challenge
The enormous amounts of data generated by smart devices, operational machinery, equipment and sensors, etc. (aka “dark” data) vastly exceed the current capabilities to process this data.
For example, a single turbine compressor blade can generate 500GB per day, while a wind farm may generate 150,000 data points per second.
Today, the majority of this data is poorly analyzed or not collected at all. The ability to analyze, synthesize and draw conclusions from the massive amounts of “dark” data enables new levels of operational effectiveness, situational awareness, and ultimately intelligent decision making.
- Cybersecurity Analytics: applying big data processing techniques to real-time and historic analysis of cybersecurity threats.
- Computer Vision: object recognition within images/video; anomaly detection & change classification; deep learning of image content; facial recognition.
- Autonomous Navigation & Swarm Theory: research of deep neural networks for collision avoidance and cooperative mission behaviors of drones.
- Time Series & IoT: condition-based maintenance and preventive analytics; IoT/sensor-based solutions; telemetry data; etc.
- Other use cases, including: news & events analysis; people behavior, etc.
- Enhancing security of complex computer networks by analyzing large streams of diverse data signals to identify cyber threat patterns and adversarial behaviors.
- Business intelligence through analysis of images, videos, and unstructured geospatial data.
- Training self-navigation software for autonomous flight missions of drones.
- Cost savings and operational availability of equipment and machinery through condition-based preventive maintenance.
- Battle space awareness by creating a seamless tactical operating picture that unifies military intelligence, operations and communications.
Case Study: Big Image Data
What is achievable?
- Given a set of input images, automatically analyze those images to find specific features of interest.
- Users may run object searches based on visual samples without individual preprocessing of image archives.
- Geolocation of photographs through fusion of satellite and street data.
- Classification of microscope images to enable cell disease diagnostics.
- Identification of agricultural crops based on high-resolution satellite imagery.