Big data and machine learning are used in a number of ways: Better AI, improved malware detection, smarter messenger apps, and much, much more. Now, they're being utilized by the military to aid in its fight against Islamic State.

On Monday, the Pentagon announced the establishment of the Algorithmic Warfare Cross-Functional Team (AWCFT), codenamed Project Maven. Its main objective is to use big data and machine learning to shift through the thousands of hours' worth of IS surveillance tapes, which are currently overwhelming military and civilian intelligence analysts.

"We have to tackle the problem a different way," Air Force Lt. Gen. John N.T. Shanahan, director of defense intelligence for warfighter support, told Defense One. "We're not going to solve it by throwing more people at the problem. That's the last thing that we actually want to do. We want to be smarter about what we're doing."

It's hoped that Maven will enable analysts to make better use of their time, 80 percent of which is currently spent doing mundane, administrative tasks like entering data into spreadsheets.

60 percent of drone video consists of the craft flying to and from the points of interest. There's also the problem of clips that show little more than hours of impenetrable bad weather. Automation could allow analysts to examine only the important content without having to trawl through all the footage.

AWCFT has 90 days to develop or acquire the data labeling algorithms. The next two phases, both of which are also 90 days long, entail procuring the required hardware and software before implementing the system into existing intelligence projects.

"Once we show success, people are going to say what else can we apply this to," added Shanahan. "To me that breaks things wide open and we're going to figure out how we really, at scale, bring in some of these capabilities into the department."