Can big data analytics predict population-level societal events such as civil unrest or disease outbreaks?

That is the subject of a two-year analysis of the
Early Model Based Event Recognition using Surrogates (EMBERS)
system. The usefulness of this predictive artificial intelligence system for population-level events could be important. If existing models, which successfully predict the past, were good enough no one would ever lose money in the stock market.

In a Big Data article, Andy Doyle and coauthors from CACI, Inc., Virginia Tech and BASIS Technology describe the structure and function of the EMBERS system. They describe EMBERS as a working example of a big data streaming architecture that processes large volumes of social media data and uses a variety of modeling approaches to make predictions.

"EMBERS represents a significant advance in our ability to make sense of large amounts of unstructured data in an automated manner," says Big Data Editor-in-Chief Vasant Dhar, Co-Director, Center for Business Analytics, Stern School of Business, New York University. "The authors present an architecture that provides a scalable method for dealing with large streams of social media data emanating from Twitter. Although the focus of the paper is on predicting social unrest globally, the methods should be usable for processing these type of data for a variety of applications."

Citation: Forecasting Significant Societal Events Using The Embers Streaming Predictive Analytics System: Doyle Andy, Katz Graham, Summers Kristen, Ackermann Chris, Zavorin Ilya, Lim Zunsik, Muthiah Sathappan, Butler Patrick, Self Nathan, Zhao Liang, Lu Chang-Tien, Khandpur Rupinder Paul, Fayed Youssef, and Ramakrishnan Naren. Big Data. December 2014, 2(4): 185-195. doi:10.1089/big.2014.0046