According to a Feb. 4 report in the BBC, researchers had anywhere from 70 to 90 percent accuracy in predicting future outbreaks of riots, deaths and disease using software that mines the web for news stories.
Scientists from Microsoft Research and the Technion-Israel Institute of Technology jointly developed this software which uses archives from the New York Times plus data from web sites like Wikipedia and Free Base to learn from web content.
Unlike other research in this area, the goal in this project was to create a model that uses a series of past news events to predict the probability of future event using more than 90 sources, according to MIT Technology Review.
For example, in 1973 the New York Times reported that there was a drought in Bangladesh and a year later reported cholera deaths. In 1983, the New York Times reported another drought in Bangladesh and just one year later reported a cholera epidemic. The same pattern happened in Africa in 2007.
With this software, a cholera epidemic could have be predicted after the drought and an alert issued to the medical community so it was prepared for a cholera outbreak.
The software's creators told MIT Technology Review that while many things about the world change, patterns of human nature and the environment remain basically the same so this software can learn from these patterns and predict what's ahead.
“Eventually this kind of work will start to have an influence on how things go for people”, Mr. Horvitz, the co-director of Microsoft Research, told MIT Technology Review.