Using AI to Extract Sociology from the Web

Practically everything on the Web is in text form so it's hard to search unless you have unlimited time.  In this research, the scientists develop a better search process in which the software refines the search as it goes.  (Science Daily: Artificial-intelligence system surfs web to improve its performance)

Information extraction -- or automatically classifying data items stored as plain text -- is thus a major topic of artificial-intelligence research. Last week, at the Association for Computational Linguistics' Conference on Empirical Methods on Natural Language Processing, researchers from MIT's Computer Science and Artificial Intelligence Laboratory won a best-paper award for a new approach to information extraction that turns conventional machine learning on its head.

- Science Daily

That's a bit melodramatic about turning anything 'on its head' but they did come up with a substantively different way to do searches.

In this one, we're most interested in searches of a disparate nature.  For instance, maybe we want to correlate stock market performance with how often kids shoot their parents with the parent's gun.  A kid got a two-fer in the last few days by shooting the parent and a sibling with one shot.  Wow.

Ed:  why?

It doesn't matter why that particular search although I am curious about that one.  The interesting part is such searches are straight out of Hari Seldon's psychohistory; analysis of disparate events yields, possibly, predictive value overall.

Ref:  "Foundation" by Isaac Asimov  (WIKI:  Foundation series)

The premise of the series is that the mathematician Hari Seldon spent his life developing a branch of mathematics known as psychohistory, a concept of mathematical sociology.  Using the laws of mass action, it can predict the future, but only on a large scale.  Seldon foresees the imminent fall of the Galactic Empire, which encompasses the entire Milky Way, and a dark age lasting 30,000 years before a second great empire arises.  Seldon also foresees an alternative where the interregnum will last only one thousand years.  To ensure the more favorable outcome, Seldon creates a foundation of talented artisans and engineers at the extreme end of the galaxy, to preserve and expand on humanity's collective knowledge, and thus become the foundation for a new galactic empire.

- WIKI

In the researcher's attempts to find an algorithmic analysis of Web traffic, it's along the lines of mathematical sociology and the concern is regarding the source information since it assumes The Web Tells Us Everything.  It's a tremendous sociological review but it's short of the tools Hari Seldon was using since the current research doesn't result in anything predictive from the information it generates.  It's predictive in terms of the way it generates the information but not in what it does with it.

Ed:  with as many gypsy fortune tellers as are running the news, how long can it be before it's used in a predictive way, most likely in an inappropriate way insofar as it extends the conclusion over a wider population than was surveyed.  That happened frequently during the course of the campaigns for the election and, not surprisingly, many were shocked by the result.

Yep, this could easily become another news media abomination.  There's nothing for it but to wait and see ... while watching what the sciencers are doing.

Note:  'science' became a verb in "The Martian" which starred Matt Damon.  When 'science' becomes a verb, a sciencer is someone who sciences.  Now you know ... scientifically speaking.