Gillian Tett - Polarisation in cyberspace tends to herald street fights

Gillian Tett made some critical observations in the FT on the role of social media. Writing here, she said:

"One small sign can be seen in the proliferation of Facebook pages in Egypt which have taken radically opposing and increasingly extremist sides. But another illustration appears in some fascinating new research by Ingmar Weber and Venkata Garimella, two data scientists at the Qatari Computing Research Institute, working in conjunction with Alaa Batayneh, a data analyst at Al Jazeera.

Earlier this summer, the Doha-based group used powerful computers to crunch through 17 million-odd tweets that had been dispatched by 7,000 Egyptians, on both sides of the political spectrum, in the year to June 2013. They did this by using a methodology developed by data scientists who track political polarisation in America: first they sorted people who tweet (or “tweeps” as they are called) into different “secularist” and “Islamic” categories, based on their tweeting activity; then they measured whether the different camps have become more polarised in what they tweet, retweet and “mention”, and – most crucially – how they use hashtags (“mentions” are tweets which contain another tweep’s username; hashtags are labels marked with a #; both enable tweets to be grouped and searched). They then plotted this on coloured charts.

The aim is to see whether groups are retreating into intellectual ghettoes in their patterns of communication, or engaging in overlapping conversation threads. If the tweeps are using hashtags on subjects of wide interest, such as the latest smartphone or a popular singer, there is often a sense of common identity; but when the tweeps are discussing the former president Mohamed Morsi, views are more polarised.

The results are thought-provoking. When the conversation patterns were plotted on a chart, it showed a sharp overall increase in polarisation over the past year. However, it fluctuated sharply within that upward trend and “for the last 12 months large values on this polarisation barometer coincided with periods of violence”, the group notes in a research paper. “We found strong indications that a measure of global hashtag polarisation, related to the overlap between hashtags used by the two political sides, works as a ‘barometer for tension’.”

Now, the researchers stress that “we have been careful not to draw conclusions about causal connections and have [just] restricted ourselves to observing correlations”. They are not, in other words, saying that polarised tweeting actually creates violence – or vice versa. And data scientists themselves are currently divided about the degree to which social media platforms are apt to exacerbate the levels of political polarisation, or not. In recent years some American commentators – such as Cass Sunstein and Eli Pariser – have won plaudits by warning that social media tend to fuel polarisation and extremism, by creating echo chambers where like-minded people only communicate with people like themselves. However, groups such as Facebook contest this: it has supported research over the past year which argues that social media actually expose their users to a wider range of viewpoints than before.

Meanwhile another fascinating and nuanced piece of research has emanated from a team at Indiana University in Bloomington, suggesting platforms such as Twitter can promote both flocking and flying apart. These researchers have analysed 250,000 tweets that were dispatched in the six weeks before the 2010 US elections, and concluded that while Democrats and Republicans were highly polarised when they were tweeting or retweeting news, they overlapped when tweeps “mentioned” other people or used hashtags. Twitter users might fly in divergent “Republican” and “Democrat” packs but they are aware of each other.

So what does that mean for the tweeters in Egypt? The data scientists in Qatar are now planning more extensive research and hope to roll this out to other arenas soon, such as the Israeli-Palestinian conflict. But in the meantime, their one early conclusion is that tweets have potent predictive power: polarisation in cyberspace tends to herald street fights. Or to put it another way, if you want to work out what is going on in a country, you do not always need to spy on individual emails with NSA-style techniques. Sometimes looking at 17 million (anonymous) tweets with big data techniques can be equally powerful at pointing to unrest –and signal to the authorities that it is time for a policy change."