Machine learning system can detect foreign social media influence campaigns using content alone

Researchers have developed an automated machine learning system they say can detect social media posts involved in coordinated political influence campaigns — such as Russia’s alleged efforts to sway the results of the 2016 elections in the United States — regardless of platform and based only on the content of the posts. Their findings show that content-based features such as a post’s word count, webpage links, and posting time can act like a digital fingerprint for such influence campaigns, which could help social media companies, users, or investigators prevent the spread of misinformation and election interference. Previous attempts to detect coordinated disinformation efforts have focused on simpler approaches, such as detecting bots or comparing the follower/friendship networks of posters. However, these approaches are often foiled by posts from human agents or from new accounts, and are often platform-specific. Meysam Alizadeh and colleagues hypothesized that large, online political influence operations use a relatively small number of human agents to post large amounts of content quickly, which would tend to make these posts similar in topic, word count, linked articles, and other features. To test this, Alizadeh et al. created a machine learning system trained on datasets of early activity from Russian, Chinese, and Venezuelan influence campaigns on Twitter and Reddit. They found the system could reliably identify those campaigns’ subsequent posts and distinguish them from regular posts by normal users. The system was less reliable when it was trained on older data and when the campaign in question was more sophisticated, indicating that such a system would not be a comprehensive solution. The authors suggest that, while widespread use of such machine learning systems could drive bad actors to change their approach and avoid detection, it could also force them to adopt tactics that are more costly or less influential to do so.

###