Russian Twitter Trolls

Russian Twitter Trolls


As part of the House Intelligence Committee investigation into how Russia may have influenced the 2016 US Election, Twitter released the screen names of almost 3000 Twitter accounts believed to be connected to Russia’s Internet Research Agency, a company known for operating social media troll accounts. Twitter immediately suspended these accounts, deleting their data from Twitter.com and the Twitter API.

A team at NBC News including Ben Popken and EJ Fox was able to reconstruct a dataset consisting of a subset of the deleted data for their investigation and using Neo4j were able to show how these troll accounts went on attack during key election moments. NBC News open-sourced the reconstructed dataset and released it as this Neo4j database.

This interactive Neo4j Browser guide will help you to explore the dataset of Russian Troll tweets by guidng you through:

  • an overview of the datamodel

  • how to explore the data using Cypher, the query language for graphs

  • some of the investigative queries used to make sense of the the dataset

Read the NBC News story about the data here

NOTE: Much of this content is offensive and explicit. Query with caution.

The Datamodel


Neo4j uses the labeled property graph datamodel to represent data.

Nodes

Nodes can have one or more node labels to describe the "type" of the node.

  • Tweet - The Tweet

  • User - A Twitter user

  • Troll - An optional node label to identify users that are on the House list of Russian troll accounts

  • Hashtag - Any hashtags used in the tweet

  • Source - The Twitter application used to post the tweet

  • URL - A link embedded in a tweet

Relationships

Relationships have a single relationship type.

  • POSTED - Connects the Tweet with the User who posted it

  • MENTIONS - Connects any Users that are @-mentioned to the Tweet

  • HAS_TAG - Connect a Tweet to any Hashtag nodes included in the tweet

  • POSTED_VIA - Connects a Tweet to its Source application node

  • RETWEETED - Connects a Tweet that retweets another Tweet

  • IN_REPLY_TO - Connects a Tweet that is a reply to another Tweet

Nodes and relationships can store arbitrary key-value pair properties. This image shows the properties and data type used in this database.

Execute this query to inspect the datamodel in the database:
call db.schema.visualization()

Introduction To Cypher


In order to work with our labeled property graph, we need a query language for graphs.

Graph Patterns

Cypher is the query language for graphs and is centered around graph patterns. Graph patterns are expressed in Cypher using ASCII-art like syntax.

Nodes are defined within parentheses (). Optionally, we can specify node label(s): (:User)

Relationships are defined within square brackets []. Optionally we can specify type and direction:

(:Tweet)+←[:POSTED]-+(:User)

Graph elements can be bound to aliases that can be referred to later in the query:

(t:Tweet)+←+[r:POSTED]-(u:User)

Predicates

Filters can be applied to these graph patterns to limit the matching paths. Boolean logic operators, regular expressions and string comparison operators can be used here.

Be sure to use the Cypher Refcard as a reference for learning Cypher syntax.

Dissecting a Cypher Statement

Let’s look at a Cypher query that answers the question "For Users with over 1000 followers, how many tweets did they post?". Don’t worry if this seems complex, we’ll build up our understanding of Cypher as we move along.

Click on the block to put the query in the top-most window on the query editor. Hit the triangular button or press Ctrl+Enter to run it and see the resulting visualization.
MATCH (u:User)-[r1:POSTED]->(t:Tweet)
WHERE u.followers_count > 1000
WITH COUNT(t) AS num
RETURN num

find

MATCH (u:User)-[:POSTED]->(t:Tweet)

Search for an existing graph pattern

filter

WHERE u.followers_count > 1000

Filter matching paths to only those matching a predicate

aggregate

WITH COUNT(t) AS num

Count number of paths matched

return

RETURN num

Specify columns to be returned by the statement

Querying The Graph - Overview


Neo4j Browser is an interactive query workbench tool for executing Cypher queries in Neo4j and working with the results in both graph visualization and tabular forms.

Graph Results

If our Cypher query returns graph objects (Nodes, Relationships, Paths) then Neo4j Browser will render a graph visualization for us to explore.

Tabular Results

Often the answer to our question however is tabular data - for example, the result of running an aggregation. We can also return rows of scalar values in Cypher. In that case Neo4j Browser will render a table of rows.

Find tweets with the hashtag thanksobama
MATCH (u:User)-[:POSTED]->(t:Tweet)-[:HAS_TAG]->(ht:Hashtag {tag: "thanksobama"})
RETURN * LIMIT 50
Overview query. What are the node labels in the graphs and how many of each?
MATCH (t)
RETURN LABELS(t), COUNT(*) AS count
ORDER BY count DESC

Querying The Graph - Users


A simple type of query we can run is one that matches on a single node label. In this database we’ve added the node label Troll to any users that show up on the House list of Russian Twitter Trolls.

We can search for the Trolls with the most followers:

MATCH (u:Troll) WHERE EXISTS(u.followers_count)
RETURN u.screen_name AS screen_name, u.followers_count AS followers
ORDER BY followers DESC LIMIT 50

Or use an aggregation function like COUNT() to see the most popular locations that Troll users list in their profiles:

MATCH (u:Troll)
RETURN u.location, COUNT(u) AS num
ORDER BY num DESC LIMIT 100

Or use a predicate filter to search for a specific Troll by screen name:

MATCH (u:Troll)
WHERE u.screen_name = "LeroyLovesUSA"
RETURN u

Querying The Graph - Tweets


We can specify more complex graph patterns using Cypher to search for users plus the tweets they’ve posted, and even the hashtags that those tweets contain.

Tweets posted by the user LeroyLovesUSA with the hashtag thanksobama
MATCH (u:User)-[:POSTED]->(t:Tweet)-[:HAS_TAG]->(ht:Hashtag {tag: "thanksobama"})
WHERE u.screen_name = "LeroyLovesUSA"
RETURN *

We can include predicates to filter for paths matching the pattern and our predicates:

Find tweets posted by Troll accounts that contain the word fraud
MATCH (t:Troll)-[:POSTED]->(tw:Tweet)
WHERE tw.text CONTAINS "fraud"
OPTIONAL MATCH p=(tw)-[:HAS_TAG|HAS_LINK|MENTIONS|IN_REPLY_TO]-(a)
RETURN * LIMIT 50

Tweet Volume


This chart from the NBC News article shows that Russian Twitter troll tweet volume spiked during key election related events.

Query for tweet volume by day
MATCH (:Troll)-[:POSTED]->(t:Tweet) WHERE t.created_str > "2016-10-01"
RETURN substring(t.created_str,0,10) AS day,
  COUNT(t) AS num ORDER BY day LIMIT 60
Query for tweet volume by month
MATCH (:Troll)-[:POSTED]->(t:Tweet)
RETURN substring(t.created_str,0,7) AS month,
  COUNT(t) AS num ORDER BY month DESC LIMIT 25

We can see that much of the tweet volume occurs leading up to and immediately following the 2016 US Election.

Hashtags


Hashtags were used by the trolls to insert themselves into conversations and gain visibility.

What are the most common hashtags used by the trolls?
MATCH (ht:Hashtag)<-[:HAS_TAG]-(tw:Tweet)<-[:POSTED]-(:Troll)
WITH ht, COUNT(tw) AS num ORDER BY num DESC
RETURN ht.tag AS hashtag, num LIMIT 10
Most popular hashtags used by Trolls by day
MATCH (:Troll)-[:POSTED]->(t:Tweet)
WHERE t.created_str > "2016-10-01" AND t.created_str < "2016-11-30"
WITH t,substring(t.created_str,0,10) AS day, [(t)-[:HAS_TAG]->(ht:Hashtag) | ht.tag] AS tags
UNWIND tags AS tag
WITH day, tag, COUNT(t) AS num ORDER BY num DESC
RETURN day, COLLECT(tag)[..5] AS toptags ORDER BY day

This streamgraph from the NBC News article shows hashtag volume by category.

Hashtags - #RejectedDebateTopics


@WordOfHashtags

One of the hashtags used by the Trolls was #RejectedDebateTopics. The Troll account @WorldOfHashtags used it as a way to annouce a "hashtag game" to encourage others to use the same hashtag. Hundreds of non-troll accounts ended up interacting with this acocount by using the #RejectedDebateTopics hashtag.

What are the tweets posted by @WorldOfHashtags with the hashtag #RejectedDebateTopics ?
MATCH (u:Troll {screen_name: "WorldOfHashtags"})-[:POSTED]->(t:Tweet)-[:HAS_TAG]->(:Hashtag {tag: "rejecteddebatetopics"})
WHERE t.created_str > "2016-10-01"
RETURN t.text, t.created_str
ORDER BY t.created_at LIMIT 50

Tweet Sources Used


A variety of applications were used by the Troll accounts for posting tweets.

What are the common sources used for posting tweets?
MATCH (:Troll)-[:POSTED]->(tw:Tweet)-[:POSTED_VIA]->(s:Source)
RETURN s.name AS source, COUNT(tw) AS tweets
ORDER BY tweets DESC

Original Content vs Replies and Retweets


Tweets can be retweets of other tweets or "original content" tweets that do not reference other tweets. Many of the Troll tweets are retweets and replies. We can filter out these retweets and replies and search only for "original content" tweets.

Find original content tweets posted by user @TEN_GOP
MATCH p=(t:Troll {user_key: "ten_gop"})-[:POSTED]->(tw:Tweet)
WHERE NOT EXISTS((tw)-[:RETWEETED|IN_REPLY_TO]->(:Tweet))
OPTIONAL MATCH (ht:Hashtag)<-[:HAS_TAG]-(tw)
OPTIONAL MATCH (u:User)<-[:MENTIONS]-(tw)
RETURN p,ht,u LIMIT 25

Inferred Relationships


Inferred relationships are important in graphs. For example, when a Troll account retweets another Troll’s tweet we could say the trolls have an inferred "AMPLIFIED" relationship: one troll is amplifying the message of the other.

Inferred AMPLIFIED relationships exist when a troll account retweets another troll
MATCH p=(r1:Troll)-[:POSTED]->(:Tweet)<-[:RETWEETED]-(:Tweet)<-[:POSTED]-(r2:Troll)
RETURN p LIMIT 1

Graph Algorithms


Graph algorithms are a way to apply analytics to the entire graph to further enhance our understanding of the data. These algorithms fall into three categories:

  • Centrality - What are the most important nodes in the network? PageRank, Betweenness Centrality, Closeness Centrality

  • Community detection - How can the graph be partitioned? Union Find, Louvain, Label Propagation, Connected Components

  • Pathfinding - What are the shortest paths or best routes available given cost? Minimum Weight Spanning Tree, All Pairs- and Single Source- Shortest Path, Dijkstra

We can run these algorithms in Neo4j with Cypher using the Neo4j Graph Algorithms procedures.

See what procedures are available in the gds package
CALL dbms.procedures()
YIELD name, signature, description
WITH * WHERE name STARTS WITH "gds"
RETURN *

Centrality


Let’s run PageRank over this inferred AMPLIFIED graph to find the most influential trolls

Run PageRank over the inferred troll amplification graph. This will write the results back to a pagerank property on the nodes
CALL gds.pageRank.write({nodeQuery:"MATCH (t:Troll) RETURN id(t) AS id", 
  relationshipQuery:"MATCH (r1:Troll)-[:POSTED]->(:Tweet)<-[:RETWEETED]-(:Tweet)<-[:POSTED]-(r2:Troll) RETURN id(r2) as source, id(r1) as target",
  writeProperty:'pagerank'})
Find Trolls with highest PageRank score
MATCH (t:Troll) WHERE EXISTS(t.pagerank)
RETURN t.screen_name AS troll, t.pagerank AS pagerank ORDER BY pagerank DESC LIMIT 25
What are the top trolls tweeting about?
MATCH (t:Troll) WHERE EXISTS(t.pagerank)
WITH t ORDER BY t.pagerank LIMIT 25
MATCH (t)-[:POSTED]->(tw:Tweet)-[:HAS_TAG]-(ht:Hashtag)
RETURN ht.tag, COUNT(tw) AS num ORDER BY num DESC LIMIT 25

PageRank is a recursive graph algorithm that defines the importance of a node proportional to the importance and number of connected nodes in the graph.

Community Detection


We can also run community detection over this inferred AMPLIFIED graph to see how the graph is partitioned

Partition the graph into communities using the Label Propagation algorithm. An additional property community is added to each node specifying the community it has been assgined to by the algorithm.
CALL gds.labelPropagation.write({nodeQuery:"MATCH (t:Troll) RETURN id(t) AS id", 
  relationshipQuery:"MATCH (r1:Troll)-[:POSTED]->(:Tweet)<-[:RETWEETED]-(:Tweet)<-[:POSTED]-(r2:Troll) RETURN id(r2) as source, id(r1) as target, count(*) as weight",
  writeProperty:'partition', relationshipWeightProperty:'weight'})

We can then see which Trolls were assigned to each community:

MATCH (t:Troll) WHERE EXISTS(t.partition)
RETURN COLLECT(t.screen_name) AS members, t.partition AS community
ORDER BY SIZE(members) DESC LIMIT 10

And finally, we can see if there are certain themes that each community was focused on, by inspecting the most common hashtags used by each community:

MATCH (t:Troll) WHERE EXISTS(t.partition)
WITH COLLECT(t) AS members, t.partition AS community
ORDER BY SIZE(members) DESC LIMIT 10
UNWIND members AS t
MATCH (t)-[:POSTED]->(tw:Tweet)-[:HAS_TAG]->(ht:Hashtag)
WITH community, ht.tag AS tag, COUNT(tw) AS num ORDER BY num DESC
RETURN community, COLLECT(tag)[..10] AS toptags
LIMIT 10

Further exploration


Ideas

We’ve only just begun to explore the data. Here are some ideas for further data analysis:

  • Location

  • Entity extraction

  • Temporal data analysis (are tweets posted at a certain time of day?)

  • Clustering the graph

  • Exploring most co-mentioned users and hashtags