New Hampshire Primary: Social Media Analytics vs. PollsFebruary 11, 2016
New Hampshire Assessment: Think. Create. Learn. ReThink. ReCreate. ReLearn.
When looking back at Iowa, we saw that our projections were more accurate than polling. With New Hampshire, the tables turned.
As with any experiment, it’s important to reassess inputs and outputs. Therefore, as we continue to test our hypothesis that social media analytics are better at predicting elections outcomes than traditional polling, we’ll likewise reassess our methodology regularly to yield more accurate results.
Today, we’ll be diving into our analytics to address potential flaws in our approach, and to course correct. We’ll let data and insights guide us. In other words, we’re going to optimize our campaign!!
The Missing Hashtags
Our methodology was founded on a consistent tracking approach across all candidates. Specifically, our social media listening tracking tags included:
- Candidate name, e.g. “Donald Trump”
- Handle, e.g. @realDonaldTrump
- Campaign hashtag, e.g. #Trump2016 and #Donald2016
Heading into New Hampshire, we quickly spotted an anomaly that we didn’t notice in Iowa. Our data predicted that Hillary would win—but every poll had Bernie leading by double digits. Rather than tinkering with our approach, we kept our methodology pure so that we could dissect it after the final results came in.
To help frame the scale of this flaw, I used our hashtag analytics report to compare Donald with Bernie and Hillary. The data in the charts below show the top three hashtags used whenever someone talked about a candidate on social media.
This was an apples-to-apples comparison, created from having searched the same time period, sentiment and geography for each candidate. Our results show that #feelthebern is still crushing it, beating out #makeamericagreatagain by 102% and #imwithher by 138%.
Our experiment only accounted for the tags “#Bernie2016” and “#Sanders2016” (official hashtags associated with Bernie’s campaign). “#feelthebern” was the brainchild of a Ms. Winnie Wong whose whimsical idea for a hashtag went viral.
The beauty of social media is its free-form self-expression, but in terms of our experiment, our data under represented Bernie compared to Hillary because it didn’t pick up posts that included #feelthebern without our monitored tags.
After using our hashtag insights, we adjusted our approach and created a second U.S. election feed that included the top unstructured hashtags for each candidate.
For South Carolina and Nevada, we’ll report on differences between U.S. Election [Feed] 1 and U.S. Election 2. U.S. Election 1 is now our control group!!!
Influence vs. Activity
Our hypothesis assumed that ZPoints®, a measure of the level of attention that each and every social media post gets,correlated to “likelihood” of an individual voting for a specific candidate.
This was very different application of ZPoints® than the one that it’s intended to have. Clients typically use ZPoints® in various way, but two of the more popular applications include:
- Campaign Optimization: To quickly identify content and market messages that resonate with their customers.
- Influencer Marketing: To understand and connect with influencers shaping the conversation about a specific topic.
We began evaluating if our KPI measuring “activity with positive sentiment” better represented a person’s voting preference than ZPoints®.
Using that KPI, any activity with positive sentiment is counted equally. In theory, this aligns with the democratic election process. The problem? It may not filter out noise. We’ll continue to compare and contrast these two metrics to see which better indicates outcomes.
There are other factors that we could consider, as noted in the post-Iowa assessment. Our Zoomph analytics suite includes numerous KPIs and signals that could help paint the picture of a complex and constantly evolving story.Though we’re only highlighting a few of the many KPIs available to us here, we’re excited to share how the benchmarks, real-time data, and the ability to pinpoint influential content can help campaign managers monitor competitors and refine their approach.
Learn more about our day-by-day analysis of the New Hampshire Primary below.
Predicting New Hampshire Results with ZPoints®
With Iowa results behind us, we turned our attention to New Hampshire to test our hypothesis:
Social media analytics are better at predicting election outcomes than traditional polling.
Each day we’ll update this post by comparing real-time social data from Zoomph to newly released polls. We’ll be tracking candidates consistently by monitoring conversations that include their name; official campaign hashtag; and official handles across Twitter, Instagram and Google+. On Facebook, we’ll be listening in on the same conversations, but only on their official campaign pages. The key metric we’re using has a combination of filters:
ZPoints®, our proprietary algorithm that scores the relative influence of each conversation. This measure is more reliable than “total mentions” because it significantly reduces weighing of spam and bots. You can read more about ZPoints® here.
We are using positive sentiment analysis because we believe it indicates a likelihood to vote.
We are looking at social media originating from New Hampshire, plus state-identification in an author’s bio.
According to real-time social data gathered between 8 AM (when the polls opened this morning) and 1 PM today, Donald continues to lead the GOP race. He has gained 3% since our analysis this morning. Marco is gaining with a 4% increase, and is now at 26%. We see Hillary leading too.
Tomorrow, we’ll compare our data to official poll results and provide further insight into our hypothesis.
The polls open at 8 AM. Today we’re comparing data to see if any candidate is surging. This analysis compares data from a 48-hour period that ended on Sunday (11:59 PM), to a 48-hour period ending on Monday (11:59 PM). The day before Iowa, we saw Bernie and Ted surging. Then on the day of the caucus, we saw Marco surging. We’ll continue to monitor details throughout the day.
- We see Donald surging 6%, the largest gain of any candidate.
- Our algorithm helps filter out noise. This is the first time we’ve seen Donald’s ZPoints® share of 39% outpace his share of Total Activity (mentions), which is at 31%.
- Ted dropped 3%.
- We continue to see Hillary leading a tight race. This is against conventional wisdom, and we’ll evaluate our methodology after the results come in.
We’ll post one more update before the polls close.
Another interesting difference in methodologies that influences accuracy is sample size.
- Recent polls average between 362 to 508 telephone interviews.
- Remember, our filters look at 48-hours + influence + positive sentiment + New Hampshire. Today’s GOP analysis is based on 4,265 people engaging with content posted by 641 Unique Authors,
We’re going to keep it simple today and compare our stats to RealClearPolitics (RCP) average of all recent polls.
- We think it’s a much tighter race. Donald’s winning margin is 13%, a 4% difference versus RCP’s poll average. ARG’s tracking Poll is the next closest to us at 14%.
- With 20%, Marco has 5% more support with Zoomph ZPoints® share-of-voice, up 1% from our analysis yesterday.
- We continue to see Hillary leading a tight race.
We expect the race to get even more exciting as people head to the voting booth.
As seen in our chart below, the GOP debate attracted a lot of attention on social media.
- Marco remains in second but slipped 5% versus our stats from yesterday
- While we have him in third (vs. fourth in ARG), Ted dropped 6%.
- The three GOP Governors are surging:
- Jeb and John rise 3% and are tied for fourth at 9%.
- Chris had the biggest lift, gaining 5%, and is now at 8%.
After the huge spike in conversations due to the debate, it’ll be interesting to see how data normalizes in the next two days. Follow us on twitter @Zoomph_Politics to get the latest updates.
Several new polls were released and can be found at RealClearPolitics. We’re comparing our updated social media analytics to The Boston Globe/Suffolk and UMass/7 News polls. We’re using a 48-hour rolling window to track movements and trends all the way through Tuesday night.
Here’s what we see:
- We agree with The Boston Globe that Marco is closing the gap. We have Trump’s lead down to 7.
- Marco overtook Ted and is in second.
- We see a tight race for fourth between Carly, Jeb and John (in rank order). All three are in the high-single digits. The Globe has Ted in fifth.
- ZPoints® has Hillary leading. The poll that has Sanders closest to Hillary has him up by 9.
Comparing Zoomph to NBC/WSJ/Marist Poll released earlier today:
- Donald’s lead (in our data) has dropped 8%, compared to 13%
- Ted is second with 24%, versus third with 15%
- Both Ted and Marco are rising, 3.6% and 3.0% respectively when compared to data we pulled from
January 29th to February 3rd
Check back tomorrow to see how the race changes. We may even add details about Hillary and Bernie.