Social intelligence: social media data as a means for market intelligence

Social intelligence: social media data as a means for market intelligence

To be successful as researchers, we need to know our customers. We seek to identify our customers’ needs, desires, and opinions about our products, brands, and competitors. In doing so, we understand better how to tailor our products and services to meet our customers’ needs.

Traditional market research identifies subgroups within a target audience through direct engagement, in-person or through email engagements. Researchers can explicitly define and ensure an individual’s association to a given audience. It’s an effective approach, but it can be expensive, time-consuming, and difficult to execute.

Where traditional market research and social media meet

At Microsoft, the Research + Insights (R+I) team focuses on answering the same business questions as sought through traditional market research. However, instead of using standard methods, we utilize the vast set of available data in social. Over the past year, we asked questions like: What if we could leverage public social media feeds to understand what specific audiences people belong to? What messages might be most likely to have a positive impact? What if we could align social market research methods to traditional market research methods to drive a greater understanding of the customer in more dynamic ways?

Microsoft’s R+I team has developed a method for identifying and grouping social media users based on their online behavior and comments. This is a “listening-based” approach that can be automated to sift through massive amounts of social media traffic and metadata to create business-relevant segments and to find relevant insights.

How a “listening-based” approach to social intelligence works

For most social users, their profile encompasses their varying identities. A single profile could perhaps represent a marketing professional, a mother, and someone who likes to travel and dine out. Within this profile, that individual is apt to identify her interests and offer commentary on each of them. A Twitter profile bio, for instance, may contain the line: “Professional marketer, mom of two kids, can’t wait for next trip to France.” Chances are, this individual makes social posts relevant to each of those three personae, as well as a variety of other topics.

The advantage of our approach is that our audience members self-identify, then engage in natural and organic conversations in a way that can’t be replicated through traditional market research. By utilizing biographical word tagging in social media, we strive to achieve a minimum content inclusion accuracy of 80 percent for audience inclusion and conversation relevancy. Meaning when we analyze 100 profiles in the social media grouping, at least 80% of those profiles are relevant to the grouping. For our Topic inclusion, we apply a similar methodology. This ensures our data is clean before we analyze it at the aggregate level.

We can expand on the biographical word tagging, using the frequency and relevancy of terms mentioned, to bucket groups for increased accuracy. For example, let’s take software developers. By assigning points based on mentions of developer languages (1 point), developer tools (2 points), developer conferences (3 points), and developer domains (5 points), we can identify group members and segment groups by amateur or professional developers, with those amassing the most points seen as “most active” developers. The methodology above is just an example of how weighted behavior across specific audiences can be leveraged to create meaningful groupings then used to distill business-relevant insights.

Below is an example output analyzing what is resonating from self-identified Microsoft Employees, Partners vs. the general population and themes in hybrid work discussion.

We can perform similar group identification among commercial, educational, and consumer customers, as well as Microsoft-centered customers (such as partners or Xbox fans). We can also infer group identification without direct self-identification. We know, for instance, that 70 percent of millennials follow Dan Price, the CEO of Gravity Payments. So, if we come across an individual who also follows Dan Price, in addition to other social behavioral markers scored similarly to the developer example above, we can infer that he or she likely is a millennial. This is a broad example, but you get the idea.

Customer privacy is at the core of what we do. When we are analyzing user conversations we are doing so at an aggregate level and not on the individual level to ensure personal identifiable information (aka PII) is not reported against. It is built into our best practices to scrub author names and users’ names from the records when we report against these audience segments.  

What’s next for social insights?

The most important piece is using our initial data to understand common characteristics in our audiences and to isolate common social traits within each audience. With that, we can broaden our groups by understanding the relevancy of behavior by the audience and then scoring (or weighting) those behaviors.

Once we’re able to group a certain number of identified behaviors, our understanding of the audience becomes more accurate. We’ll also be able to include individuals who don’t self-identify as part of our target audience. If they follow patterns similar to people who do self-identify, then we can include them within a group. At the end of the day, all of this is done to better understand how to address the needs of our customers in order to continue to empower them to achieve more.

Authors: Allie Webster & Justin Schoen

Source: Greenbook