Navigating Predictive Analytics: Benefits and Challenges of Using Graph Theoretic Methods Leveraging Contextual Network Information for Better Accuracy

Nick Lim is the CEO and Founder of Sonamine, LLC. Over the past 15 years, Nick has worked in analytic environments with large amounts of data. Leveraging his training at Harvard, he has led product and strategy teams at MicroStrategy, Enpocket and Nokia to build systems that predicted ad clicks and consumer behavior using terabytes of information.

Predictive Analytics: Benefits and Challenges of Using Graph Theoretic Methods Leveraging Contextual Network Information for Better Accuracy

The practice of predictive analytics has come a long way since the advent of operations research. Current predictive models routinely use more than 100 variables, most of them are characteristics of the object being analyzed, including age and gender for customer analysis, or color and text for advertisements. In this article, we will explore how graph methods can be used in predictive analytics.

Graph methods refer to techniques that analyze a network of objects, rather than pie charts and trend lines. Networks are made up of nodes and connections among the nodes. Other synonyms include vertices, edges, and arcs. Graph methods analyze the link structure, rate of diffusion and the clustering of nodes in a network. Many concepts in the popular press such as Google’s page rank, social network analysis, influencer marketing and driving directions all utilize graph theoretic methods.
Real World Use Cases
Why are these graph methods useful for predictive analytics? One reason is that additional contextual predictors can be derived using graph methods. Let’s start with a well known example: Google uses page rank as one of many predictors for the relevance of a web page. The link structure in the world-wide-web network provides valuable contextual information about which pages are deemed most relevant by the web page creators—this contextual link structure is then used to predict relevance for a user’s query.

Another example: If you are trying to predict the revenue contribution of each employee, in addition to the educational level, job responsibility and experiences, you might consider how well the employee relates to the customers or other managers. The contextual information added here is not something inherent about the employee. IBM conducted such a study with 2600 of their consultants, and created a model with revenue as the outcome target variable and three sets of predictor variables.1 In addition to the standard employee and job-related characteristics, two sets of contextual variables were added. One was related to the structure of the web of relationships, and the other was related to how each employee was related to managers and customers. It turned out that graph-derived contextual predictors were highly correlated with revenue.
Enriching Context Where It Seemingly Did Not Exist
One common question raised is how much of the contextual information is already captured in the demographic data. After all, there is a common understanding that birds of the same feather flock together. To tackle this question, a group of researchers at MIT studied how the Yahoo instant messenger network affected the adoption of the Yahoo Go product.2 They discovered that only 50% of the contextual network effect could be explained by the common underlying demographic variables.

This finding has big implications for predictive analytics in data poor environments. A data poor environment would benefit greatly from using contextual predictors that you already have. For example, if you are trying to predict churn in a prepaid mobile customer base where there is little to no demographic information, using the social network contextual information can greatly improve the prediction accuracy and lower the false positive rate.

Another type of contextual network information that can be included into predictive analytics is the underlying form of the social community that one belongs to. In the study with Yahoo data above that was trying to predict product adoption, they found that the chances of you adopting a product rises if more people in your social community have already adopted the product.

Besides social contextual network data that can be added to predictive functions, you can also deduce contextual information. For example, in trying to predict fraud from a set of credit applications, we can construct a network of information attributes. Using the basic idea of that people with bad credit tended to relate to other similar people, some researchers generated the graph-based hub and authority variables from such a network and added them to a support vector machine prediction function.3 The graph-derived context variables improved the fraud prediction by approximately 20%.

Clearly, the benefits of adding graph-based predictors are better lifts and lower rates of false positives. The improvements range from 20% to 100% depending on the richness of data available today. These benefits translate directly into bottom line results to the extent that your company is already putting predictive analytics into daily operations. So what are the practical aspects of putting this into operation?
How Do You Get Started?
Let’s start with the data: Where do you obtain the data to construct such contextual networks? The IBM study looked at the anonymized email traffic, address books and buddy lists. If you were a retailer like Amazon, you could use wish lists, product gift transactions and share-the-love data to construct the social context for predictions. Financial services companies can use the trading data to construct the contextual network among the traders or credit application data to construct an information network. Telecommunications companies can leverage the call detail records. In fact, the explosion of data collection means that most organizations already have this contextual network information just waiting to be accessed and utilized.

One unique aspect of contextual network predictors is that they are usually specific to the target outcome you want to predict. This is the definition of contextual after all. So if you want to predict churn, then you need to understand how the contextual network predisposes a subscriber to churn. Such contextual churn insight will certainly be different if you were trying to predict employee revenue or product adoption.

Since the contextual environment is changing, the window that an enterprise has to leverage the additional predictive power of contextual networks is also dynamic. Most customers will switch vendors very soon after they learn about a bad experience from a friend, so vendors must respond in a timely manner. Practically, this means that the calculation of graph-based predictors must occur more frequently.

Some contextual network information is relatively stable over time. For example, if someone has a preferred takeout restaurant, she will likely continue to call that restaurant even after she gets a new phone number. These “creatures of habits” can be more easily identified, say in the case of fraud or terrorism prevention.
Challenges
One potential impediment to leveraging contextual network information involves the privacy and data security issues. Since contextual networks tend to involve personal transactions, companies would like to ensure the highest security to prevent breaches. Various regulatory bodies at different state, country or global levels might have conflicting guidelines on what information is considered private and cannot be used for data analytics.

Another obstacle today is the scalability challenge. Often these contextual networks are many times the size of the current data used for analysis. Contextual networks rely on relationship information, so sampling runs the risk of losing valuable information. James Kobelius of Forrester postulates that the advent of social network analysis will catalyze the need for petabyte-sized data warehouses.4

Another obstacle is the lack of software tools that will generate these graph-based predictors. Types of specialized graph based calculations include community detection, diffusion simulation, vertex similarity, topology and centrality analysis. Open source tools such as Jung and Pajek work well for R&D, while proven commercial tools such as Sonamine are geared toward large scale and production graph mining.

A last obstacle is education. Contextual network predictors are unlike other types of data being captured and analyzed. Enterprises must first capture this new data, and that will require some mind-set shift. Setting up an infrastructure to model the contextual network will require new tools and approaches. Finally, using these contextual networks in predictive analytics that improve the bottom line will require the cooperation of multiple disciplines—analytics, marketing and operations.

Despite obstacles, the time to investigate and leverage these graph-based methods is now. IDAnalytics, a company that provides identity theft protection, has been issued a patent for a system and method for identity-based fraud detection through graph anomaly detection. How will you start adding graph-derived predictors into your predictive analytics?

Dr. John F. Elder heads a data mining consulting team with offices in Charlottesville, Virginia and Washington DC. Founded in 1995, Elder Research, Inc. focuses on scientific and commercial applications of pattern discovery and optimization, including stock selection, image recognition, text mining, biometrics, drug efficacy, credit scoring, cross-selling, investment timing, and fraud detection.

Dr. Elder has authored innovative data mining tools, is active on Statistics, Engineering, and Finance conferences and boards, is a frequent keynote conference speaker, and is General Chair of the 2009 Knowledge Discovery and Data Mining conference in Paris. John’s courses on data analysis techniques – taught at dozens of universities, companies, and government labs – are noted for their clarity and effectiveness. Dr. Elder was honored to serve for 5 years on a panel appointed by the President to guide technology for National Security.

Check out the three minute interview on the Predictive Analyticswebsite.

Transcript:

Bill Cullifer: I am on the phone with Dr. John Elder, CEO and Founder of Elder Research out of Virginia. Good afternoon. Thanks for agreeing to this interview.

John Elder: Thanks for having me.

Bill Cullifer: You bet. You recently provided a presentation at the conference entitled the High ROI of Data Mining for Innovative Organizations. Can you summarize that session for the subscribers of this podcast?

John Elder: Sure Bill. I just gave a talk about nine of the kind of more interesting success stories we have had at Elder Research over the last decade or so and we will get them at a higher level. So, it handled the way data mining really helps and I have broken it into three parts, approving and emphasizing good, detecting and eliminating bad, streamlining and automating are… I gave examples of how each of those had worked [Voice Cross Over]

Bill Cullifer: Can you cite an example?

John Elder: Sure. One kind of interesting one was Anheuser-Busch[Phonetic] pictures of beer on the shelf, they like to correlate how the positioning of their product related to [Inaudible], they also want to know if they are missing any stock [Inaudible] so, that’s a relatively laborious process, to take a picture and turn it into some [Inaudible] interpretation of what exact product and what exact orientation is there. So, of course… so we did a pilot process, automated identification of what[Phonetic] product was there. Recognition is a very hard [Inaudible] is one that’s well lived, well under[Phonetic] good condition. Products are trying to make themselves be recognized a whole lot easier than say identifying somebody who just canceled on a [Inaudible] might be for instance but we were able to get 90% accuracy which was our goal instantaneously essentially meaning that they would spend one tenth as much time editing it as they would doing it by previous metric.

Bill Cullifer: Time is money. Right?

John Elder: Exactly. So, that would save them millions of dollars there. So, that was an example of automating and speeding up. There were a couple of examples of return by eliminating, but that there… the examples [Inaudible]. We had projects we did for the IRS and projects that we did for a large [Inaudible] electronics firm that saved tens of millions of dollars, actually the numbers were astoundingly large compared to the [Inaudible].

Bill Cullifer: So, if I… and the subsribers of this podcast were considering entering into the predictive analytics realm, any high level recommendations on first step?

John Elder: Yeah, the… one of the extreme[Phonetic] things that came out as I was putting this talk together were you know what were some characteristics that were common to some of these successful projects and all of them were technically successful, but not all of them were completed and that was interesting to see what were the differences there. For instance, on the Anheuser-Busch[Phonetic] one, they never put it in production because the day that we were to sign the development contract was 09/11; that was definitely a whole lot of a hell of a tragic event and there were some other times when a project didn’t see the way to appreciate it[Phonetic], but most of the nine examples I gave complete business successes saw their way into implementation and one of the key characteristics that we needed was a champion who would take the business risk inside the organization to see it all the way through and what you really want[Phonetic] there is nothing harder than starting something brand new because all those who talk about the old way of doing it will hold it and those who talk about the new way only lukewarmly support it and can still do [Inaudible]. So, we found that the innovative organization part of the title was really [Inaudible] organization that allowed it’s… that rewarded success for its people, not just punished because that is the environment in which a project will thrive and the return can be [Voice Cross Over]

Bill Cullifer: Sounds like excellent advice and we certainly appreciate your time.

John Elder: Thanks for having me.

Today’s topic is Predictive Analytics and this is a continuation in a series of audio podcast interviews on this topic. Dr. Eric Siegel is a former computer science professor at Columbia University and the President of Prediction Impact.

Q: Dr. Siegel Could you provide an example case of deploying predictive analytics?

A: The two most established places predictive analytics is applied are 1) increasing the response rate in direct marketing and 2) improving customer retention by predicting churn. Let’s discuss a case of the first of these two applications, improving direct marketing response — which is also known as response modeling, since the thing predicted for each customer by the model is whether or not they will respond to the offer.

Predictive analytics supplies the predictive score for each customer, and it’s really simple how these scores can be used used to improve response rate and campaign profit.  First, you sort your list by the scores, with customers that received a higher predictive score up top. This means the first customers in your list are now the most likely to respond.  Then you target your campaign only towards customers higher up on the list.

So, the next question is, where do you draw the line going down the list – how many do you contact?  Well, suppose you are contacting a million prospects, and it costs $2 per contact (this example could be for either a mailing or a phone campaign). The average profit per response, let’s say, is $20, and a little over 7% of those contacted respond.  Under these conditions, a campaign will not be profitable — with a little arithmetic, you can see that it actually loses $566,000, because the response rate’s just not high enough relative to the cost that you’re spending and how much you make on those that do respond.

But with a predictive model, this can be turned around and you can take a loss and make it a profit.  If the cost is expended only to contact the top, let’s say, 25% of the customers, which contains the lion’s share of responses anyway, the campaign will turn out to be profitable, and could land a profit approaching half a million dollars. So knowing which customers to contact can make all the difference in the world.

Let me make one last note about this.  The predictive model that you use to do this can continuously be improved, as long as the responses to each campaign, even a campaign that’s targeted with the model in the first place, are tracked.  Each customer’s response or non-response, if you’ve tracked it, then becomes a learning case with which better predictive models can be developed.