In this third and final interview of Kim Larsen, Director of Advanced Analytics at Charles Schwab & Co. Kim shares his thoughts regarding strategies tp Maximize Marketing Impact with Net Lift Models.

Net Lift Models are designed to maximize the incremental impact by targeting the undecided clients that can be motivated by marketing.

Check out the three minute interview on the Predictive Analyticswebsite.

Transcript:

Kim Larson: That approach is my own sort of favorite because it’s very easy to do and it’s very intuitive. It’s using a k-NN, it’s like a nearest neighborhood classifier. So, it’s very simple essentially. First you identify say 10 or 20 variables that you think are great net lift variables meaning that they can differentiate conversion rates between test and control group and you can think of many ways to do that, but that’s actually quite easy for a single variable to see is this variable really a variable that can differentiate the self selectors from the… just the three other converters. Then, once you have… let’s say you got about 20 variables or so; based on those 20 variables, you find… you would go to a client, you will find say the 100 clients that look most like that client. So, you will basically find the 100 closest neighbors to that client like the “friends” of that client and then we are in that neighborhood of a 100 people, you basically just do a voting so to speak. You calculate the net lift weight for those 100 people and that net lift weight is the estimated net conversion rate for that client and then so forth. You can use SAS to do this or any other software that provides a k-NN classifier. In SAS, you can use [Inaudible] and I am sure about [Inaudible] and all those other packages have similar… [Voice Cross Over] it’s very easy to do, that’s the last one.

Bill Cullifer: Great.

Kim Larsen: Tends to give good results as well.

Bill Cullifer: Question for you. Is this a function of a business unit or information technology or of combination of both?

Kim Larsen: The LIFT model?

Bill Cullifer: Yeah, I mean, LIFT Model.

Kim Larsen: Yeah. I think typically you would… these models would be undertaken by some analytical team within your company like the team I worked in. I think typically most companies will have a modeling team and they would be the ones doing that. They would need to work with the marketing people to implement this into a campaign and the biggest challenge there is when you have got… when you have got a net model and you are telling people, “Well, these are the clients that I intend to target.” People will notice that these are not the typical people that you target. What I always tell people is that if you forget about all the fancy modeling and all the fancy stuff, there is a general rule of thumb that I have found. Your most engaged, wealthy, happy clients that love you the most, they are not the ones you need to hit with direct marketing. They are going to buy your stuff regardless, because they already like you, they already think you provide great service. Where you need to put your money is on the fence sitters, the people that are right below your best clients. Those are the people that are sleeping a little bit. They have all the propensity and the wealth and all that’s needed to really… to drive your bottom line, but they are not doing it fully because something… maybe they are not engaged with you. They don’t know what you offer, etc. So, that’s a general rule of thumb. Now, you as the analytical manager or the analytical analyst, you need to convince your marketing colleagues that that’s the way to go and that sometimes… I mean at Schwab it’s never really been an issue for me, but I know some other places that that can be an issue.

Bill Cullifer: Yeah, fair enough, I bet it can be.

Kim Larsen: Because you think…

Bill Cullifer: Then it’s an huge issue, right?

Kim Larsen: Yeah.

Bill Cullifer: And part of it is an educational process.

Kim Larsen: Exactly.

Bill Cullifer: Yeah, fair enough. Well, that’s an excellent perspective and I appreciate you sharing that as well and for your time today.

Kim Larsen: You are welcome.

In this second in a series of interviews (2-3) with Kim Larsen, Director of Advanced Analytics at Charles Schwab & Co Kim shares his thoughts regarding strategies tp Maximize Marketing Impact with Net Lift Models. Net Lift Models are designed to maximize the incremental impact by targeting the undecided clients that can be motivated by marketing.

Check out the three minute interview on the Predictive Analyticswebsite.

A full transcript to follow in forty-eight hours.

Kim Larsen is the Director of Advanced Analytics at Charles Schwab & Co. with headquarters in San Francisco, CA.

In this first of a series of interviews, (1-3) Kim shares his thoughts regarding Maximizing Marketing Impact with Net Lift Models. Net Lift Models are designed to maximize the incremental impact by targeting the undecided clients that can be motivated by marketing.

Check out the three minute interview on the Predictive Analyticswebsite.

Transcript:

Bill Cullifer: I am on the phone with Kim Larsen, Director of Advanced Analytics, Charles Schwab & Co. with headquarters in San Francisco. Good morning Kim and thanks for agreeing to the interview.

Kim Larsen: Thank you.

Bill Cullifer: Kim, you recently presented a case study regarding maximizing marketing impact with net lift model at Charles Schwab & Co.

Kim Larsen: Yeah.

Bill Cullifer: Could you summarize that session for the subscribers of this podcast with a walk away?

Kim Larsen: Yeah. Sure.

Bill Cullifer: Thank you

Kim Larsen: We call it the net lift model and net lift model is really motivated around when whether it is [Inaudible] or a resale or when you launch marketing, you measure the impact of the campaign by what we call increments [Inaudible] a test group of people that get a offer that’s related to your product. Then you will have core[Inaudible] groups for that book[Phonetic] like those people that did not [Inaudible]. The ultimate test comes of how well your program worked, is what’s the difference in performance in the[Phonetic] test group, i.e., for that [Inaudible] offer versus those that did not offer. Problem is that many times marketing says targeting strategies are not time[Phonetic] optimized increment impact. So, people might not [Inaudible] and make it a great response rate, but your core group[Phonetic] does this as well that simply [Inaudible] bottom line. That’s the motivation. The purpose of this talk was basically to talk about [Inaudible]. We basically think of your client base as consisting of three base segments.

You have a segment of clients that are not interested in whatever you are trying to sell. So, those, just leave alone and in fact if you market to them, it might have an adverse effect. Then, you have some clients that we call self selectors. These are clients that will mostly likely purchase your products on their own and marketing does not give motivation and actually leave them alone so to speak and they will buy the products regardless. The third segment is called the swing [Inaudible] and you know this sort of… this name is sort of derived from a parallel that I have been drawing with presidential candidates. Think about it. When you look at for example the latest [Inaudible] Obama, they spent most of their money on TV time in the swing[Phonetic] State of Ohio for that [Inaudible]. If you live in California like me or in Bay Area of California like I hadn’t seen a single McCain ad or Obama ad. However, when I went to Las Vegas for a conference, turned on CNN and I saw four ads for an hour. So, the presidential candidates have already figured out that spend your marketing money on the swing voter, the undecided and leave the decided alone and optimize market spend. So, that’s… we had the technical swing [Inaudible] they certainly have a tendency to buy the products by themselves, however, they need motivation. So, the purpose of net lift model is to find swing [Inaudible]. Two options when people do targeting, they end up finding the self selector. So, they build typical targeting models where really identifying self selectors, those models are those that… those models are typically done by looking at the clients that you offer the products to and then became converted, but what net models do… net lift models do that they don’t just look at that population, i.e., the population you mail to or contacted.

They also look at the core[Phonetic] group. Those are the people that you did not contact. Many of them also converted. Understand what’s different in the profile of self selectors versus the average converter and if you have those dimensions, you can focus in and find the swing [Inaudible] that will… it’s not always easy to do, but when you do it, it can be your impact of your marketing direct[Phonetic]. I have seen… we can talk typically about Charles Schwab itself, but I have seen campaigns where a typical targeting model to get to a incremental impact of about say five basis point that could essentially be that the people who you contacted converted at say 1%, so 1% of them bought products, but the people who you didn’t contact, they converted at say 98[Phonetic] basis points, so the difference was just two basis points which would not cover your cost. I think cases like this where a net modeling gains your net… your net impact into something very possible.

Dr. John F. Elder heads a data mining consulting team with offices in Charlottesville, Virginia and Washington DC. Founded in 1995, Elder Research, Inc. focuses on scientific and commercial applications of pattern discovery and optimization, including stock selection, image recognition, text mining, biometrics, drug efficacy, credit scoring, cross-selling, investment timing, and fraud detection.

Dr. Elder has authored innovative data mining tools, is active on Statistics, Engineering, and Finance conferences and boards, is a frequent keynote conference speaker, and is General Chair of the 2009 Knowledge Discovery and Data Mining conference in Paris. John’s courses on data analysis techniques – taught at dozens of universities, companies, and government labs – are noted for their clarity and effectiveness. Dr. Elder was honored to serve for 5 years on a panel appointed by the President to guide technology for National Security.

Check out the three minute interview on the Predictive Analyticswebsite.

Transcript:

Bill Cullifer: I am on the phone with Dr. John Elder, CEO and Founder of Elder Research out of Virginia. Good afternoon. Thanks for agreeing to this interview.

John Elder: Thanks for having me.

Bill Cullifer: You bet. You recently provided a presentation at the conference entitled the High ROI of Data Mining for Innovative Organizations. Can you summarize that session for the subscribers of this podcast?

John Elder: Sure Bill. I just gave a talk about nine of the kind of more interesting success stories we have had at Elder Research over the last decade or so and we will get them at a higher level. So, it handled the way data mining really helps and I have broken it into three parts, approving and emphasizing good, detecting and eliminating bad, streamlining and automating are… I gave examples of how each of those had worked [Voice Cross Over]

Bill Cullifer: Can you cite an example?

John Elder: Sure. One kind of interesting one was Anheuser-Busch[Phonetic] pictures of beer on the shelf, they like to correlate how the positioning of their product related to [Inaudible], they also want to know if they are missing any stock [Inaudible] so, that’s a relatively laborious process, to take a picture and turn it into some [Inaudible] interpretation of what exact product and what exact orientation is there. So, of course… so we did a pilot process, automated identification of what[Phonetic] product was there. Recognition is a very hard [Inaudible] is one that’s well lived, well under[Phonetic] good condition. Products are trying to make themselves be recognized a whole lot easier than say identifying somebody who just canceled on a [Inaudible] might be for instance but we were able to get 90% accuracy which was our goal instantaneously essentially meaning that they would spend one tenth as much time editing it as they would doing it by previous metric.

Bill Cullifer: Time is money. Right?

John Elder: Exactly. So, that would save them millions of dollars there. So, that was an example of automating and speeding up. There were a couple of examples of return by eliminating, but that there… the examples [Inaudible]. We had projects we did for the IRS and projects that we did for a large [Inaudible] electronics firm that saved tens of millions of dollars, actually the numbers were astoundingly large compared to the [Inaudible].

Bill Cullifer: So, if I… and the subsribers of this podcast were considering entering into the predictive analytics realm, any high level recommendations on first step?

John Elder: Yeah, the… one of the extreme[Phonetic] things that came out as I was putting this talk together were you know what were some characteristics that were common to some of these successful projects and all of them were technically successful, but not all of them were completed and that was interesting to see what were the differences there. For instance, on the Anheuser-Busch[Phonetic] one, they never put it in production because the day that we were to sign the development contract was 09/11; that was definitely a whole lot of a hell of a tragic event and there were some other times when a project didn’t see the way to appreciate it[Phonetic], but most of the nine examples I gave complete business successes saw their way into implementation and one of the key characteristics that we needed was a champion who would take the business risk inside the organization to see it all the way through and what you really want[Phonetic] there is nothing harder than starting something brand new because all those who talk about the old way of doing it will hold it and those who talk about the new way only lukewarmly support it and can still do [Inaudible]. So, we found that the innovative organization part of the title was really [Inaudible] organization that allowed it’s… that rewarded success for its people, not just punished because that is the environment in which a project will thrive and the return can be [Voice Cross Over]

Bill Cullifer: Sounds like excellent advice and we certainly appreciate your time.

John Elder: Thanks for having me.

Andreas S. Weigend, Ph.D. was Chief Scientist at Amazon.com, where he specialized in understanding how people behave online, helping Amazon build its customer-centric, measurement-focused culture.

Has built his expertise is built on my academic background in predictive modeling, machine learning, cognitive science, and behavioral economics. 15 years of consulting have enabled me to apply this knowledge successfully in the real world. He now works as an independent consultant with firms including World Economic Forum, Alibaba, Lufthansa, MySpace, and Nokia, helping some of the most brilliant executives around the world leverage user data to produce innovative products and business models. He also teaches at Stanford and UC Berkeley, and is often invited to speak at international events.

Check out the three minute podcast on the PredictiveAnalytics.org website.

Transcript:

Bill Cullifer, PredictiveAnalytics.org: Good afternoon. Dr. Weigend Bill Cullifer, calling with predictiveanalytics.org

Andreas S. Weigend, Ph.D: Hello.

Bill Cullifer, PredictiveAnalytics.org: Thanking you for agreeing to the call.

Andreas S. Weigend, Ph.D: Sure.

Bill Cullifer, PredictiveAnalytics.org: On the topic of predictive analytics, for the subscribers of this podcast us with a couple of specific walk aways.

Andreas S. Weigend, Ph.D: All right. So, let’s unpack the term predictive analytics. One element is analytic, the other element is predict. What predictive analytics for me is the focus on prediction; in the ’90s, a lot of emphasis has been on given a set of data, what insight can we get. Often those insights were not actionable. Then in the late ’90s, people focused on making predictions, but still predictions off dead data for dead data and I spend lots of tutorial time for top Wall Street quantitative traders explaining to them how to do backtesting and cross validation and all of those things and then in the last two years as the corporate communication has dropped dramatically and we actually are now able to do experiments very cheaply, we move from what I call dead data to live data. So, the problem has been no longer to be given a set of data what insights can we get, but given the problem what data can we get. So, that is the shift which I elaborate on and if you think about companies here, of course is a beautiful example, where people just give lots of data about themselves and if you reveal your intention then it is often much easier to make predictions than if we just have your data from the past. If you say, you want to go on a date on Valentine’s day, then that is information, that is we try to figure out from your clicking behavior on Facebook what you might want to do on Valentine’s day.

Bill Cullifer, PredictiveAnalytics.org:: Very well said. I am curious to know are there barriers to entry at this point or is it just that simply an educational process for those decision makers to understand how predictive analytics.

Andreas S. Weigend, Ph.D: Well, it truly is a paradigm shift and I typically call it the consumer or the customer data revolution and it is no longer about which analysis package you buy or which algorithm to use or what the mindset that CRM, customer relationship management, which we are at the end of predictive analytics seeing one who they should be sending their mailing to, what’s is called CMR, not CRM, but CMR, customer managed relationship, where the customer is now at the center and manages the relationship with potential vendors. Sometimes, I also call it that we move from e-business where the firm in the center to me-business where again the customer is in the center. So, it is not really about buying software packages, but it is about companies realizing that the customer is willing to give them information if he gets value out of it and not just to help the company, but ultimately it will help themselves.

Bill Cullifer, PredictiveAnalytics.org: Excellent. Well said. Terrific summary. I certainly appreciate the perspective and your time today.

Andreas S. Weigend, Ph.D: Absolutely.