Doctor of philosophy

Improvement of

Prediction of Customer Response

in Direct Marketing

THESIS
Submitted in partial fulfillment

of the requirements for the degree of

DOCTOR OF PHILOSOPHY

by

A. SUNDARAMURTHY

Under the Supervision of

Prof. Dr. M.J.XAVIER



BIRLA INSTITUTE OF TECHNOLOGY AND SCIENCE

PILANI (RAJASTHAN) INDIA
2009

ACKNOWLEDGEMENT

I express my gratitude to Prof.L.K.Maheshwari, Vice Chancellor, BITS, Pilani for giving me this opportunity to pursue PhD and Prof.Ravi Prakash , Dean, Research & Consultancy Division, BITS, Pilani for his continuous encouragement and support in carrying out my PhD work smoothly. I thank Prof.S.P.Regalla, Assistant Dean and Dr.Hemant Jadav, Ms.Monica Sharma, Mr.Dinesh Kumar, Mr.Gunjan Soni, Mr.Sharad Srivastava and Mr.Amit Singh, nucleus members of Research &Consultancy Division, BITS, Pilani for their constant help and advice at all stages of my Ph/d work. I also thank the other office staff of Research & Consultancy Division, BITS, Pilani who rendered a lot of help in organizing various forms of paper work related to PhD progress.

I express my deep felt sense of gratitude and sincere thanks to my PhD supervisor Dr.M.J.Xavier, without whose constant guidance, help and tutelage this PhD work would not have been completed .He has been a constant source of inspiration and encouragement throughout my PhD work.

I thank Dr.Anil K Bhat and Prof.S.B.Mishra, my Doctoral advisory committee (DAC) members at BITS, Pilani for their constructive criticism and useful suggestions that helped me in immensely improving the contents and quality of presentation of my PhD thesis.

My heartfelt thanks for all the support and guidance to Dr.Anil K Bhatt, Group Leader of Management Department who has supported, inspired, encouraged and guided me during the various phases of my research work.

My sincere thanks to Dr.G.Balasubramaniam, Dean, IFMR who taught me data analysis, using artificial neural network and has been a constant source of encouragement.

I dedicate this thesis to my Beloved Mother who was a Noble Soul of Selflessness, who passed away recently.

A.Sundaramurthy

SYNOPSIS

Improving prediction of Customer Response in Direct Marketing

Objective

The objective of this Research is to improve the accuracy of prediction of customer response in direct marketing.

Scope

The Research focuses on methods used for prediction and their relative accuracy.

Milestones

1. Identification of methods of research

2. Data collection

3. Data Analysis and model development

4. Validation of model

Methodology

The sector chosen is financial services marketing and the model developed is based on qualitative and quantitative analysis of data from the data bases of customer. Data is used to score customers where improvement in prediction is shown and which is useful in direct marketing campaigns for cross selling and up selling.

The data is divided into three parts one for model building one for testing and the third for validation.

Different methods like Linear Regression, Logistic Regression, Discriminant analysis and neural network are used for model development and their prediction accuracy is compared.

First the transformation of business from the orbit of production centric to that of ever increasing consumer centric orbit is explained. Thus Customer insight as the heart of the business in the 21^st century is brought out. The complexity of the consumer behavior , various attempts to model the consumer behavior by various experts in the field of marketing, economics and psychology and attempts to quantify the behavior in the science of psychology ,sociology and anthropology follow next.

The recent advances in studying human behviour in the fields of neuro psychology is explained.

Next the fundamentals of models and model building, different methods of model development, application of various mathematical methods, validation and application of models in various fields is brought out.

Response models and their crucial role in marketing in sustaining the competitive edge by improving the ROI of marketing is brought out. Specifically the effectiveness and efficiency of marketing campaigns, which are playing increasingly a dominant role in acquiring and retaining customers by direct marketing and maintaining customer relations to harvest their life time value is brought out.

Birth and evolution of financial services as a business and its birth and growth in India is discussed. How the rapid growth of the sector has given rise to the snowballing debt and non performing assets due to the bad repayment behavior of the borrowers and how this has led to the urgent and important need to assess the customer as good or bad and the development of the field of risk management is discussed.

Different methods of risk modeling and their merits and demerits are discussed.

Neural network has been used and compared with other methods for model development and neural network has been found to be most accurate. Hence NN its theory and application is discussed in detail. Once the classification of good and bad customers is done, the data of good customers are useful to profile customers based on various factors like their financial needs, repayment behavior, their risk ranking, their risk taking ability, their thirst for risk, their financial and buying behavior etc and other data to develop a response model.

The possible potential areas of future research are given briefly in the conclusion.

The improvement in prediction accuracy achieved in this research enables accurate scoring and profiling of customers in financial service
and its importance is globally important as has been proved by the global crisis triggered by Subprime lending.

LIST OF TABLES

Tables No.	Title	Page No.
	Consumer choice model	76
	ANN’s relative contribution factor	78
	Out of sample forecast	80
	C5 Algorithm training data	83
	Comparative model performance	123
	NN model -parameters for comparison	165
	Logistic Regression- parameters for comparison	165
	Markov Model -parameters for comparison	166
	Multi Variate proportional hazard estimation -parameters for comparison	166
	Logistic Binary regression model -parameters for comparison	167
	Classification & Regression Tree-parameters for comparison	167
	Evaluation of classification models	168
	Summary of variables in creditworthiness models across the globe	186
	Customer Profile-Age	205
	Customer Profile-Educational qualification	207
	Customer Profile-No. of dependents	209
	Customer Profile-Income	211
	Customer Profile-Other Income	213
	Customer Profile-Experience	215
	Customer Profile-Type of House	217
	Customer Profile-Rent	219
	Customer Profile-Down payment	221
	Customer Profile-TV ownership	223
	Customer Profile-Music system ownership	225
	Customer Profile –Fridge	227
	Customer Profile-Washing machine	229
	Customer Profile-Two wheeler	231
	Customer Profile-Four wheeler	233
	Customer Profile-Amount Overdue	235
	Customer Profile-no of Dues	237
	Customer Profile-Classification	239
	Age-Customer classification	241
	Qualification-Customer classification	243
	No. of dependents-Customer classification	245
	Income-Customer classification	247
	Other Income-Customer classification	249
	Experience-Customer classification	251
	Type of house-Customer classification	253
	Rent-Customer classification	255
	Down payment-Customer classification	257
	TV ownership-Customer classification	260
	Music system ownership-Customer Classification	263
	Fridge ownership-Customer classification	266-267
	Washing machine-Customer classification	269
	Two wheeler ownership-Customer classification	272
	Four wheeler ownership-customer classification	275
	Variables extracted in data analysis	296
	Confusion matrix to measure model fit	325
	Multiple Regression model Summary	328
	Multiple Regression -ANOVA	329
	Multiple Regression –Significant Variables	329
	Multiple Regression-Coefficients	330
	Logistic Regression- Significant Variables	330
	Logistic Regression-Non significant Variables	331
	Discriminant analysis-classification results	331
	Factor Analysis-Rotated Component Matrix	332
	Neural Network –performance Table	332
	Classification Table for NN	334
	Summary Of Models	334
	Optimum MLP configuration	335

LIST OF FIGURES

Figure No.	Title	Page No.
	Howard Model of Buyer behavior 1974 version	26
	Howard Model 1977 version	27
	Engel Black Well Kollat Model	31
	Behavioral model –A model of decision making	56
	Customer response model	82
	Case based reasoning model 1	84
	Case based reasoning model2	85
	Sales Response Models	88
	Rural Economy Statistics	139
	Classification Tree	168
	Process of credit appraisal and loan disbursal	190
	An Integrated model for credit scoring	203
	Customer Profile-Age	206
	Customer Profile-Educational qualification	208
	Customer Profile-No. of dependents	210
	Customer Profile-Income	212
	Customer Profile-Other Income	214
	Customer Profile-Experience	216
	Customer Profile-Type of House	218
	Customer Profile-Rent	220
	Customer Profile-Down payment	222
	Customer Profile-TV ownership	224
	Customer Profile-Music system ownership	226
	Customer Profile –Fridge	228
	Customer Profile-Washing machine	230
	Customer Profile-Two wheeler	232
	Customer Profile-Four wheeler	234
	Customer Profile-Amount Overdue	236
	Customer Profile-no of Dues	238
	Customer Profile-Classification	240
	Age-Customer classification	242
	Qualification-Customer classification	244
	No. of dependents-Customer classification	246
	Income-Customer classification	248
	Other Income-Customer classification	250
	Experience-Customer classification	252
	Type of house-Customer classification	254
	Rent-Customer classification	256
	Down payment-Customer classification	259
	TV ownership-Customer classification	262
	Music system ownership-Customer Classification	265
	Fridge ownership-Customer classification	268
	Washing machine-Customer classification	271
	Two wheeler ownership-Customer classification	274
	Four wheeler ownership-customer classification	277
	MLP NN architecture	288
	Over learning in NN	309
	Statistical Power	316
	Probability of error Vs No of comparisons	321
	Expected Error Vs Comparisons	321
	Comparison of ROC curves	325
	NN topology adopted	333
	ROC of the NN Model	333

LIST OF ABBREVIATIONS

ANN - Artificial Neutral Network
AQRE - Agent Based Quantal Response Equilibrium
BF - Basis Function
BP - Back Propogation
CART - Classification and Regression Trees
CEM - Conditional Expectation Maximization
DM - Direct Marketing
FF - Feed Forward
FFT - Fast Fourier Transform
FMCG - Fast Moving Consumer Goods
GA - Genetic Algorithm
GDP - Gross Domestic Product
KNN - K Nearest Neighbour
MARS - Multiple Adaptive Regressing Splines
MCR - Magnetic Character Recognition
MFI - Micro Financial Institution
MFT - Mean Field Theory
MLP - Multi Layer Perceptron
NASA - National Aeronautics and space Administrators.
NFC - Need for Closure
NN - Neural Network
NPA - Non Performing Assets
OCH - Optimum Collective Harvest
OCR - Optical Character Recognition
OLS - Ordinary Least Squares
RBF - Radial Basis Function
ROC - Receiver Operating Curve
ROI - Return on Investment
SE - Sequential Equilibrium
SES - Socio Economic Status
SHG - Self Help Groups
T.E.D.I.C. - Technological, Economic, Demographic Institutional & Cultural Developments

ABSTRACT

Business is Consumer and all successful businesses have always understood their customers. This research is in consumer behaviour and the objective is to get an insight of customer. This work is basically to predict the customers who would be good customers among the hundreds of thousands of borrowers and then who could be targeted for cross selling and up selling to improve the response to campaigns. This work has compared various methods like regression, discriminant analysis, logistic regressions and neural network and compared their prediction accuracy and established that a neural network gives the maximum prediction accuracy. Input to neutral network is factor scores from factor analysis. Also both qualitative and quantitative analysis is done based on customer data. Another important feature of this study is that extensive discussion, interviews and focus groups study have been carried out with both good bad customers and an attempt has been made incorporate to the behavioural aspects of the customer in the model.