Application of Logistic Regression using R Programming by Statswork

OCT 10, 2019

Research paper

0 1 LOGISTIC REGRESSION ANALYSIS

0 2 0 3

Logistic regression is a type of predictive model when the target variable is a categorical variable with two categories.

Logistic regression is used for the prediction of the probability of occurrence of an event by fitting the data into a logistic curve.

It makes use of predictor variables either numerical or categorical.

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

USES OF LOGISTIC REGRESSION ANALYSIS Predicts the presence or absence of a characteristic or outcome based on values 01 of a set of predictor variables.

02 Estimates odds ratios (OD) for each of the independent variables in the model.

Suitable for models where dependent variable is dichotomous.

Predicts the probability of occurrence of an event by fitting data to a logit function.

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

EXAMPLE OF LOGISTIC REGRESSION ANALYSIS Question 1 Prediction among patient having high blood pressure (BP) or not along with other observations such as Age, Smoking habits, Weight, or Body mass Index BMI, blood cholesterol levels, Diastolic and Systolic BP value, Gender, etc. Table 1. Blood Pressure Range

B P

Low

Norma l

Borderlin e

High

Systoli c

<90

90130

131140

14 0

Diastoli c

<60

60180

8190

>90

•Source: Adapted from Nimmala et al., 2018 Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

PROCEDURES INVOLVED IN CONDUCTING LOGISTIC REGRESSION ANALYSIS

01 02 03 04 05

Select the response or dependent variable (patients having high BP or not). Other variables are considered as explanatory or independent variables. Dependent variable needs to be coded as 0 and 1. Explanatory variable can be a continuous variable or ordinal variable. Outcome is predicted by applying logistic regression model.

RESULT Outcome is predicted by applying logistic regression model. Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

Question 2Prediction among child having high blood patient having high blood pressure (BP) or not along with other observations such as Age, Smoking habits, Weight, or Body mass Index BMI, blood cholesterol levels, Diastolic and Systolic BP value, Gender, etc.

Table 1. Prediction model for childhood high blood pressure

Adapted from Hamoen et al., 2018

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

LOGISTIC REGRESSION IMPLEMENTATION USING LOGIT FUNCTION AND R PROGRAMMING Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

LOGISTIC REGRESSION IMPLEMENTATION USING LOGIT FUNCTION Logistic regression generates the coefficients of a formula to predict a logit transformation of the probability of the presence of the characteristic of interest: Logit (p) = b0 + b1X1 + b2X2 +…+ bkXk where p- probability of the presence of the characteristic of interest. The logit transformation is defined as the logged odds:

Odds = p = probability of prthe esence of the characteristic 1-p probability of absence of the characteristic & Logit (p) = log (p/(1-p)) = log (p) – log (1-p) Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

LOGISTIC REGRESSION IMPLEMENTATION USING R PROGRAMMING

Fitting of binary logistic model for the Titanic dataset that is available in Kaggle. Procedure • Frame the objective of the study as survival variable: 1 - survived, 0 - not survived and other - independent variables. • Loading the training data into the console using the function read.csv (). train.data<- read.csv('train.csv',header=T,na.strings=c("")) • Check missing data before fitting using sapply() function in R. sapply(train.data,function(x) sum(is.na(x))) •

Check other missing entries in the subset data. There are different ways to replace the NA’s with either the mean or median of the data data$Age[is.na(data$Age)] <- mean(data$Age,na.rm=T)

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

LOGISTIC REGRESSION IN R PROGRAMMING

R is an easier platform to fit a logistic regression model using the function glm(). Question

PassengerId

Survived

Pclass

Name

Sex

Age

SibSp

Parch

177

Cabin

Embarked

687

Ticket 0

Fare 0

“Cabin” and “PassengerId” variables are missing which are skipped and made as subset of the data as new with the subset () function. data <- subset(train.data,select=c(2,3,5,6,7,8,10,12)) where the numeric values is the columns in the data file.

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

FITTING LOGISTIC REGRESSION MODEL

•

Before fitting, Split the data into two sets: training set ( to fit the model) testing set (for testing). traindata<- data[1:800,] testdata<- data[801:889,]

•

Specify family = binomial in glm() function since our response is binary. lrmodel<- glm(Survived ~.,family=binomial(link='logit'),data=traindata)

•

The result of the model can be obtained using the following command: summary(lrmodel)

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

RESULT INTERPRETATION: From the p-value, • “Embarked”, “Fare”, and “SibSp” variables- not significant. • “sex “variable has lesser p-value so there is a strong association of passengers with the chance of survival. It is important to note that the response variable is log odds ln(odds) = ln(p/(1-p)) = a*x1 + b*x2 + … + z*xn. To analyse the deviance, use anova() function in R. anova(lrmodel, test="Chisq") Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

•

Wider difference between the null deviance and the residual deviance indicates better model performance.

•

Large p-value indicates that the logistic regression model without that particular variable explains the same amount of variation.

Hence found that Akaike Information Criterion (AIC) is the best model.

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

PREDICT THE MODEL ON A NEW SET OF DATA R enables us to do with predict() function. Let threshold for the new response data be P(y=1|X) > 0.5 then y = 1 otherwise y=0. The threshold changes according to the researchers needs. The following Table presents the accuracy prediction formulas especially when machine learning algorithms are applied.

Table 4. Measures and formula fit.results<predict(lrmodel,newdata=subset(test,select=c( 2,3,4,5,6,7,8)),type='response') fit.results<- ifelse(fit.results> 0.5,1,0) Error<- mean(fit.results != test$Survived) print(paste('Accuracy',1-Error))

Contd..

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

"Accuracy 0.842696629213483.“ •

Accuracy= 84.26% (here data is split manually).

Finally, we find the area under the curve by plotting Receiver Operating Characteristic (ROC) curve using the following commands: library(ROCR) A<-predict(lrmodel,newdata=subset(test,select=c(2,3,4,5,6,7,8)), type="response") Pre<- prediction (A, test$Survived) Pre1<- performance(Pre, measure = "tpr", x.measure = "fpr") plot(Pre1) auc<- performance(Pre, measure = "auc") auc<- auc@y.values[[1]] auc

0.8647186 Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

RESULTS Since Area Under the Curve (AUC) is closer to 1, then this is an good predicting model.

ROC curve generates the true positive rate against false positive rate, similar to the sensitivity and specificity.

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

Statswork Lab @ Statswork.com www.statswork.com

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics

PHONE NUMBER

: +44-1143520021

INDIA

: +91-4448137070

EMAIL ADDRESS info@statswork.com

GET IN TOUCH WITH US

Research Planing | Data Collection | Semantic Annotation | Consumer & Retail Analytics | Econometrics