IBM HR Analytics Employee Attrition & Performance
===
###### tags: `Data Science` `Human Resource` `People Analytics`
Digi+ Talent III Internship Project
[Kenny Hsieh](https://www.facebook.com/kennyhiseh1111), 2017/9/29

## Description
[Kaggle Feature Dataset](https://www.kaggle.com/pavansubhasht/ibm-hr-analytics-attrition-dataset)

This is a fictional data set created by IBM data scientists. Aim to uncover the factors that lead to **employee attrition** and construct the binary classcification prediction model.

## Dataset Overview
1470 observations

**Dependent Variable** : Attrition

**Independent Variable** (31 different variables in total) :
- Basic Info:
Age, Gender, Education, Education Field, Martial Status, Distance From Home

- Work Info:
Department, Job Role, Job Level, Over Time, Business Travel, Performance Rating, Stock Option Level, Job Involvement

- Statisfaction:
Work-Life Balance, Job Satisfaction, Relationship Satisfaction, Enviroment Satisfaction

- Salary Related:
Monthly Income, Monthly Rate, Daily Rate, Hourly Rate, Percent Salary Hike

- Time Related:
Total Working Years,Traning Time Last Year,Years At Company,Years In Current Role,Years Since Last Promotion,Years With Current Manager,Num Companies Worked

## Exploratary Data Analysis, EDA
### The Distribution of Attrition
There are 1470 observations, consists of 237 obs attrition(leave), and the rest of 1233 obs are all non-attrition(stay).

As we seen here, the proportion of attrition or not is near **16%(237/1470)**, shows that **this dataset is highly unbalanced**. Thus, during the modeling process, we should work out [some techniques](https://www.analyticsvidhya.com/blog/2017/03/imbalanced-classification-problem/) to tackle this problem.


### The Distribution of Age
The majority of employee are between 28 - 36 years old, while the range of age are 18 - 60.


### Correlation Heatmap
From the plot, we can clearly observe quite a lot of feeatures are poorly correlated with one another. This would be greatful for us to construct the predictive model, we don't have to apply complex feature engineering, in order to reduce correalation between features.


### The Distribution of Monthly Income
A large majority of **low monthly income** tends to leave than the group of high monthly income.


### OverTime or not
There is a relatively higher percentage of people **working overtime** in the group of those who left.


### Education
The plot shows that the employee of **Bachelor** degree tends to leave company than the people have any other degrees.
(1 'Below College', 2 'College', 3 'Bachelor', 4 'Master', 5 'Doctor')


### Department
The ratio of Attrition in **Sales Dept**. is 20%(92/446), the highest among other departments, which is greater than R&D Dept. by 7%.
(Three Dept. in company: R&D, Sales, HRM)


### Business Travel Fequency
70% of the employee are Travel_Rarely, where about 12% are Non-Travel and 18% are Travel_Frequently.

There seems to be a clear indication that **those who left travelled more frequently compared to others**.


### Martial Status
**The employee whose status labeled single** have clear indication to leave the company, while the others who are married tend to be stable.


### The Relationship between Work-Life Balance & Distance From Home
Those who rated their work-life balance relatively low were **commuting from farther away** in comparison with those who rated their work-life balance as perfect.

## Model Stage
**Accuracy Baseline: 83%**
Due to the imbalance, random guessing all the employee won't leave would archieve **Acc: 83%(1233 / 1470)**.

Guessing whole the prediction to majority seems to be unrealistic, though it could get you a high Acc score. We should put more attention on how well does model predicts the minority, thus, focusing on **Receiver Operating Curve AUC(ROC-AUC)** in this case.

Applying **Ensemble Learning** Methodolgy to construct robust predictive model. Then using **Synthetic Minority Over-sampling Technique(SMOTE)** to solve the imbalanced problem.
- Majority Voting Commitee : Random Forest
- Gradient Boosting : XGBoost
- XGBoost + SMOTE

### 1. Random Forest
#### The importance features of RF


**Top Decisive Variables :**
Monthly Income, Total Working Years, OverTime, Age..

#### Metrics

### 2. XGBoost (eXtreme Gradient Boosting)
#### The importance features of XGBoost


**Top Decisive Variables :**
Total Working Year, Monthly Income, Enviroment Statisfication, OverTime, Education_Marketing...

#### Metrics

### 3. XGBoost with SMOTE (solve the imbalanced problem)
#### The importance features of XGBoost with SMOTE


**Top Decisive Variables :**
OverTime, Job Satisfaction, Stock Option Level, Work-Life Balance, Age.

#### Metrics

### Comparison
As the result, **XGBoost with SMOTE** performs outstanding. Not only Acc but also ROC-AUC are highest among three models we constructed.

| Model | Accuracy | ROC-AUC |
| -------- | -------- | -------- |
| Random Forest | 0.8665 | 0.5959 |
| XGBoost | 0.8552 | 0.6059 |
| **XGBoost with SMOTE** | **0.8946** | **0.8923** |

## Conclusion
We have constructed a simple pipeline of predicting employee attrition, from some basic Exploratory Data Analysis to implementing Ensemble Learning models in the form of a Random Forest and XGBoost. In addition, we tackle the imbalanced data with SMOTE strategy to make the model more practical. **The final model takes less than five seconds to run and it even returns a 89% accuracy and 0.8921 auc score in its predictions.**

### Reference
- [Workday, market application publishier](https://www.workday.com)
- [The Geeks Arrive In HR: People Analytics Is Here](https://www.forbes.com/sites/joshbersin/2015/02/01/geeks-arrive-in-hr-people-analytics-is-here/3/#40ff20a29841)
- [The Datafication of HR: People Science is Here](https://www.slideshare.net/jbersin/the-datafication-of-hr-people-science-is-here)
- [Project Oxygen: 8 Ways Google Resuscitated Management](https://blog.impraise.com/360-feedback/project-oxygen-8-ways-google-resuscitated-management)
- [There’s No Such Thing as Big Data in HR](https://hbr.org/2017/06/theres-no-such-thing-as-big-data-in-hr)
- [人資大未來 - 挖掘巨量資料，探尋問題癥結](https://www.slideshare.net/ssuser9c0ce41/2013-sd?next_slideshow=1)
- [大數據分析「潛進」辦公室，決定你的薪水還可預測你何時會離職](https://technews.tw/2014/11/10/big-data-into-office-predict-when-you-will-leave/)