---
title: "BCCML"
author: Yuxuan Guo, Puyuan Liu, Qinyi Huang
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{BCCML}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---

knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)

# Implement Machine Learning Methods
___
## Step 1: Data Preprocessing
Preprocess data by handling missing values and standardize data in a data frame.

### Handling Missing Values
The first phase is handling missing values. Within a feature, mean value of the
available data is used to fill in the blank spaces if there are any. The function
responsible for this task is __handle_missing_values(data_frame)__.

library(BCCML)
data_frame <- data.frame(feature_1 = c(1, 2, NA, 3), feature_2 = c(4, NA, NA, 5))
print(data_frame)
new_data_frame <- handle_missing_values(data_frame)
print(new_data_frame)

### Data Standardization
The second phase is standardize values. The function for this task is
__standardize(data_frame, method_number)__ There are four methods available in this
package. Users can enter 1 for the second parameter for the z score method. 2
corresponds to min-max scaling method. 3 corresponds to standard deviation method.
Finally, 4 is for the range method

library(BCCML)
data_frame <- BCCML::data_set[1:length(BCCML::data_set)-1] # The last column is label
print(head(data_frame))
new_data_frame <- standardize(data_frame, 1)
print(head(new_data_frame))

library(BCCML)
data_frame <- BCCML::data_set[1:length(BCCML::data_set)-1] # The last column is label
print(head(data_frame))
new_data_frame <- standardize(data_frame, 2)
print(head(new_data_frame))

library(BCCML)
data_frame <- BCCML::data_set[1:length(BCCML::data_set)-1] # The last column is label
print(head(data_frame))
new_data_frame <- standardize(data_frame, 3)
print(head(new_data_frame))

library(BCCML)
data_frame <- BCCML::data_set[1:length(BCCML::data_set)-1] # The last column is label
print(head(data_frame))
new_data_frame <- standardize(data_frame, 4)
print(head(new_data_frame))

## Step 2: Data Dimensionality Reduction
Linear Discriminant Analysis and Principal Component Analysis can be chosen to reduce
the dimension of the original data

### Linear Discriminant Analysis
The __lda(X, y, d)__ function reduces data dimension by linear discriminant analysis.
X is the original data matrix. y is the label vector which specify the class each
data point belongs to. d is the new dimension of the transformed matrix. Here the example
data has 5 attributes so d can be 1 to 4. To demonstration the d is set to be 1.

library(BCCML)
data_matrix <- as.matrix(BCCML::data_set[1:length(BCCML::data_set) - 1]) # The last column is label
print(head(data_matrix))
label_vector <- BCCML::data_set[, length(BCCML::data_set)]
print(label_vector)
new_matrix <- lda(data_matrix, label_vector, 1)
print(head(new_matrix))

### Principal Component Analysis
The function that does principal component analysis is __pca(X, d)__. X is the
original data matrix. d is the new dimension of the transformed matrix. Note that
the number inputted should be less than the original dimension. Here the example
data has 5 attributes so d can be 1 to 4. To demonstration the d is set to be 3.

library(BCCML)
data_matrix <- as.matrix(BCCML::data_set[, 1:(length(BCCML::data_set) - 1)]) # The last column is label
print(head(data_matrix))
new_matrix <- pca(data_matrix, 3)
print(head(new_matrix))

## Step 3: Dataset Splitting

### trainTestPartition
The function aims to split the data into training set and test set. The function split data into two parts randomly. For example, when rate = 0.7, we randomly sample 70% of the data from the data set to generate the training set, and the remain part is the test set. The input of the function are the dataframe of dataset that requires splitting and the splitting rate (floating number in range(0, 1))

library(BCCML)
splitted_dt <- trainTestPartition(BCCML::data_set, 0.7)
train_set <- splitted_dt[[1]]
test_set <- splitted_dt[[2]]
print(head(train_set))
print(head(test_set))

## Step 4: Supervised & Unsupervised Learning

### k-Nearest-Neighbors (Supervised Learning)

Neighbors-based classification is a type of instance-based learning or non-generalizing learning: it does not attempt to construct a general internal model, but simply stores instances of the training data. Classification is computed from a simple majority vote of the nearest neighbors of each point: a query point is assigned the data class which has the most representatives within the nearest neighbors of the point.

The $k$-neighbors classification is the most commonly used technique. The optimal choice of the value $k$ is highly data-dependent: in general a larger $k$ suppresses the effects of noise, but makes the classification boundaries less distinct.

The distance metrics can be different (*e.g.,* Gaussian kernel), which will be supported in the next version.

The input of the function:
1. the training dataset (integers), a matrix or dataframe for model training.
2. the training labels (integers starting from 1), a matrix, dataframe, or vector for model training. Its number of elements should fit with the row number of training dataset.
3. the testing dataset (integers), a matrix or dataframe for model testing.
4. the hyper-parameter $k$, which is $5$ by default.

The function will return a vector containing the predicted labels (integers) of the testing dataset.

Examples:

library(BCCML)
splitted_dt <- trainTestPartition(BCCML::data_set, 0.8)
trainX <- splitted_dt[[1]][, 1:5]
trainY <- splitted_dt[[1]][, 6]
testX <- splitted_dt[[2]][, 1:5]
trueY <- splitted_dt[[2]][, 6]
print(trueY)
print(kNN(trainX, trainY, testX, 3))

### k-Means (Unsupervised Learning)

The KMeans algorithm clusters data by trying to separate samples in $n$ groups of equal variance, minimizing a criterion known as the inertia or within-cluster sum-of-squares (see below). This algorithm requires the number of clusters to be specified. It scales well to large number of samples and has been used across a large range of application areas in many different fields.

The k-means algorithm divides a set of $N$ samples $X$ into $K$ disjoint clusters $C$, each described by the mean $\mu_j$ of the samples in the cluster. The means are commonly called the cluster "centroids ; note that they are not,in general, points from X, although they live in the same space.

The K-means algorithm aims to choose centroids that minimize the **inertia**, or **within-cluster sum-of-squares criterion**:

$$
\sum _ { i = 0 } ^ { n } \operatorname { min }_{\mu_j \in C} ( | | x _ { i } - \mu _ { j } || ^ { 2 } )
$$

Inertia can be recognized as a measure of how internally coherent clusters are. It is not a normalized metric: we just know that lower values are better and zero is optimal. But in very high-dimensional spaces, Euclidean distances tend to become inflated (this is an instance of the so-called “curse of dimensionality”). Running a dimensionality reduction algorithm such as Principal component analysis (PCA) prior to k-means clustering can alleviate this problem and speed up the computations.

The input of the function:
1. the unlabeled dataset (integers), a matrix or dataframe for clustering.
2. the hyper-parameter $k$, which is $3$ by default.

The function will return a list containing:
1. a vector named "labels" containing predicted labels (integers) of the data.
2. a $m \times n$ matrix named "centroids" containing the centroids coordinates, where $m$ is the number of centroids and $n$ is the number of dimensions.

Examples:

library(BCCML)
print(kMeans(BCCML::data_set[, 1:5], 3))

## Step 5: Model Evaluations

### k-Fold Cross Validation (Optimizer for kNN)

When evaluating different settings (“hyperparameters”) for estimators, there is still a risk of overfitting on the test set because the parameters can be tweaked until the estimator performs optimally. This way, knowledge about the test set can “leak” into the model and evaluation metrics no longer report on generalization performance. To solve this problem, yet another part of the dataset can be held out as a so-called “validation set”: training proceeds on the training set, after which evaluation is done on the validation set, and when the experiment seems to be successful, final evaluation can be done on the test set.

However, by partitioning the available data into three sets, we drastically reduce the number of samples which can be used for learning the model, and the results can depend on a particular random choice for the pair of (train, validation) sets.

A solution to this problem is a procedure called cross-validation (CV for short). A test set should still be held out for final evaluation, but the validation set is no longer needed when doing CV. In the basic approach, called $k$-fold CV, the training set is split into $k$ smaller sets (other approaches are described below, but generally follow the same principles). The following procedure is followed for each of the $k$ “folds”:

* A model is trained using $k - 1$ of the folds as training data;
* the resulting model is validated on the remaining part of the data (i.e., it is used as a test set to compute a performance measure such as accuracy).

The performance measure reported by $k$-fold cross-validation is then the average of the values computed in the loop. This approach can be computationally expensive, but does not waste too much data (as is the case when fixing an arbitrary validation set), which is a major advantage in problems such as inverse inference where the number of samples is very small.

In this version of pacakge, this function is used to determine the optimal $k$ for k-Nearest-Neighbor algorithm.

The input of the function:
1. the training dataset (integers), a matrix or dataframe for training.
2. the training labels (integers starting from 1), a matrix, dataframe, or vector for model training. Its number of elements should fit with the row number of training dataset.
3. the hyper-parameter range, a vector containing all possible hyper-parameter values. Because this function only uses kNN, the elements should all be integers.
4. the number of fold, which is $10$ by default.

The function will return the optimal hyper-parameter picked from the given hyper-parameter range.

Examples:

library(BCCML)
X <- BCCML::data_set[, 1:5]
y <- BCCML::data_set[, 6]
print(kFoldCV_kNN(X, y, seq(1, 9, 2)))

### lossOfKMeans

The loss function of K-Means Method can be computed by the sum of squares of the Euclidean distance of each predicted labels to the centroid of its own class. The loss function can be used to evaluate the performance of the K-Means model. After changing the value of k and generate different models, this function can help to evaluate and compare the performance of each model.

The input of the function:
1. the dataset used to test the model (dataframe format).
2. the corresponding vector (integers) of predicted labels by the training algorithm.
3. the matrix (integers) of the coordinates of centroids. Each row of the matrix is the coordinate of the centroid of a class.

The function will return the loss of the currently trained model.

For example, we randomly assign the predicted labels and centroids to calculate the loss:

library(BCCML)
predictedLabels  = sample(c(rep(1,23),rep(2,16), rep(3,20)))
centroids = matrix(c(1, 4, 6, 2, 5, 10, 16, -3, 6, -9, 16, -20, 22, -1, 12), nrow = 3, byrow = T)
print(lossOfKMeans(BCCML::data_set[, 1:5], predictedLabels, centroids))

### ElbowMethod

This function can help you plot the loss of all k's appling the k-Means Method and determine the possible best k's in the model. The function also incorporates a reasonable threshold to identify all the possible elbows, which can help you to identify the possible elbows in the plot.

The input of the function:
1. X: a m by n feature matrix or data frame, where m is the sample size and n is the number of features. All of the elements should be numeric.
2. hyperParaRange: an integer denoting the number of dimension of features, i.e. the number of attributes in the data set.

The function will generate a plot with a line characterizing the loss behavior of the K-Means model
with all possible k. It will also display the possible elbows by red points.

The function will return a list with components:
1. possibleElbows- a vector containing possible elbows(k's) in the K-Means model which serve as best k's.
2. elbowLoss - a vector containing loss of all possible elbows(k's) in the K-Means model.

library(BCCML)
print(ElbowMethod(BCCML::data_set[,1:5], 5))

### knnEvaluation
In order to evaluate the performance of the KNN model, you can use the function to compute the confusion matrix and prediction accuracy.

You need to input two vectors as parameters:
1. the corresponding vector (integers) of predicted labels by the training algorithm.
2. the vector (integers) of the actual labels of test data.

The function will return a list of two element:
1. A matrix - The confusion matrix of predicted and actual labels.
2. value - The accuracy score of prediction

For example, we randomly assign the predicted labels and actual abels with the same length:

library(BCCML)
predictedLbs = sample(c(rep(1,23),rep(2,16), rep(3,20)))
actualLbs = sample(c(rep(1,19),rep(2,18), rep(3,22)))
print(knnEvaluation(predictedLbs, actualLbs))

### k-Means Result Visualization
Visualization of k-Means method will facilitate users' decision on suitable model and parameter choice.

This function utilizes PCA to reduce the dataset dimensionality to 2. It can visualize the data points and color them according to which cluster that they belong to. The centriods are also plotted as crosses while the data points as circles.

The input of the function
1. a list which is the return value of function \code{kMeans}.
2. a $m$ by $n$ feature matrix or data frame, the same one passed in the function *kMeans*, $m$ is the sample size and $n$ is the number of features. All of the elements should be numeric.

The function displays a plot and returns nothing.

Examples:

library(BCCML)
X <- BCCML::data_set[, 1:5]
result <- kMeans(X, 3)
kMeansPlot(result, X)

___
## Advantages of the Package
1. Minimized loops in main algorithms to improve performance.
1. Established an integrated and comprehensive data analysis pipeline.
1. Allowed cutomized function inputs for users.
1. Provided mature and detailed user guide to help users get started.
1. Provided plots to visualize the performance of the models.
1. Constructed automatic optimizers for both kNN and kMeans algorithms.

___
## Knowledge Points
1. Identifying NA by is.na().
1. Extract the target columns and rows of data frames.
1. branching with "if" argument.
1. Using apply() to transform data matrix.
1. Applying matrix multiplication.
1. The use of basic plot function, applying useful optional arguments such as lty, lwd.
1. Use table() function to summarize the data.
1. Looping with for.
1. Return of functions: automatically return the last expression.
1. The use of built-in function of vectors such as sum(x).
1. The use of logical expressions such as >, <, ==.
1. Extracting data frames by $x, [,i], [[i]].
1. Using dim() to get the dimension of dataframe.