# All Features (29/11/2021) Iterative

## Iteration 1 - Original keypoints

### Overall score
Training accuracy 0.8929524765107294 +/- 0.013995257489632991
Cross-validation accuracy: 0.6831016694808427 +/- 0.026483745356732525
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('kbest', SelectKBest(k=360))])),
('classifier',
RidgeClassifier(alpha=0.02030917620904739,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8742382271468144 +/- 0.8742382271468144
Fold 0: Cross-validation accuracy: 0.7098445595854922 +/- 0.7098445595854922

Fold 1: Training accuracy 0.909778812572759 +/- 0.909778812572759
Fold 1: Cross-validation accuracy: 0.6680761099365751 +/- 0.6680761099365751

Fold 2: Training accuracy 0.8868335146898803 +/- 0.8868335146898803
Fold 2: Cross-validation accuracy: 0.7195467422096318 +/- 0.7195467422096318

Fold 3: Training accuracy 0.9087136929460581 +/- 0.9087136929460581
Fold 3: Cross-validation accuracy: 0.6527777777777778 +/- 0.6527777777777778

Fold 4: Training accuracy 0.8851981351981352 +/- 0.8851981351981352
Fold 4: Cross-validation accuracy: 0.6652631578947369 +/- 0.6652631578947369

### Fold distributions
Training folds are well distributed, test folds not so much.
Remark: folds with more easy to recognize signs perform better.

## Iteration 2 - Original keypoints without face features

### Overall score
Training accuracy 0.8366827771203214 +/- 0.006371249668605516
Cross-validation accuracy: 0.65505040849567 +/- 0.03401498477793019
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('kbest', SelectKBest(k=360))])),
('classifier',
RidgeClassifier(alpha=0.8376776400682924,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8271468144044322 +/- 0.8271468144044322
Fold 0: Cross-validation accuracy: 0.6632124352331606 +/- 0.6632124352331606

Fold 1: Training accuracy 0.8416763678696159 +/- 0.8416763678696159
Fold 1: Cross-validation accuracy: 0.6448202959830867 +/- 0.6448202959830867

Fold 2: Training accuracy 0.8454842219804135 +/- 0.8454842219804135
Fold 2: Cross-validation accuracy: 0.71671388101983 +/- 0.71671388101983

Fold 3: Training accuracy 0.8340248962655602 +/- 0.8340248962655602
Fold 3: Cross-validation accuracy: 0.621031746031746 +/- 0.621031746031746

Fold 4: Training accuracy 0.835081585081585 +/- 0.835081585081585
Fold 4: Cross-validation accuracy: 0.6294736842105263 +/- 0.6294736842105263

### Fold distributions
Worse score than using all features but scores of folds are a bit closer to each-other. One fold outperforms them all (2) but we notice that that one contains a lot of HAAS-oos (a class that is easily classified)

## Iteration 3 - Original keypoints without face features and fix-undetected and sync

### Overall score
Training accuracy 0.8430460818940008 +/- 0.008887841207751565
Cross-validation accuracy: 0.6643151309647638 +/- 0.011563874678297214
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('kbest', SelectKBest(k=360))])),
('classifier',
RidgeClassifier(alpha=0.8376776400682924,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8343490304709141 +/- 0.8343490304709141
Fold 0: Cross-validation accuracy: 0.6839378238341969 +/- 0.6839378238341969

Fold 1: Training accuracy 0.8591385331781141 +/- 0.8591385331781141
Fold 1: Cross-validation accuracy: 0.6511627906976745 +/- 0.6511627906976745

Fold 2: Training accuracy 0.8378672470076169 +/- 0.8378672470076169
Fold 2: Cross-validation accuracy: 0.6685552407932012 +/- 0.6685552407932012

Fold 3: Training accuracy 0.8458802608180201 +/- 0.8458802608180201
Fold 3: Cross-validation accuracy: 0.6547619047619048 +/- 0.6547619047619048

Fold 4: Training accuracy 0.837995337995338 +/- 0.837995337995338
Fold 4: Cross-validation accuracy: 0.6631578947368421 +/- 0.6631578947368421

### Fold distributions
Folds are closer to each other! We also have a better score although the distribution of the folds remains the same.

Why fold 2 worse? Maybe it learned that untracked hands belong to HAAS-oor. When untracking was fixed, haas-oor could be classified less accurate but lies closer to the other folds.

## Iteration 4 - Orig. kp no face, fix-undetect, sync and center

### Overall score
Training accuracy 0.8360936126616796 +/- 0.01372835908822496
Cross-validation accuracy: 0.6668011186354829 +/- 0.02922201137874495
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('kbest', SelectKBest(k=300))])),
('classifier',
RidgeClassifier(alpha=0.49238826317067413,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.821606648199446 +/- 0.821606648199446
Fold 0: Cross-validation accuracy: 0.6994818652849741 +/- 0.6994818652849741

Fold 1: Training accuracy 0.8608847497089639 +/- 0.8608847497089639
Fold 1: Cross-validation accuracy: 0.6257928118393234 +/- 0.6257928118393234

Fold 2: Training accuracy 0.8253536452665942 +/- 0.8253536452665942
Fold 2: Cross-validation accuracy: 0.6997167138810199 +/- 0.6997167138810199

Fold 3: Training accuracy 0.8352104327208062 +/- 0.8352104327208062
Fold 3: Cross-validation accuracy: 0.6626984126984127 +/- 0.6626984126984127

Fold 4: Training accuracy 0.8374125874125874 +/- 0.8374125874125874
Fold 4: Cross-validation accuracy: 0.6463157894736842 +/- 0.6463157894736842

### Fold distributions
Score is slightly better and we only use 300 features instead of 360

## Iteration 5 - Orig. kp no face, fix-undetect, sync, center and unify shoulder

### Overall score
Training accuracy 0.8531281167534821 +/- 0.014770899640708575
Cross-validation accuracy: 0.663717254019289 +/- 0.021415229930152112
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('kbest', SelectKBest(k=380))])),
('classifier',
RidgeClassifier(alpha=0.8376776400682924,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8354570637119113 +/- 0.8354570637119113
Fold 0: Cross-validation accuracy: 0.694300518134715 +/- 0.694300518134715

Fold 1: Training accuracy 0.8783469150174622 +/- 0.8783469150174622
Fold 1: Cross-validation accuracy: 0.6342494714587738 +/- 0.6342494714587738

Fold 2: Training accuracy 0.8422198041349293 +/- 0.8422198041349293
Fold 2: Cross-validation accuracy: 0.6770538243626062 +/- 0.6770538243626062

Fold 3: Training accuracy 0.8518079430942501 +/- 0.8518079430942501
Fold 3: Cross-validation accuracy: 0.6666666666666666 +/- 0.6666666666666666

Fold 4: Training accuracy 0.8578088578088578 +/- 0.8578088578088578
Fold 4: Cross-validation accuracy: 0.6463157894736842 +/- 0.6463157894736842

### Fold distributions
We have a worse (-0.003) score when adding unified shoulder (why?)

## Iteration 6 - Orig. kp no face, fix-undetect, sync, center, unify shoulder and LD mothafucking A

### Overall score
Training accuracy 0.8876427949409464 +/- 0.012702597974971514
Cross-validation accuracy: 0.671975836843225 +/- 0.03355775359978403
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select',
ColumnTransformer(transformers=[('old_feats_body', LDA(42)
('classifier',
RidgeClassifier(alpha=100.0, class_weight='balanced',
max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8692520775623269 +/- 0.8692520775623269
Fold 0: Cross-validation accuracy: 0.7253886010362695 +/- 0.7253886010362695

Fold 1: Training accuracy 0.9068684516880093 +/- 0.9068684516880093
Fold 1: Cross-validation accuracy: 0.6257928118393234 +/- 0.6257928118393234

Fold 2: Training accuracy 0.8813928182807399 +/- 0.8813928182807399
Fold 2: Cross-validation accuracy: 0.6855524079320113 +/- 0.6855524079320113

Fold 3: Training accuracy 0.8855957320687611 +/- 0.8855957320687611
Fold 3: Cross-validation accuracy: 0.6726190476190477 +/- 0.6726190476190477

Fold 4: Training accuracy 0.8951048951048951 +/- 0.8951048951048951
Fold 4: Cross-validation accuracy: 0.6505263157894737 +/- 0.6505263157894737

### Fold distributions
We have a pretty good score for only 42 features! BUT... test and cross-validation gap still significant(67 vs 63) and training and cross-validation gap still stagnating (might be explained by the few features)

## Iteration 7 - Orig. kp no face, fix-undetect, sync, center, unify shoulder, LDA and finger lenghts

### Overall score
Training accuracy 0.8766903000353615 +/- 0.012473732068323124
Cross-validation accuracy: 0.6854659564375706 +/- 0.026140365610837244
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select', LDA and new features)])),
('classifier',
RidgeClassifier(alpha=2424.462017082326,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8581717451523546 +/- 0.8581717451523546
Fold 0: Cross-validation accuracy: 0.7331606217616581 +/- 0.7331606217616581

Fold 1: Training accuracy 0.8963911525029103 +/- 0.8963911525029103
Fold 1: Cross-validation accuracy: 0.6659619450317125 +/- 0.6659619450317125

Fold 2: Training accuracy 0.8710554951033732 +/- 0.8710554951033732
Fold 2: Cross-validation accuracy: 0.6940509915014165 +/- 0.6940509915014165

Fold 3: Training accuracy 0.8772969768820391 +/- 0.8772969768820391
Fold 3: Cross-validation accuracy: 0.6646825396825397 +/- 0.6646825396825397

Fold 4: Training accuracy 0.8805361305361306 +/- 0.8805361305361306
Fold 4: Cross-validation accuracy: 0.6694736842105263 +/- 0.6694736842105263

### Fold distributions
Adding new features yielded an increase of about 1%. Fold validation scores lie a bit closer to each-other but the most extreme fold (1) has increased even more.

## Iteration 8 - Orig. kp no face, fix-undetect, sync, center, unify shoulder, LDA and finger lenghts (longer interpol)

### Overall score
Training accuracy 0.8767418496522925 +/- 0.014295838228407583
Cross-validation accuracy: 0.6848646725148474 +/- 0.028474976829306636
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select',LDA + kbest = 30)]))
('classifier',
RidgeClassifier(alpha=2424.462017082326,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8548476454293629 +/- 0.8548476454293629
Fold 0: Cross-validation accuracy: 0.7357512953367875 +/- 0.7357512953367875

Fold 1: Training accuracy 0.8981373690337602 +/- 0.8981373690337602
Fold 1: Cross-validation accuracy: 0.6638477801268499 +/- 0.6638477801268499

Fold 2: Training accuracy 0.8699673558215452 +/- 0.8699673558215452
Fold 2: Cross-validation accuracy: 0.6968838526912181 +/- 0.6968838526912181

Fold 3: Training accuracy 0.8778897451096621 +/- 0.8778897451096621
Fold 3: Cross-validation accuracy: 0.6646825396825397 +/- 0.6646825396825397

Fold 4: Training accuracy 0.8828671328671329 +/- 0.8828671328671329
Fold 4: Cross-validation accuracy: 0.6631578947368421 +/- 0.6631578947368421

### Fold distributions
We dont notice any great improvement. This means that adding more interpolation frames doesnt really improve overall score.

## Iteration 9 - Orig. kp no face, fix-undetect, sync, center, unify shoulder, LDA and body angles

### Overall score
Training accuracy 0.8745785410139281 +/- 0.01572602870822609
Cross-validation accuracy: 0.6852942924720761 +/- 0.03307577664995871
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select',LDA + all body angles)]))
('classifier',
RidgeClassifier(alpha=2424.462017082326,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8509695290858725 +/- 0.8509695290858725
Fold 0: Cross-validation accuracy: 0.7435233160621761 +/- 0.7435233160621761

Fold 1: Training accuracy 0.8993015133876601 +/- 0.8993015133876601
Fold 1: Cross-validation accuracy: 0.6448202959830867 +/- 0.6448202959830867

Fold 2: Training accuracy 0.8683351468988031 +/- 0.8683351468988031
Fold 2: Cross-validation accuracy: 0.6940509915014165 +/- 0.6940509915014165

Fold 3: Training accuracy 0.8743331357439241 +/- 0.8743331357439241
Fold 3: Cross-validation accuracy: 0.6746031746031746 +/- 0.6746031746031746

Fold 4: Training accuracy 0.87995337995338 +/- 0.87995337995338
Fold 4: Cross-validation accuracy: 0.6694736842105263 +/- 0.6694736842105263

### Fold distributions
We see an improvement compared to iteration 6 which might mean that the body angles are useful features to add.

## Iteration 10 - Orig. kp no face, fix-undetect, sync, center, unify shoulder, LDA and body angles (more interpol 4)

### Overall score
Training accuracy 0.8745752189269901 +/- 0.01514973094040969
Cross-validation accuracy: 0.6867412423567442 +/- 0.029966149309334698
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select': LDA + kbest=32)]))
('classifier',
RidgeClassifier(alpha=2424.462017082326,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8509695290858725 +/- 0.8509695290858725
Fold 0: Cross-validation accuracy: 0.7409326424870466 +/- 0.7409326424870466

Fold 1: Training accuracy 0.8975552968568102 +/- 0.8975552968568102
Fold 1: Cross-validation accuracy: 0.6511627906976745 +/- 0.6511627906976745

Fold 2: Training accuracy 0.8688792165397171 +/- 0.8688792165397171
Fold 2: Cross-validation accuracy: 0.6912181303116147 +/- 0.6912181303116147

Fold 3: Training accuracy 0.8755186721991701 +/- 0.8755186721991701
Fold 3: Cross-validation accuracy: 0.6746031746031746 +/- 0.6746031746031746

Fold 4: Training accuracy 0.87995337995338 +/- 0.87995337995338
Fold 4: Cross-validation accuracy: 0.6757894736842105 +/- 0.6757894736842105

### Fold distributions
We have a slight improvement by adding more frames. Unlike adding frames with finger lengths, this one improved a bit. This might imply that body angles contain more temporal information than finger lengths.

## Iteration 11 - Orig. kp no face, fix-undetect, sync, center, unify shoulder, LDA and body angles (more interpol 8)

### Overall score
Training accuracy 0.8802791943125555 +/- 0.014468509930616727
Cross-validation accuracy: 0.6911229426592922 +/- 0.032164458538195606
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select',LDA + kbest 116)]))
('classifier',
RidgeClassifier(alpha=1511.77507061566,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8587257617728532 +/- 0.8587257617728532
Fold 0: Cross-validation accuracy: 0.7383419689119171 +/- 0.7383419689119171

Fold 1: Training accuracy 0.9027939464493597 +/- 0.9027939464493597
Fold 1: Cross-validation accuracy: 0.6448202959830867 +/- 0.6448202959830867

Fold 2: Training accuracy 0.8732317736670294 +/- 0.8732317736670294
Fold 2: Cross-validation accuracy: 0.7138810198300283 +/- 0.7138810198300283

Fold 3: Training accuracy 0.8814463544754001 +/- 0.8814463544754001
Fold 3: Cross-validation accuracy: 0.6785714285714286 +/- 0.6785714285714286

Fold 4: Training accuracy 0.8851981351981352 +/- 0.8851981351981352
Fold 4: Cross-validation accuracy: 0.68 +/- 0.68

### Fold distributions
Yay, we have an even better score now! Unfortunately, selectKbest selected 116 angle features instead of the previous 32 angle features. This is only a slight improvement for many more features added.

## Iteration 12 - Orig. kp no face, fix-undetect, sync, center, unify shoulder, LDA and body distances

### Overall score
Training accuracy 0.8648751415401035 +/- 0.015861879270712077
Cross-validation accuracy: 0.6831993575663304 +/- 0.03170117761353051
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select',LDA + all feat)]))
('classifier',
RidgeClassifier(alpha=6235.507341273912,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8404432132963989 +/- 0.8404432132963989
Fold 0: Cross-validation accuracy: 0.7409326424870466 +/- 0.7409326424870466

Fold 1: Training accuracy 0.889988358556461 +/- 0.889988358556461
Fold 1: Cross-validation accuracy: 0.6469344608879493 +/- 0.6469344608879493

Fold 2: Training accuracy 0.8601741022850925 +/- 0.8601741022850925
Fold 2: Cross-validation accuracy: 0.6855524079320113 +/- 0.6855524079320113

Fold 3: Training accuracy 0.8672199170124482 +/- 0.8672199170124482
Fold 3: Cross-validation accuracy: 0.6646825396825397 +/- 0.6646825396825397

Fold 4: Training accuracy 0.8665501165501166 +/- 0.8665501165501166
Fold 4: Cross-validation accuracy: 0.6778947368421052 +/- 0.6778947368421052

### Fold distributions
Better score but not as good as the angles. Folds are bit less equal.

## Iteration 13 - Orig. kp no face, fix-undetect, sync, center, unify shoulder, LDA and body distances (more interpol 8)

### Overall score
Training accuracy 0.8676540071976543 +/- 0.014847237268801883
Cross-validation accuracy: 0.685021932408308 +/- 0.03154627789688863
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select',LDA + kbest=52)]))
('classifier',
RidgeClassifier(alpha=3888.155180308085,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8426592797783934 +/- 0.8426592797783934
Fold 0: Cross-validation accuracy: 0.7357512953367875 +/- 0.7357512953367875

Fold 1: Training accuracy 0.889406286379511 +/- 0.889406286379511
Fold 1: Cross-validation accuracy: 0.638477801268499 +/- 0.638477801268499

Fold 2: Training accuracy 0.8683351468988031 +/- 0.8683351468988031
Fold 2: Cross-validation accuracy: 0.6940509915014165 +/- 0.6940509915014165

Fold 3: Training accuracy 0.8684054534676942 +/- 0.8684054534676942
Fold 3: Cross-validation accuracy: 0.6726190476190477 +/- 0.6726190476190477

Fold 4: Training accuracy 0.8694638694638694 +/- 0.8694638694638694
Fold 4: Cross-validation accuracy: 0.6842105263157895 +/- 0.6842105263157895

### Fold distributions
We have a slightly better score but maybe not worth it for the extra amount of features

## Iteration 14 - Orig. kp no face, fix-undetect, sync, center, unify shoulder, LDA and every above

### Overall score
Training accuracy 0.8741393729422737 +/- 0.013570018607200444
Cross-validation accuracy: 0.6951640244979987 +/- 0.02591457020895899
Best estimator:
Pipeline(steps=[('preprocessing',
Pipeline(steps=[('scaler', StandardScaler())])),
('feature_selection',
Pipeline(steps=[('feat_select', LDA + body_dist=1 + body_angles=81 + finger_lengths = ?)]))
('classifier',
RidgeClassifier(alpha=2335.7214690901214,
class_weight='balanced', max_iter=10000))])

### Fold scores
Fold 0: Training accuracy 0.8548476454293629 +/- 0.8548476454293629
Fold 0: Cross-validation accuracy: 0.7331606217616581 +/- 0.7331606217616581

Fold 1: Training accuracy 0.8946449359720605 +/- 0.8946449359720605
Fold 1: Cross-validation accuracy: 0.653276955602537 +/- 0.653276955602537

Fold 2: Training accuracy 0.8645266594124048 +/- 0.8645266594124048
Fold 2: Cross-validation accuracy: 0.6997167138810199 +/- 0.6997167138810199

Fold 3: Training accuracy 0.8778897451096621 +/- 0.8778897451096621
Fold 3: Cross-validation accuracy: 0.6865079365079365 +/- 0.6865079365079365

Fold 4: Training accuracy 0.8787878787878788 +/- 0.8787878787878788
Fold 4: Cross-validation accuracy: 0.7031578947368421 +/- 0.7031578947368421

### Fold distributions