Skip to content

Latest commit

 

History

History
129 lines (109 loc) · 6.58 KB

regression_result.md

File metadata and controls

129 lines (109 loc) · 6.58 KB
Raw

DataMiningHW2

logistic regression results, without SMOTE

dataset acc recall ROC_AUC PR_AUC F1 time_used
1year 0.710 0.143 0.430 0.011 0.421 1.581
2year 0.787 0.231 0.514 0.018 0.458 1.690
3year 0.872 0.182 0.536 0.028 0.499 1.868
4year 0.924 0.179 0.564 0.046 0.547 1.787
5year 0.925 0.421 0.682 0.109 0.621 1.026

logistic regression result, with SMOTE

dataset acc recall ROC_AUC PR_AUC F1 time_used
1year 0.668 0.286 0.479 0.012 0.410 20.871
2year 0.643 0.692 0.667 0.029 0.422 19.106
3year 0.650 0.682 0.666 0.040 0.436 19.910
4year 0.754 0.643 0.700 0.064 0.501 22.184
5year 0.813 0.789 0.802 0.112 0.561 16.805

results, threshold = sum(y_train) / len(y_train) + 0.05

1year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.954 0.000 0.500 0.046 0.488 0.495 2.253
smote 0.780 0.406 0.602 0.062 0.509 0.530 11.215
smote1 0.927 0.156 0.560 0.065 0.562 0.561 20.536
smote2 0.927 0.156 0.560 0.065 0.563 0.561 20.426
pipeline1 0.926 0.156 0.559 0.064 0.561 0.560 20.859
pipeline2 0.926 0.156 0.559 0.064 0.561 0.560 20.873

2year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.962 0.000 0.500 0.038 0.490 0.496 3.097
smote 0.732 0.461 0.602 0.050 0.478 0.494 12.245
smote1 0.962 0.000 0.500 0.038 0.490 0.496 12.252
smote2 0.962 0.000 0.500 0.038 0.490 0.496 11.468
pipeline1 0.962 0.000 0.500 0.038 0.490 0.496 12.536
pipeline2 0.962 0.000 0.500 0.038 0.490 0.496 11.958

3year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.957 0.000 0.500 0.043 0.489 0.496 3.118
smote 0.739 0.418 0.586 0.054 0.483 0.499 21.605
smote1 0.925 0.082 0.522 0.046 0.523 0.523 20.154
smote2 0.924 0.082 0.522 0.046 0.522 0.522 20.016
pipeline1 0.924 0.082 0.522 0.046 0.522 0.522 20.081
pipeline2 0.922 0.104 0.531 0.048 0.530 0.531 20.312

4year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.948 0.000 0.500 0.052 0.487 0.495 2.791
smote 0.747 0.634 0.693 0.097 0.528 0.568 17.780
smote1 0.914 0.242 0.596 0.091 0.591 0.594 20.142
smote2 0.915 0.255 0.603 0.096 0.596 0.600 19.833
pipeline1 0.915 0.261 0.606 0.097 0.598 0.603 20.413
pipeline2 0.914 0.261 0.605 0.096 0.597 0.602 20.120

5year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.932 0.000 0.500 0.068 0.482 0.493 1.594
smote 0.790 0.711 0.753 0.164 0.596 0.650 20.057
smote1 0.905 0.471 0.704 0.202 0.676 0.691 19.062
smote2 0.905 0.479 0.708 0.206 0.678 0.695 18.492
pipeline1 0.905 0.479 0.708 0.206 0.678 0.695 19.323
pipeline2 0.902 0.496 0.714 0.207 0.678 0.698 19.040

results, threshold = sum(y_train) / len(y_train)

1year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.954 0.000 0.500 0.046 0.488 0.495 2.221
smote 0.781 0.417 0.607 0.064 0.511 0.534 11.058
smote1 0.927 0.156 0.560 0.065 0.562 0.561 20.419
smote2 0.927 0.156 0.560 0.065 0.562 0.561 20.531
pipeline1 0.926 0.156 0.559 0.064 0.561 0.560 20.590
pipeline2 0.926 0.156 0.559 0.064 0.561 0.560 20.623

2year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.962 0.000 0.500 0.038 0.490 0.496 3.086
smote 0.727 0.461 0.599 0.050 0.476 0.491 13.868
smote1 0.962 0.000 0.500 0.038 0.490 0.496 12.344
smote2 0.962 0.000 0.500 0.038 0.490 0.496 11.581
pipeline1 0.962 0.000 0.500 0.038 0.490 0.496 12.580
pipeline2 0.962 0.000 0.500 0.038 0.490 0.496 11.808

3year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.957 0.000 0.500 0.043 0.489 0.496 3.071
smote 0.739 0.425 0.589 0.055 0.484 0.501 20.920
smote1 0.925 0.082 0.522 0.046 0.523 0.523 20.002
smote2 0.924 0.082 0.522 0.046 0.522 0.522 19.927
pipeline1 0.924 0.082 0.522 0.046 0.522 0.522 20.075
pipeline2 0.922 0.097 0.528 0.047 0.527 0.527 20.195

4year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.948 0.000 0.500 0.052 0.487 0.495 2.764
smote 0.746 0.641 0.696 0.098 0.528 0.569 19.306
smote1 0.915 0.248 0.600 0.093 0.594 0.597 20.167
smote2 0.915 0.255 0.603 0.096 0.596 0.600 19.810
pipeline1 0.915 0.261 0.606 0.098 0.599 0.603 20.359
pipeline2 0.913 0.261 0.605 0.096 0.596 0.601 20.255

5year

method Accuracy Recall ROC_AUC PR_AUC F1 F2 Time
None 0.932 0.000 0.500 0.068 0.482 0.493 1.599
smote 0.788 0.719 0.756 0.165 0.596 0.651 19.717
smote1 0.905 0.471 0.704 0.202 0.676 0.691 18.974
smote2 0.905 0.479 0.708 0.206 0.678 0.695 18.712
pipeline1 0.905 0.479 0.708 0.206 0.678 0.695 19.165
pipeline2 0.904 0.496 0.715 0.209 0.680 0.699 18.890