03_Model_selection.ipynb

{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {
    "vscode": {
     "languageId": "raw"
    }
   },
   "source": [
    "---\n",
    "title: \"ClassifyAnything Part 3: Model Selection\"\n",
    "author: \"Joseph Chao-Chung Kuo (ckuo@ukaachen.de)\"\n",
    "date: today\n",
    "output:\n",
    "  general:\n",
    "    output_stdout: true\n",
    "    output_error: false\n",
    "    output: true\n",
    "format:\n",
    "  html:\n",
    "    toc: true\n",
    "    toc-depth: 3\n",
    "    html-math-method: katex\n",
    "    code-fold: true\n",
    "    code-tools: true\n",
    "    number_sections: true\n",
    "    embed-resources: true\n",
    "    page-layout: full\n",
    "execute:\n",
    "    echo: true\n",
    "    warning: false\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "_Supervised learning for classification_\n",
    "\n",
    "Model selection includes splitting the data into training and test sets. The training set is used to train individual models and the test set is for evaluating the performances of the models. The codes for training each model are separated in different files (one model one file). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import pachages\n",
    "from sklearn.model_selection import train_test_split\n",
    "import os\n",
    "import pickle\n",
    "import pandas as pd\n",
    "from sklearn.feature_selection import VarianceThreshold\n",
    "from sklearn.model_selection import KFold\n",
    "from IPython.display import display\n",
    "\n",
    "with open(\"outputs/02_Variables.pkl\", 'rb') as file:\n",
    "    (data, labels, y) = pickle.load(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Splitting data into training/test sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(data, \n",
    "                                                    y, \n",
    "                                                    test_size=0.4, \n",
    "                                                    random_state=41, \n",
    "                                                    stratify=y)\n",
    "print(X_train.shape)\n",
    "print(y_train.shape)\n",
    "print(X_test.shape)\n",
    "print(y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 K-Fold Cross-Validation\n",
    "\n",
    "The folds of cross-validation are generated once and used by different models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kfold = KFold(n_splits=5, random_state=42, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Save"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "directory = 'outputs'\n",
    "if not os.path.exists(directory):\n",
    "    os.makedirs(directory)\n",
    "# Save variables to a file\n",
    "with open('outputs/03_Variables.pkl', 'wb') as file:\n",
    "    pickle.dump((X_train, X_test, y_train, y_test, kfold), file)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}