AfriSenti_SemEval_2023_Starter_Notebook_Task_A.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AEkdLm_JF84s"
      },
      "source": [
        "<center>\n",
        "<img src=\"https://raw.githubusercontent.com/afrisenti-semeval/afrisent-semeval-2023/main/afrisenti-logo.png\" width=\"30%\" />\n",
        "</center>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pXY0zNpmLM1Z"
      },
      "source": [
        "<center>\n",
        "\n",
        "#SemEval 2023 Shared Task 12: AfriSenti (Task A)\n",
        "\n",
        "###Starter Notebook\n",
        "\n",
        "</center>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jURfhK3zGM9x"
      },
      "source": [
        "#Leveraging Pre-trained A Language Model to Train A Sentiment Classifier\n",
        "\n",
        "**Authors:**\n",
        "[Idris Abdulmumin](https://www.hausanlp.org/author/idris-abdulmuminu/), [David Adelani](https://dadelani.github.io/) and [Shamsuddeen Hassan Muhammad](https://www.hausanlp.org/author/shamsuddeen-hassan-muhammad/).\n",
        "\n",
        "**Introduction:** \n",
        "\n",
        "You are welcome to participate in our first-of-its-kind SemEval Shared Task! \n",
        "\n",
        "In this starter notebook, we will take you through the process of fine-tuning a pre-trained language model on a sample data to build a sentiment classifier. The notebook was adapted from a [Hugginface implementation]( https://github.com/huggingface/transformers/blob/main/examples/pytorch/text-classification/run_xnli.py) for such tasks.\n",
        "\n",
        "**Languages (Tracks)**\n",
        "* Track 1: Hausa \n",
        "* Track 2: Yoruba\n",
        "* Track 3: Igbo\n",
        "* Track 4: Nigerian_Pidgin\n",
        "* Track 5: Amharic\n",
        "* Track 6: Algerian Arabic\n",
        "* Track 7: Moroccan Arabic/Darija,\n",
        "* Track 8: Swahili\n",
        "* Track 9: Kinyarwanda (data to be released soon)\n",
        "* Track 10: Twi (data to be released soon)\n",
        "* Track 11: Mozambican Portuguese\n",
        "* Track 12: Setswana (data to be released soon)\n",
        "* Track 13: isiZulu (data to be released soon)\n",
        "\n",
        "**Level:** <font color='blue'>`Beginner to Intermediate`</font>\n",
        "\n",
        "**Outline:** \n",
        "\n",
        "1. Installation and importation of necessary libraries\n",
        "2. Setting up the project parameters.\n",
        "3. Running training and evaluation\n",
        "\n",
        "**Before you start:**\n",
        "\n",
        "It is **strongly advised** that you use a GPU to speed up training. To do this, go to the \"Runtime\" menu in Colab, select \"Change runtime type\" and then in the popup menu, choose \"GPU\" in the \"Hardware accelerator\" box.\n",
        "\n",
        "NB: \n",
        "\n",
        "- **The codes in this notebook are provided to familiarize yourselves with fine-tuning language models for sentiment classification. You may extend and (or) modify as appropriate to obtain competitive performances**\n",
        "\n",
        "- **We also use the data as it is, without any cleaning such as removal of emoji and hyperlinks.**\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V4BB4JDx5W8d"
      },
      "source": [
        "#1) Installations and imports"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DObkW3ulM7yg"
      },
      "source": [
        "##a. Mount drive (if you are running on colab)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z0IelbK7NBZZ",
        "outputId": "b38e1635-51e3-498d-8b64-6a6df54dc1c4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U-OZxUWIMqtq"
      },
      "source": [
        "##b. Clone or update competition repository\n",
        "\n",
        "After cloning, under MyDrive, you will see afrisenti-semeval-2023 folder with all the the data for the afrisenti shared task (training and dev) "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2392TKadMvqT"
      },
      "outputs": [],
      "source": [
        "%cd /content/drive/MyDrive\n",
        "\n",
        "import os\n",
        "\n",
        "PROJECT_DIR = '/content/drive/MyDrive/afrisent-semeval-2023'\n",
        "PROJECT_GITHUB_URL = 'https://github.com/afrisenti-semeval/afrisent-semeval-2023.git'\n",
        "\n",
        "if not os.path.isdir(PROJECT_DIR):\n",
        "  !git clone {PROJECT_GITHUB_URL}\n",
        "else:\n",
        "  %cd {PROJECT_DIR}\n",
        "  !git pull {PROJECT_GITHUB_URL}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xb03Gp9fUN8C"
      },
      "source": [
        "##c. Install required libraries\n",
        "\n",
        "- Set the project dire\n",
        "ctory in the cell below, where the requirements file should also be located, and run the cell"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4Cbmi_mQ4k3a"
      },
      "outputs": [],
      "source": [
        "if os.path.isdir(PROJECT_DIR):\n",
        "  #The requirements file should be in PROJECT_DIR\n",
        "  if os.path.isfile(os.path.join(PROJECT_DIR, 'starter_kit/requirements.txt')):\n",
        "    !pip install -r starter_kit/requirements.txt\n",
        "  else:\n",
        "    print('requirements.txt file not found')\n",
        "\n",
        "else:\n",
        "  print(\"Project directory not found, please check again.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2zszKhh2Ufb3"
      },
      "source": [
        "##d. Import libraries\n",
        "\n",
        "Import libraries below"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "8QIl420aUM1O"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "# Please don not edit anything here\n",
        "languages = ['am', 'dz', 'ha', 'ig', 'ma', 'pcm', 'pt', 'sw', 'yo']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SoRyMJMDJ7lF"
      },
      "source": [
        "#2) Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3wk5vMnrXMSS"
      },
      "source": [
        "##a. Formatting\n",
        "\n",
        "The training dataset that was provided for the competition is in the following format:\n",
        "\n",
        "| ID | text | label |\n",
        "| --- | --- | --- |\n",
        "| twt001 | example text | negative |\n",
        "| twt002 | example text | positive |\n",
        "| ... | ... | ... |\n",
        "\n",
        "However, the code in the starter kit do not expect the \n",
        "ID and require the training (and evaluation) data to be in the following format\n",
        "\n",
        "|text | label |\n",
        "|--- | --- |\n",
        "|example text | negative |\n",
        "|example text | positive |\n",
        "|... | ... |\n",
        "\n",
        "To reformat the data run the following cell"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Training Data Paths\n",
        "\n",
        "TASK = 'SubtaskA'\n",
        "TRAINING_DATA_DIR = os.path.join(PROJECT_DIR, TASK, 'train')\n",
        "FORMATTED_TRAIN_DATA = os.path.join(TRAINING_DATA_DIR, 'formatted-train-data')\n",
        "\n",
        "if os.path.isdir(TRAINING_DATA_DIR):\n",
        "  print('Data directory found.')\n",
        "  if not os.path.isdir(FORMATTED_TRAIN_DATA):\n",
        "    print('Creating directory to store formatted data.')\n",
        "    os.mkdir(FORMATTED_TRAIN_DATA)\n",
        "else:\n",
        "  print(TRAINING_DATA_DIR + ' is not a valid directory or does not exist!')"
      ],
      "metadata": {
        "id": "QzoSbWC678Zm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ssyIZOUJMrzM"
      },
      "outputs": [],
      "source": [
        "%cd {TRAINING_DATA_DIR}\n",
        "\n",
        "training_files = os.listdir()\n",
        "\n",
        "if len(training_files) > 0:\n",
        "  for training_file in training_files:\n",
        "    if training_file.endswith('.tsv'):\n",
        "\n",
        "      data = training_file.split('_')[0]\n",
        "      if not os.path.isdir(os.path.join(FORMATTED_TRAIN_DATA, data)):\n",
        "        print(data, 'Creating directory to store train, dev and test splits.')\n",
        "        os.mkdir(os.path.join(FORMATTED_TRAIN_DATA, data))\n",
        "      \n",
        "      df = pd.read_csv(training_file, sep='\\t', names=['ID', 'text', 'label'], header=0)\n",
        "      df[['text', 'label']].to_csv(os.path.join(FORMATTED_TRAIN_DATA, data, 'train.tsv'), sep='\\t', index=False)\n",
        "    else:\n",
        "      print(training_file + ' skipped!')\n",
        "else:\n",
        "  print('No files are found in this directory!')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7S2Dup8GHl1Q"
      },
      "source": [
        "After running the code above, a new folder (called formated-train-data) with formated files is created in the \"datasets\" folder in the train sub-folder."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5LgeVN_wXGrq"
      },
      "source": [
        "##b. <font color='red'>`(Optional) Creating Evaluation (Dev and Test) sets from the available training data`</font>\n",
        "\n",
        "You may wish to create train and evaluation (dev and test) sets from the training data provided. If you wish to do so, you can run any of the cells below`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "APxVxL06lfux"
      },
      "source": [
        "###i. If you want to create both the Dev and Test sets, run this cell"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aVq1Blz0YF2b"
      },
      "outputs": [],
      "source": [
        "if os.path.isdir(FORMATTED_TRAIN_DATA):\n",
        "  print('Data directory found.')\n",
        "  SPLITTED_DATA = os.path.join(TRAINING_DATA_DIR, 'splitted-train-dev-test')\n",
        "  if not os.path.isdir(SPLITTED_DATA):\n",
        "    print('Creating directory to store train, dev and test splits.')\n",
        "    os.mkdir(SPLITTED_DATA)\n",
        "else:\n",
        "  print(FORMATTED_TRAIN_DATA + ' is not a valid directory or does not exist!')\n",
        "\n",
        "%cd {FORMATTED_TRAIN_DATA}\n",
        "formatted_training_files = os.listdir()\n",
        "\n",
        "if len(formatted_training_files) > 0:\n",
        "  for data_name in formatted_training_files:\n",
        "    formatted_training_file = os.path.join(data_name, 'train.tsv')\n",
        "    if os.path.isfile(formatted_training_file):\n",
        "      labeled_tweets = pd.read_csv(formatted_training_file, sep='\\t', names=['text', 'label'], header=0)\n",
        "      train, dev, test = np.split(labeled_tweets.sample(frac=1, random_state=42), [int(.7*len(labeled_tweets)), int(.8*len(labeled_tweets))])\n",
        "\n",
        "      if not os.path.isdir(os.path.join(SPLITTED_DATA, data_name)):\n",
        "        print(data_name, 'Creating directory to store train, dev and test splits.')\n",
        "        os.mkdir(os.path.join(SPLITTED_DATA, data_name))\n",
        "\n",
        "      train.sample(frac=1).to_csv(os.path.join(SPLITTED_DATA, data_name, 'train.tsv'), sep='\\t', index=False)\n",
        "      dev.sample(frac=1).to_csv(os.path.join(SPLITTED_DATA, data_name, 'dev.tsv'), sep='\\t', index=False)\n",
        "      test.sample(frac=1).to_csv(os.path.join(SPLITTED_DATA, data_name,'test.tsv'), sep='\\t', index=False)\n",
        "    else:\n",
        "      print(training_file + ' is not a supported file!')\n",
        "else:\n",
        "  print('No files are found in this directory!')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cZy_rzouJZFz"
      },
      "source": [
        "After running the code above, a new folder (called splitted-train-dev-test) with train-dev-test split is created in the \"datasets\" folder in the train sub-folder. Here, the train-dev-test split is 70/10/20\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V4UObc2ql-Zd"
      },
      "source": [
        "###ii. If you want to create only the Dev set from the training data, please run this"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "tU24jW_gmFu9"
      },
      "outputs": [],
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "if os.path.isdir(FORMATTED_TRAIN_DATA):\n",
        "  print('Data directory found.')\n",
        "  SPLITTED_DATA = os.path.join(TRAINING_DATA_DIR, 'splitted-train-dev')\n",
        "  if not os.path.isdir(SPLITTED_DATA):\n",
        "    print('Creating directory to store train, dev and test splits.')\n",
        "    os.mkdir(SPLITTED_DATA)\n",
        "else:\n",
        "  print(FORMATTED_TRAIN_DATA + ' is not a valid directory or does not exist!')\n",
        "\n",
        "%cd {FORMATTED_TRAIN_DATA}\n",
        "formatted_training_files = os.listdir()\n",
        "\n",
        "if len(formatted_training_files) > 0:\n",
        "  for data_name in formatted_training_files:\n",
        "    formatted_training_file = os.path.join(data_name, 'train.tsv')\n",
        "    if os.path.isfile(formatted_training_file):\n",
        "      labeled_tweets = pd.read_csv(formatted_training_file, sep='\\t', names=['text', 'label'], header=0)\n",
        "      train, dev = train_test_split(labeled_tweets, test_size=0.3)\n",
        "\n",
        "      if not os.path.isdir(os.path.join(SPLITTED_DATA, data_name)):\n",
        "        print(data_name, 'Creating directory to store train, dev and test splits.')\n",
        "        os.mkdir(os.path.join(SPLITTED_DATA, data_name))\n",
        "\n",
        "      train.sample(frac=1).to_csv(os.path.join(SPLITTED_DATA, data_name, 'train.tsv'), sep='\\t', index=False)\n",
        "      dev.sample(frac=1).to_csv(os.path.join(SPLITTED_DATA, data_name, 'dev.tsv'), sep='\\t', index=False)\n",
        "    else:\n",
        "      print(training_file + ' is not a supported file!')\n",
        "else:\n",
        "  print('No files are found in this directory!')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "usfr00QhKSnd"
      },
      "source": [
        "After running the code above, a new folder (called splitted-train-dev) with train-dev split is created in the \"datasets\" folder in the train sub-folder. Here, the train-dev split is 70/30\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aDoyRlje3Rm7"
      },
      "source": [
        "#3) Training setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8AaXec415s0f"
      },
      "source": [
        "##a. Set project parameters\n",
        "\n",
        "For a list of models that be used for fine-tuning, you can check [HERE](https://huggingface.co/models)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "M0TKIFrE5ybV"
      },
      "outputs": [],
      "source": [
        "%cd {PROJECT_DIR}\n",
        "\n",
        "# Language to train sentiment classifier for\n",
        "LANGUAGE_CODE = 'ha'\n",
        "\n",
        "if LANGUAGE_CODE in languages:\n",
        "  # Model Training Parameters\n",
        "  MODEL_NAME_OR_PATH = 'Davlan/afro-xlmr-mini'\n",
        "  BATCH_SIZE = 32\n",
        "  LEARNING_RATE = 5e-5\n",
        "  NUMBER_OF_TRAINING_EPOCHS = 5\n",
        "  MAXIMUM_SEQUENCE_LENGTH = 128\n",
        "  SAVE_STEPS = -1\n",
        "\n",
        "  print('Everything set. You can now start model training.')\n",
        "\n",
        "else:\n",
        "  print(\"Invalid language code/Dataset not released. Please input any of the following released data\\n\\n\\t- 'am'\\n\\t- 'dz'\\n\\t- 'ha'\\n\\t- 'ig'\\n\\t- 'ma'\\n\\t- 'pcm'\\n\\t- 'pt'\\n\\t- 'sw'\\n\\t- 'yo'\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I2qcCQnU8dgQ"
      },
      "source": [
        "##b. Train the model\n",
        "\n",
        "In the section below, we provide three options: \n",
        "\n",
        "- 1) training model without any validation; \n",
        "- 2) training model with validation but without testing; \n",
        "- 3) training a model with validation and test set."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8fEw-qcEhYnx"
      },
      "source": [
        "###i. Training on only Train set, without any evaluation"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kAgg2MmAhiiW"
      },
      "outputs": [],
      "source": [
        "DATA_DIR = os.path.join(TRAINING_DATA_DIR, 'formatted-train-data', LANGUAGE_CODE)\n",
        "OUTPUT_DIR = os.path.join(PROJECT_DIR, 'models', LANGUAGE_CODE + '_no_eval')\n",
        "\n",
        "!CUDA_VISIBLE_DEVICES=0 python starter_kit/run_textclass.py \\\n",
        "  --model_name_or_path {MODEL_NAME_OR_PATH} \\\n",
        "  --data_dir {DATA_DIR} \\\n",
        "  --do_train \\\n",
        "  --per_device_train_batch_size {BATCH_SIZE} \\\n",
        "  --learning_rate {MAXIMUM_SEQUENCE_LENGTH} \\\n",
        "  --num_train_epochs {NUMBER_OF_TRAINING_EPOCHS} \\\n",
        "  --max_seq_length {MAXIMUM_SEQUENCE_LENGTH} \\\n",
        "  --output_dir {OUTPUT_DIR} \\\n",
        "  --save_steps {SAVE_STEPS}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wZtt5lBBLPz1"
      },
      "source": [
        "As you may observe, the training loss is very large. As a start, you can tune the training parameters and model to get a competitive result. \n",
        "\n",
        "You can observe also, there is no validation metrics (e.g., accuracy, loss etc) since we are only training without validtaion "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MpFSMaAzhS5Q"
      },
      "source": [
        "###ii. Training on only Train and Dev sets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3TMAD_iLhpgI"
      },
      "outputs": [],
      "source": [
        "DATA_DIR = os.path.join(TRAINING_DATA_DIR, 'splitted-train-dev', LANGUAGE_CODE)\n",
        "OUTPUT_DIR = os.path.join(PROJECT_DIR, 'models', LANGUAGE_CODE + '_no_test')\n",
        "\n",
        "!CUDA_VISIBLE_DEVICES=0 python starter_kit/run_textclass.py \\\n",
        "  --model_name_or_path {MODEL_NAME_OR_PATH} \\\n",
        "  --data_dir {DATA_DIR} \\\n",
        "  --do_train \\\n",
        "  --do_eval \\\n",
        "  --per_device_train_batch_size {BATCH_SIZE} \\\n",
        "  --learning_rate {MAXIMUM_SEQUENCE_LENGTH} \\\n",
        "  --num_train_epochs {NUMBER_OF_TRAINING_EPOCHS} \\\n",
        "  --max_seq_length {MAXIMUM_SEQUENCE_LENGTH} \\\n",
        "  --output_dir {OUTPUT_DIR} \\\n",
        "  --save_steps {SAVE_STEPS}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_3I2VJlCMJTN"
      },
      "source": [
        "Now, you can observe, there is evalidation metrics (e.g., accuracy, loss etc) since we are evaluating our model performance on the validation set we created from the \n",
        "training data.\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jW99ncOhhDEm"
      },
      "source": [
        "###iii. Training with Train, Dev and Test sets"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "zaVssq7a1uwk"
      },
      "outputs": [],
      "source": [
        "DATA_DIR = os.path.join(TRAINING_DATA_DIR, 'splitted-train-dev-test', LANGUAGE_CODE)\n",
        "OUTPUT_DIR = os.path.join(PROJECT_DIR, 'models', LANGUAGE_CODE)\n",
        "\n",
        "!CUDA_VISIBLE_DEVICES=0 python starter_kit/run_textclass.py \\\n",
        "  --model_name_or_path {MODEL_NAME_OR_PATH} \\\n",
        "  --data_dir {DATA_DIR} \\\n",
        "  --do_train \\\n",
        "  --do_eval \\\n",
        "  --do_predict \\\n",
        "  --per_device_train_batch_size {BATCH_SIZE} \\\n",
        "  --learning_rate {MAXIMUM_SEQUENCE_LENGTH} \\\n",
        "  --num_train_epochs {NUMBER_OF_TRAINING_EPOCHS} \\\n",
        "  --max_seq_length {MAXIMUM_SEQUENCE_LENGTH} \\\n",
        "  --output_dir {OUTPUT_DIR} \\\n",
        "  --save_steps {SAVE_STEPS}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2lLKHsrEPucP"
      },
      "source": [
        "Now that you trained your best model and find the best  parameters, you can submit your prediction on dev or test set on CodaLab competition page."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g9DeJD4CVmEl"
      },
      "source": [
        "#4) Submission\n",
        "\n",
        "- For submission after training your model, unlabeled tweets were provided for dev (development phase) and test (evaluation phase). \n",
        "\n",
        "- To generate their sentiment prediction, provide the path to the file containing the unlabeled tweets.\n",
        "\n",
        "**What the code does**\n",
        "1. Predicting sentiments of the unlabeled tweets (dev or test)\n",
        "2. Create a file in the submission format"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LYL3KcmoWXK1"
      },
      "outputs": [],
      "source": [
        "%cd {PROJECT_DIR}\n",
        "\n",
        "OUTPUT_DIR = os.path.join(PROJECT_DIR, 'models', LANGUAGE_CODE)\n",
        "FILE_NAME = os.path.join(PROJECT_DIR, TASK, 'dev', LANGUAGE_CODE + '_dev.tsv')\n",
        "TEXT_COLUMN = 'tweet'\n",
        "\n",
        "!python starter_kit/run_predict.py \\\n",
        "  --model_path {OUTPUT_DIR} \\\n",
        "  --file_name {FILE_NAME} \\\n",
        "  --text_column {TEXT_COLUMN} \\\n",
        "  --lang_code {LANGUAGE_CODE}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JpoGeFwSEfVt"
      },
      "source": [
        "- Congratulations. You now trained sentiment classifier and predict on the unlabelled tweets.\n",
        "\n",
        "- The prediction file (pred_{lang}.tsv) is in \"afrisenti-semval-2023\" folder under \"submissions\" folder. For example, since we trained hausa (ha), you will be able to see pred_ha.tsv file ready for submission. The submission file is in the format below:\n",
        "\n",
        "<center>\n",
        "\n",
        "|ID | label |\n",
        "|--- | --- |\n",
        "|hau_dev_00001| negative |\n",
        "|hau_dev_00002| positive |\n",
        "|... | ... |\n",
        "\n",
        "</center>\n",
        "\n",
        "- Inside the same folder, you will also see a file \"ha_predictions.tsv\" with the format below to see tweets with corresponding sentiment predictions. This file is not for submission.\n",
        "\n",
        "\n",
        "<center>\n",
        "\n",
        "|ID | text | label |\n",
        "|--- | --- | --- | \n",
        "|hau_dev_00001| @user Allah Miki albarkah 🙏🙏🙏 |  positive |\n",
        "|hau_dev_00002| @user Kidan ma zai dadi😂\t |  negative |\n",
        "|... | ... | ... |\n",
        "\n",
        "</center>\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "DObkW3ulM7yg"
      ],
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3.9.12",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.9.12"
    },
    "vscode": {
      "interpreter": {
        "hash": "d203a7fbe37afbb990fedfc21c321928443618f3d7b991e0237ff71906aa031f"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}