OptimalWordAnalysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c87eb7c3",
   "metadata": {},
   "source": [
    "# Notebook Summary\n",
    "This notebook will analyze the effectiveness of 3 separate policies to solving the Wordle game. We make certain assumptions when playing the game:\n",
    "1. We can only choose from the remaining set of words that are left after feedback is given from a guess.\n",
    "2. We lose the game if we have not made the correct guess in 6 tries\n",
    "\n",
    "There will be 3 policies analyzed:\n",
    "1. A simple similarity metric based on words and letter counts\n",
    "2. A similarity metric based on if letters match on their position\n",
    "3. A blend of both\n",
    "\n",
    "We assume that in each policy, we choose the word which is most similar to the other words. We posit that this strategy maximizes the likelihood of winning the game by gaining the most information. The reasoning is that if our word chosen is NOT the target word, we eliminate majority candidates, and thus give ourselves the best chance in the subsequent round.\n",
    "\n",
    "In addition to analyzing policies, we can perform a Monte Carlo simulation, by randomly picking a start word, and averaging over all paths. Although this could be computationally expensive, it is interesting to look at. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e2cedb9",
   "metadata": {},
   "source": [
    "## Import Solver Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "1c4a6516",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import pandas as pd\n",
    "with open(\"saved_models/results.json\",\"r\") as f:\n",
    "    results_dict = json.loads(f.read())\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "1fb2c07c",
   "metadata": {},
   "outputs": [],
   "source": [
    "rows = []\n",
    "for word in results_dict:\n",
    "    solver_results = results_dict[word]\n",
    "    for solver in solver_results:\n",
    "        solved = solver_results[solver][\"solved\"]\n",
    "        start_word = solver_results[solver][\"round_results\"][\"0\"]\n",
    "        guesses = len(solver_results[solver][\"round_results\"])\n",
    "        rows.append({\"target_word\":word,\"solver\":solver,\"solved\":solved,\"start_word\":start_word,\"num_guesses\":guesses})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "3088a96b",
   "metadata": {},
   "outputs": [],
   "source": [
    "data_df = pd.DataFrame(rows)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "df274329",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>solved</th>\n",
       "      <th>num_guesses</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>solver</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>lcw</th>\n",
       "      <td>0.923836</td>\n",
       "      <td>4.793555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mix</th>\n",
       "      <td>0.933010</td>\n",
       "      <td>4.826395</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>psw</th>\n",
       "      <td>0.930620</td>\n",
       "      <td>4.879356</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          solved  num_guesses\n",
       "solver                       \n",
       "lcw     0.923836     4.793555\n",
       "mix     0.933010     4.826395\n",
       "psw     0.930620     4.879356"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_df.groupby(\"solver\").mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "42d13122",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>solved</th>\n",
       "      <th>num_guesses</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>solver</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>lcw</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4.611649</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mix</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4.670330</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>psw</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4.721256</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        solved  num_guesses\n",
       "solver                     \n",
       "lcw        1.0     4.611649\n",
       "mix        1.0     4.670330\n",
       "psw        1.0     4.721256"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_df[data_df.solved == True].groupby(\"solver\").mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d46b9144",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}